Jobs
Some operations that you perform in OrgFlow are executed instantly and synchronously when you click the button. This is mostly true for simple and quick operations that don't require significant processing, such as removing a member from your workspace.
Other operations take a longer time to complete, require more complex processing, and might involve interacting with your Git repository and Salesforce orgs. Such operations not executed immediately when you click the button, but instead are queued as jobs for asynchronous execution in the background.
Some examples of operations that are executed as jobs include:
- Initializing a new stack
- Creating and deleting environments
- Retrieving metadata (flow in)
- Deploying metadata (flow out)
- Moving metadata between environments (flow merge)
- Running Apex tests
Jobs can be monitored using the jobs page in the OrgFlow web app. This page shows currently running jobs, as well the full job history, in a single list that can be filtered on various job properties.
Data retention
Job history is subject to the data retention period included in your subscription. Jobs older than the data retention period are permanently deleted.
Additionally, the job history shown in the web app is currently limited to 14 days, with no possibility to load older jobs (other than accessing them directly by job URL). We are planning to add ways to load more history.
Execution models
Managed worker jobs
Jobs that are started through the OrgFlow web app are executed by managed workers, which are essentially container instances running on infrastructure that we provide (the OrgFlow cloud). When a job gets queued for execution, a new managed worker is created, assigned to process that job, and then deleted. Each managed worker is dedicated to a single job, and managed workers are never shared between workspaces.
Limits and billing
The number of managed worker jobs that can execute simultaneously is constrained by the number of concurrent workers included in your subscription. Starting additional concurrent jobs beyond this limit will keep those jobs in a pending state until concurrent workers become available.
Managed worker jobs are metered. Their execution time is categorized as follows:
- Active worker minutes are the total number of minutes that the job spent doing actual work
- Passive worker minutes are the total number of minutes that the job spent waiting for input from a user
All subscriptions include a certain number of concurrent workers and a certain amount of active and passive worker minutes for free. Worker minutes consumed beyond the free allowance are billed on a consumption basis (see our pricing page for details).
The amount of managed worker jobs you can run in a given billing period is constrained by the active and passive worker minutes per month limits configured in your workspace billing and usage settings. If either of those limits has been reached for the current billing period, starting additional jobs will fail. Unless you have provided payment details, those limits are hard-capped to the free allowance to ensure you don't incur any spending.
CLI jobs
Jobs can also be executed by CLI workers by running commands in the OrgFlow CLI. Jobs started this way are executed fully by the CLI itself, and all work is performed using whatever compute resources and infrastructure you're running the CLI on.
Because they run entirely on your infrastructure and don't consume OrgFlow-provided resources, CLI jobs are not metered and are not subject to the concurrent worker and worker minute constraints that apply to managed worker jobs.
Both managed worker jobs and CLI jobs can be monitored using the jobs page in the web app, though CLI jobs currently show less detailed progress information.
Job lifecycle
The following diagram illustrates the important events and states a job transitions through during its lifecycle:
For CLI jobs, the following apply:
- Pending and queued states are skipped; jobs are created directly in running state
- Active and passive worker minutes are not applicable
Inquiries
Jobs sometimes need input from a user before processing can continue. This is done using inquiries. When a job needs input, it initiates an inquiry and pauses execution while waiting for the user to respond.
OrgFlow conducts the following types of inquiries:
| Inquiry | Scenario |
|---|---|
| Device code authorization | User needs to sign into a Salesforce org and authorize OrgFlow to connect to it |
| Merge conflicts | User needs to resolve merge conflicts that could not be automatically resolved |
| Deployment confirmation | User needs to review and confirm the contents of a deployment |
Device code authorization inquiries are initiated whenever OrgFlow needs a new access token to connect to a connected Salesforce org (and no password has been saved).
Merge conflict inquiries and deployment confirmation inquiries can be enabled or disabled on a per-job basis using job options. Merge conflicts can occur in a variety of scenarios, including during metadata retrieve (flow in) and during merge between environments (flow merge). See the topic about handling merge conflicts for more information.
Inquiries are made visible on the job timeline on the job details page in the web app, where they can be examined and responded to by any user with sufficient permissions in the workspace. If no user responds to an inquiry within 10 minutes, and email is sent to the user who started the job. If no user responds to an inquiry within 18 hours, the inquiry times out and the job fails.
The managed worker remains running while awaiting the inquiry. The time spent awaiting an inquiry is categorized as passive worker minutes.
Inquiries in CLI jobs
CLI jobs currently do not conduct inquiries through the web app. Instead the CLI uses the current shell to prompt for any input it requires. This can be impractical in some CI/CD scenarios; we are planning to add the ability for CLI jobs to conduct inquiries through the web app
Artifacts
Most jobs produce output in the form of artifacts. Artifacts are essentially files collected by the worker during the processing of a job, stored in our infrastructure, and made available for download to users.
OrgFlow collects the following types of artifacts:
| Artifact | Description | Managed | CLI |
|---|---|---|---|
| Output | Detailed results in JSON format; structure and contents are specific per job type | Yes | Yes |
| Log | Log file in text format for diagnostic purposes | Yes | Yes |
| Retrieved metadata | ZIP files for every batch of metadata as retrieved from Salesforce | Opt-in | No |
| Deployment archive | Deployment ZIP files for every attempted deployment | Opt-out | Yes |
As seen in the above table, the CLI does not support collecting retrieved metadata artifacts. Instead the CLI provides arguments to save these items to the local file system on the machine where the CLI is executing.
Artifacts can be downloaded from the job details page in the web app.
Data retention
Artifacts are stored in the OrgFlow cloud and are subject to the data retention period included in your subscription. Artifacts older than the data retention period are permanently deleted.
Dependencies
Jobs can have dependencies on other jobs. One scenario where this is used, is when you start a flow merge job from one source environment into multiple target environments, and the job includes a flow in (retrieve) of the source environment.
Let's say for example that you start a flow merge from environment Production into environments Dev1 and Dev2 and you configure the job to include a flow in (retrieve) of Production. When you submit this request, OrgFlow actually expands it into three separate jobs:
- Flow in (retrieve) of environment Production
- Flow merge from Production into Dev1 (depends on #1)
- Flow merge from Production into Dev2 (depends on #1)
This is done for efficiency. The flow in of Production only needs to happen once, so it can be executed to completion first, and its resulting commits in Git can then be used as the starting point by both other jobs. The flow merge into Dev1 and Dev2 must await the completion of the first job, but once done, these two jobs can then execute concurrently.
Time limits
Managed worker jobs are subject to the following time limits:
- If a job has spent more than 6 hours in pending state, it is timed out
- If a job has spent more than 24 hours in some combination of states other than ended, it is timed out
- If a job has spent more than 18 hours awaiting a single inquiry, it is failed