Apex test runs

OrgFlow can execute the Apex tests in your Salesforce orgs. This can be either as part of a deployment, or as part of an ad-hoc execution of tests. In both cases, OrgFlow will create an Apex test run.

Starting Apex test runs

Job types that can start Apex test runs are:

• Flow out
• Flow merge
• Run Apex tests

Flow out and flow merge will only start Apex test runs if Apex tests are executed as part of the deployment to Salesforce.

To run Apex tests outside the context of a deployment, you can use the Run apex tests job type. This job can be started from either the Environments tab, or from the Test runs tab. This job type allows you control over which test classes or suites to run, as well as some other options such as stopping the Apex test run once a specific number of tests have failed.

Apex test runs (that were started as part of an Apex test run job) can be re-run, either from the list of historical Apex test runs, or from the Apex test run details page. This is handy if you are working on an area of functionality and want to frequently re-run the tests that cover that area of functionality.

On a recurring schedule

A common scenario is to run Apex tests on a nightly (or other frequent) basis. This allows you to catch failing tests early so that the functionality under test can be fixed sooner - either before the failing tests block a deployment, or before your users start to notice issues with the functionality under test.

This can be achieved with schedules, by creating a schedule that executes the Run apex tests job type.

With the CLI

Jobs started with the OrgFlow CLI will also create Apex test runs that can be viewed in the web app's UI.

Optionally, the OrgFlow CLI can also produce a jUnit results file. This file can be read by other CI/CD platforms (such as GitHub, GitLab, and Azure DevOps) to allow them to display the test results in their own UI.

Viewing Apex test runs

A summary of the Apex test run is shown on the job details page whenever a job starts an Apex test run. The Apex test run is continuously updated as the tests are executed in Salesforce. This allows you to monitor the progress of the test run, and view any failures as soon as they happen. The Apex test run summary also includes a link to the Apex test run details page, where you can view the Apex test run in more details.

You can also view a history of previous Apex test runs from the Test runs tab. This list of historical Apex test runs are filterable, so you can easily find the Apex test runs against a specific environment, or with a specific outcome.

Apex test runs are subject to your data retention period

Your subscription dictates the data retention period available to you. Apex test runs that are older than the maximum age allowed by your data retention period are automatically deleted by OrgFlow. See billing for more information.

You can click each Apex test run in the list of historical runs to view it in more detail. Just like the Apex test run summary on the job detail pages, the Apex test run details pages are updates in real time as Salesforces executes each of your tests:

Limitations

The data contained within an Apex test run is collected from Salesforce by OrgFlow and then standardized in order to produce the output that you can see. This means that limitations in the data that Salesforce provides will affect the data in Apex test runs. There are currently two known limitations with this data:

The count of enqueued tests may be higher than the sum of the passed, skipped, and failed tests. Salesforce doesn't include test set up methods in the count of enqueued tests, but it does include them in the list of test results, which causes this issue.

The list of test results may be incomplete while an Apex test run is running during a deployment. When Apex tests are executed as part of a deployment, Salesforce reports the failing tests as and when they fail, but doesn't report on passed and skipped tests as they finish. Salesforce will however report on the overall count of tests executed as the deployment test run progresses. This means that you will be able to see the count of passed and skipped tests increase while the Apex test run is running, but you won't be able to see the passed or skipped tests appear in the list of executed tests until the Apex test run is finished. Failed tests will appear in this list as they fail, and passed and skipped tests all appear in the list once the Apex test run is finished. This issue does not affect Apex test runs executed outside the context of a deployment.