Test Sharding Made Easy with Parallelism

Changelog
1

Parallelism is a powerful new feature that enables you to split the execution of a single workflow over many parallel runners in a single instruction. Let’s take a look at some use cases to help you understand what it is and how it can help you.

Test sharding

Test sharding is a great strategy for reducing wall clock time. But up until now, implementing test sharding on Bitrise involved repeating the same boilerplate code over and over again, something like this:

pipelines:
 my-pipeline:
   workflows:
    workflow1: {}
    workflow2: {}
    workflow3: {}
    
workflows:
 test_workflow: 
    { implementation goes here }
 workflow1
  before_run:
  - run_tests
  envs:
   - test_shard_index: 1
   - test_shard_count: 3
workflow2
  before_run:
  - run_tests
  envs:
   - test_shard_index: 2
   - test_shard_count: 3
workflow3
  before_run:
  - run_tests
  envs:
   - test_shard_index: 3
   - test_shard_count: 3

There’s a lot of repeat code to manage here, which slows down debugging and CI config maintenance. Furthermore, changing the number of shards is an arduous process.

The new Parallelism feature brings you the parallel keyword, which allows you to simplify the above configuration to this:

pipelines:
 my-pipeline:
  workflows:
   run_tests:
    parallel: 3

workflows:
  run_tests:
    { implementation goes heres }

When my-pipeline is triggered, 3 copies of run_tests will execute (named run_tests_0, run_tests_1 and run_tests_2). Each copy will receive two new environment variables:

  • $BITRISE_IO_PARALLEL_INDEX: a zero based index for each workflow (in this case it will run from 0 to 2)
  • $BITRISE_IO_PARALLEL_TOTAL: this will be 3 for all the run_tests_* workflows

When you view the build graph, this is what you’ll see:

You can leverage these environment variables to run test shards. The exact technique might depend on your testing framework, with yarn it might look like this:

...
workflows:
 ...
 run-tests:
  steps:
   ... // cloning, setup etc
   - script@1.2.1:
      inputs:
      - content: |-
         yarn test --ci --silent --shard=$((BITRISE_IO_PARALLEL_INDEX + 1))/$BITRISE_IO_PARALLEL_TOTAL
   ...

The parallel configuration is also available in the Workflow Editor UI as “Parallel copies” in the “Pipeline Conditions” section:

Dynamic capability

The parallel value can also be specified as an environment variable unlocking powerful dynamic capabilities. For example you can configure your pull requests and nightly regression builds to use the same pipeline, employing 6 shards overnight and just 2 shards for PRs - thereby preserving runner availability during the working day and giving you a single pipeline to maintain.

pipelines:
 my-pipeline:
  workflows:
   run_tests:
    parallel: $NUMBER_OF_SHARDS

White labels

While test sharding is the primary use case for Parallelism it can also be used in other situations where you want to reuse a workflow simultaneously across many runners e.g. a white label organisation building 50 customer app variants from one codebase, for example:

pipelines:
 my-ci-pipeline:
  workflows:
   build-and-test:
    parallel: 50
workflows:
 build-and-test: // implementation goes here
                 // use $BITRISE_IO_PARALLEL_INDEX
                 // to reference a config file

You can find more information in our DevCenter. If you have any questions, feel free to comment below. Happy building!