Skip to content
compiler.dev
Start free

What Is Test Sharding in CI?

Last updated

Test sharding is splitting one test suite into several slices, called shards, and running each slice in parallel on its own CI machine so the whole suite finishes sooner.

How it works

You choose a shard count, say 4. Each job receives its index (1 to 4) and the total, and runs only the tests assigned to it. When all shards pass, the suite passes. If tests are evenly sized, wall-clock time falls to roughly a quarter, minus the fixed setup cost every shard pays (checkout, install, build).

Most test runners support this natively: Playwright has --shard=1/4, Jest has --shard=1/4, Vitest has --shard=1/4, and pytest uses plugins such as pytest-split or pytest-xdist for in-machine parallelism. A matrix build is the usual way to launch the shards.

Example

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx playwright test --shard=${{ matrix.shard }}/4

Splitting strategies

  • By file count: simple, but uneven if some files are slow.
  • By historical timing: balances shards using previous run durations. Best results.
  • By test count: what many built-in --shard flags do. Fine when tests are similar in length.

Use the test shard calculator to estimate the best shard count, and read test sharding in GitHub Actions for a full walkthrough.

Common mistakes

  • Too many shards. Setup overhead is paid per shard. Past a point, more shards cost more minutes and save no time.
  • Shared database state. Shards running in parallel against one database collide. Give each shard its own database or schema.
  • Unbalanced shards. One slow shard sets the total time. Check durations per shard.
  • Hiding flakiness. Parallel runs expose order dependence. See flaky tests.
  • Not merging reports. Each shard produces partial coverage or reports; merge them in a final job.

Sharding trades more total CI minutes for less waiting. Faster machines reduce the per-shard cost too; compiler.dev's benchmarks use 10 shards, see /benchmarks.

Related terms

FAQ

How many shards should I use?

Start where each shard runs for 2 to 5 minutes. Below that, setup overhead dominates.

Is sharding the same as parallelism?

Sharding spreads tests across machines. Parallelism within one machine uses multiple workers or CPUs. Use both.

Does sharding reduce cost?

Usually not. Total compute time stays similar or grows slightly. It reduces how long developers wait.

Related