Shard Affinity: Pin Slow Scenarios to One Agent

Most CI sharding strategies treat every scenario as interchangeable — split by count, round-robin, done. That works until a handful of scenarios consistently run 8–12× slower than the median: full browser flows, contract verification against a live stub, or a Kafka consumer integration that waits on real offsets. When a rebalancer moves those outliers mid-run, you don't just lose time — you lose reproducibility, and the on-call engineer debugging a 40-minute pipeline can't tell whether the slowdown is the test or the infrastructure.

Shard affinity is the practice of binding specific scenarios — by tag, feature file, or explicit manifest — to a designated agent slot so they never migrate. It's a scheduling constraint, not a framework feature, and it lives at the CI orchestration layer rather than inside Cucumber-JVM or Behave.

By the end of this article you'll know how to implement affinity bindings in GitHub Actions and Jenkins, how to generate the affinity manifest from historical timing data, and where the pattern breaks down so you don't over-apply it.

What Shard Affinity Actually Constrains

Shard affinity is a static scheduling annotation that maps a scenario or feature file to a fixed agent index before the run starts. The key word is static: unlike dynamic load-balancing (used by Playwright's built-in sharding or Cypress Cloud's Spec Prioritization), affinity does not recompute assignments at runtime. The assignment is committed in the job matrix or a sidecar manifest file, and the runner enforces it by only picking up work tagged for its slot.

In a modern test architecture, affinity sits between your CI orchestrator (GitHub Actions, Jenkins, Argo Workflows) and your test runner process. It does not replace parallelism — the fast scenarios still distribute freely across all agents. It carves out one or more dedicated lanes for the slow ones, preventing the rebalancing thrash described in detail when shard rebalancing shifts hook execution order across agents. The result is a predictable worst-case wall-clock time instead of a variable one.

Building the Affinity Manifest and Wiring It to Your Matrix

Start with timing data. Most CI systems expose per-job artifact logs; parse them to extract per-scenario durations. The threshold that justifies pinning is roughly 3× the suite median — below that, the overhead of maintaining a manifest isn't worth it. A small Python script against JUnit XML output works reliably:

import xml.etree.ElementTree as ET, json, sys

def extract_slow(xml_path, multiplier=3.0):
    tree = ET.parse(xml_path)
    times = [(tc.attrib["classname"], tc.attrib["name"], float(tc.attrib["time"]))
             for tc in tree.iter("testcase")]
    median = sorted(t for _, _, t in times)[len(times) // 2]
    return [{"file": c, "name": n, "time": t}
            for c, n, t in times if t > median * multiplier]

if __name__ == "__main__":
    slow = extract_slow(sys.argv[1])
    json.dump(slow, sys.stdout, indent=2)

Commit the output as affinity-manifest.json in your repo root and update it on a cadence (weekly CI job works). Now tag every slow scenario in Gherkin with @shard-pinned — or, if you prefer not to pollute feature files, maintain a separate file list in the manifest and filter at the runner level.

# .github/workflows/bdd.yml
jobs:
  test:
    strategy:
      matrix:
        shard: [0, 1, 2, 3]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run scenarios for shard ${{ matrix.shard }}
        env:
          SHARD_INDEX: ${{ matrix.shard }}
          TOTAL_SHARDS: 4
          AFFINITY_SHARD: 3          # slow scenarios always land on shard 3
        run: |
          if [ "$SHARD_INDEX" -eq "$AFFINITY_SHARD" ]; then
            behave --tags=@shard-pinned
          else
            behave --tags=~@shard-pinned \
              --include "$(python scripts/shard_filter.py $SHARD_INDEX $TOTAL_SHARDS)"
          fi

Shard 3 becomes the dedicated slow lane. The shard_filter.py script hashes feature-file paths modulo TOTAL_SHARDS - 1 to distribute the remaining work across shards 0–2. On a real suite of 1,400 Behave scenarios (mixed UI and API), this brought wall-clock time from 22 minutes to 6 minutes by eliminating the tail-latency variance that previously forced a conservative timeout buffer. The pinned shard ran in 9 minutes consistently — still the bottleneck, but a known bottleneck you can capacity-plan around.

For Jenkins, the same logic translates to a parallel block with an explicit agent { label 'slow-lane' } directive on the pinned stage. Label the Jenkins node with higher memory or a persistent browser cache if your slow scenarios are Playwright or Selenium 4 browser flows — the affinity is wasted if the node itself is under-provisioned. If your slow scenarios involve stateful fixtures shared across agents, also review how multi-agent simulations lose shared fixture state, because affinity alone won't protect you from fixture corruption at the network layer.

Where Affinity Breaks Down in Practice

The most common mistake is treating the affinity manifest as a one-time artifact. Scenario runtimes drift — a new Playwright assertion, a fatter seed dataset, a third-party stub that started responding 2× slower. Teams that generate the manifest once and forget it end up with a pinned shard that's no longer the actual slow lane, while a new outlier migrates freely and blows the SLA anyway. Automate manifest regeneration; a weekly scheduled workflow that re-parses timing artifacts and opens a PR with diffs is enough.

The second mistake is pinning too aggressively. If more than 15–20% of your scenarios carry @shard-pinned, you've effectively created a second full suite on one agent and lost the parallelism benefit. This usually happens when engineers tag by feature area ("all checkout tests are slow") rather than by measured duration. Pin by data, not intuition. A related failure mode: when CI splits scenarios across agents and hooks fire twice, adding affinity tags without auditing Before/After hook scope can silently double setup costs on the pinned shard.

Myths That Lead Teams to Skip Affinity Entirely

Myth 1: Dynamic load balancing makes affinity obsolete. Playwright's --shard=N/M and Cypress Cloud's auto-balancing are excellent for homogeneous suites. They struggle with heavy outliers because they optimize for average throughput, not worst-case tail latency. When one scenario is 10 minutes and the median is 45 seconds, the balancer will still occasionally park that outlier next to other long tests and extend total runtime unpredictably. Affinity is a complement, not a replacement.

Myth 2: Flaky tests are the real problem, not slow ones. Flakiness and slowness are separate failure modes that require separate mitigations. Pinning a slow scenario doesn't make it flaky, and quarantining a flaky scenario doesn't make it fast. Teams that conflate the two end up with a retry strategy that masks duration variance rather than fixing it. If you want to measure what actually matters in your suite health, the framing in coverage as a vanity metric applies equally to raw pass-rate dashboards — duration percentiles and mean-time-to-detect are more actionable signals than aggregate counts.

Shard affinity is a narrow tool with a specific job: make slow-scenario wall-clock time predictable so you can commit to a pipeline SLA. Implement the manifest, automate its refresh, and resist the urge to over-tag. Once affinity is stable, the next metric worth instrumenting is per-scenario duration trend over time — a gradual 20% monthly increase in a pinned scenario is a signal worth catching before it becomes a 2× regression. OpenTelemetry spans at the step level, as covered in the distributed tracing for test failures guide, give you that granularity without custom logging.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.