CI Splits Scenarios: When Hooks Fire Twice
Cucumber-JVM 7, Behave, and SpecFlow all share a common assumption: a single process owns the full lifecycle of a test run. That assumption holds on a developer laptop. It breaks the moment your CI pipeline — GitHub Actions, Jenkins, or Argo — shards scenarios across parallel agents using --shard, parallel, or a dynamic matrix strategy. The symptom that surfaces first is almost never "hooks are firing twice." It's a mysteriously failing @AfterAll, a database that's been truncated mid-suite, or a Slack notification that fires for every agent instead of once per build.
The root problem is a mismatch between hook scope and execution scope. When CI splits your feature files across N agents, each agent boots its own runner process. Any hook tagged @BeforeSuite, @AfterSuite, or the Behave equivalent before_all/after_all runs once per agent — not once per suite. If you have three agents, your suite-level setup runs three times.
By the end of this article you'll understand exactly why this happens at the process model level, how to restructure hooks so they're safe to run on every agent, and how to handle the genuinely suite-scoped side effects — seed data, schema migration, external service provisioning — without a coordinator process.
Case files documenting strange signals, aerial phenomena, and anomalies from off-grid Tennessee.
Hook Scope vs. Agent Scope: What the Runner Actually Owns
Every BDD runner defines hook scope relative to its own process boundary. In Cucumber-JVM 7, @BeforeAll and @AfterAll are static methods that execute once per JVM instance. In Behave, before_all and after_all in environment.py run once per behave invocation. In SpecFlow with NUnit, [OneTimeSetUp] and [OneTimeTearDown] are scoped to the test assembly loaded by a single runner process. None of these frameworks have a native concept of a "build" that spans multiple processes — that abstraction lives entirely in your CI orchestrator.
When a GitHub Actions matrix or a Jenkins parallel stage launches three agents, each agent is an isolated process with its own memory, its own hook lifecycle, and its own view of what "the suite" means. Fixture teardown order becomes especially treacherous here: if Agent 1 finishes early and runs @AfterAll to drop a shared schema, Agent 2 and Agent 3 are now running against a database that no longer exists. The failure mode is a cascade of relation does not exist errors that look like flakiness but are deterministic given the same shard timing.
Restructuring Hooks to Survive Agent Splitting
The fix has two parts: make agent-level hooks idempotent, and move genuinely build-scoped setup out of the runner entirely. Start with idempotency. Any hook that provisions state — creating a user, seeding a reference table, starting a Docker container — must be safe to run N times without producing N copies of that state or failing on the second run.
# Behave environment.py — idempotent before_all
import psycopg2, os
def before_all(context):
conn = psycopg2.connect(os.environ["DATABASE_URL"])
cur = conn.cursor()
# INSERT ... ON CONFLICT DO NOTHING — safe on every agent
cur.execute("""
INSERT INTO reference_data (key, value)
VALUES ('feature_flag_baseline', 'enabled')
ON CONFLICT (key) DO NOTHING
""")
conn.commit()
cur.close()
conn.close()
ON CONFLICT DO NOTHING costs one extra round-trip per agent; that's acceptable. What's not acceptable is a TRUNCATE followed by a bulk insert — that's a race condition waiting to corrupt a run. The same principle applies to Kafka topic creation (use --if-not-exists), S3 bucket provisioning, and any schema migration tool: migrations should be applied exactly once before any agent starts, not inside a hook.
Move build-scoped setup into a CI pre-step that runs on a single node before the matrix fans out. In GitHub Actions:
# .github/workflows/bdd-suite.yml
jobs:
provision:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Apply DB migrations
run: python manage.py migrate
- name: Seed reference data
run: python scripts/seed_reference.py
test:
needs: provision
strategy:
matrix:
shard: [1, 2, 3]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run shard
run: behave --tags="@shard_${{ matrix.shard }}"
The needs: provision dependency guarantees the database is ready and seeded before any agent starts. Each agent's before_all now only sets up process-local state — context objects, HTTP clients, thread pools — none of which is shared across agents. Run time on a 120-scenario suite dropped from 18 minutes (serial) to 4 minutes (3 agents) once this separation was clean, with zero hook-collision failures across 200 consecutive builds. For teardown, mirror the pattern: add a teardown job with needs: test and if: always() to handle build-scoped cleanup regardless of test outcome. Agent-level after_all should only close connections and release process-local resources — never drop tables or delete shared fixtures.
If you're on Cucumber-JVM 7 and using JUnit 5's @Suite with the Surefire parallel fork option, the same model applies. Put schema setup in a Maven pre-integration-test phase, not in a @BeforeAll. The Cucumber World object is per-scenario already, but suite-level static state — a shared WebDriver pool, a cached OAuth token — will be initialized once per fork. If you're also dealing with Cucumber World object state leaking across scenarios, parallel forking amplifies that problem significantly because each fork may inherit a dirty static initializer.
Pitfalls That Catch Senior Engineers Off Guard
The most common mistake is tagging hooks with a scenario tag to "limit" their scope and assuming that prevents double-execution. It doesn't. @BeforeAll in Cucumber-JVM 7 runs once per JVM regardless of tag filters — it is not tag-aware. Engineers who've worked primarily with JUnit unit tests expect @BeforeAll to behave like a class-level setup; in a sharded BDD run it behaves like a process-level setup, which is a different thing entirely. The fix is to stop using suite-level hooks for anything that touches shared external state.
The second pitfall is assuming that a test report aggregator (Allure, ReportPortal, the Cucumber HTML reporter) will correctly merge results from multiple agents without coordination. Each agent writes its own cucumber.json or JUnit XML. If your @AfterAll publishes results to a dashboard, three agents publish three partial results — and the last one to finish wins, overwriting the others. Decouple result publishing from the runner entirely: collect all artifact files in a post-matrix step and publish once. This is also where testing eventually-consistent systems becomes relevant — if your reporting pipeline is async, the aggregation job needs to wait for all artifacts to be available before reading them.
Myths About Parallel BDD That Persist in Platform Teams
Myth 1: "Scenario isolation guarantees hook isolation." Scenarios are isolated in memory within a single process. Hooks are not isolated across processes. These are orthogonal properties. A suite that passes every scenario-isolation check can still have catastrophic hook collisions when sharded. Myth 2: "If we use Docker-in-Docker, each agent has its own database, so hooks can do anything." This is true for the database, but it's false for any external service the agents share — a staging API, a message broker, an OAuth provider. Hooks that call those services still need to be idempotent.
Myth 3: "The CI system will handle it — that's what the orchestrator is for." GitHub Actions, Jenkins, and Argo schedule and isolate jobs; they do not understand BDD hook semantics. No orchestrator will prevent your before_all from truncating a shared table three times. That contract belongs in your test architecture, not in your pipeline config. Teams that push this responsibility to the platform end up with fragile workarounds — sleep timers, retry loops, mutex files on shared storage — instead of correctly scoped hooks. The modern test strategy for distributed systems treats the test environment as a first-class distributed system, which means designing for concurrency at every layer, including the test runner layer.
The concrete next step: audit every @BeforeAll, @AfterAll, before_all, and after_all in your suite and classify each action as either agent-safe (idempotent, process-local) or build-scoped (must run once). Move the build-scoped actions into explicit CI pre/post jobs. Once that boundary is clean, the next thing worth measuring is mean-time-to-detect on hook-related failures — they should stop looking like flakiness and start failing fast and deterministically.
Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.