Cucumber-JVM has shipped a major version roughly every 18 months for the last decade. Most teams still run scenarios whose step definitions were written against a UI flow, an API contract, or a domain model that no longer exists in the same shape. The tests stay green because the steps still execute — they just no longer verify what the feature file claims they verify.

The technical problem is semantic drift: the Gherkin scenario describes business intent at a point in time, but the step definition underneath it silently absorbs refactors, schema changes, and UI restructures until the two are only loosely coupled. No linting tool catches this. CI stays green. The scenario title still reads like documentation. It isn't.

By the end of this article you'll be able to identify the structural conditions that cause drift, instrument your step layer to surface it early, and apply a review heuristic that keeps your step definitions honest without rewriting your entire suite.

Semantic Drift: What It Is and Where It Lives in Your Stack

Semantic drift is the gradual divergence between what a Gherkin scenario asserts in plain language and what the underlying step definition actually exercises at runtime. It is distinct from a broken test (which fails) and from a flaky test (which fails intermittently). A drifted test passes reliably — it just no longer tests the thing it says it tests. The scenario becomes a liability masquerading as coverage.

In a modern test architecture — where BDD scenarios sit above a service layer, a Playwright or Selenium 4 browser driver, and a set of API clients — drift accumulates at every seam. A step like When the user submits the order might have started life calling a single POST endpoint. After three quarters of feature work it may now orchestrate a GraphQL mutation, a Kafka event assertion, and a Pulsar topic poll, none of which appear in the scenario text. The scenario still reads as a one-liner. The implementation is a distributed transaction. That gap is where production surprises live.

Instrumenting the Step Layer to Surface Drift Before It Ships

The most effective early-warning system is a step-definition changelog enforced in CI. Every step definition file carries a # @last-reviewed annotation. A GitHub Actions job diffs the annotation date against the git log of the corresponding feature file; if the feature was modified more recently than the step was reviewed, the build emits a warning annotation. This costs about 30 lines of Python and catches the most common source of drift: a product team updates the scenario copy to reflect a new flow, but nobody touches the step implementation.

# .github/workflows/bdd-drift-check.yml
name: BDD Drift Check
on: [pull_request]

jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Check step review dates
        run: python scripts/check_step_drift.py --features features/ --steps steps/

# scripts/check_step_drift.py  (Python 3.11+)
import subprocess, re, sys
from pathlib import Path

ANNOTATION = re.compile(r"#\s*@last-reviewed:\s*(\d{4}-\d{2}-\d{2})")

def last_git_change(path: Path) -> str:
    result = subprocess.run(
        ["git", "log", "-1", "--format=%cs", str(path)],
        capture_output=True, text=True
    )
    return result.stdout.strip()

def reviewed_date(step_file: Path) -> str | None:
    for line in step_file.read_text().splitlines():
        m = ANNOTATION.search(line)
        if m:
            return m.group(1)
    return None

drift_found = False
for step_file in Path("steps").rglob("*.py"):
    reviewed = reviewed_date(step_file)
    if not reviewed:
        print(f"::warning file={step_file}::No @last-reviewed annotation")
        continue
    feature = Path("features") / step_file.with_suffix(".feature").name
    if feature.exists() and last_git_change(feature) > reviewed:
        print(f"::warning file={step_file}::Feature updated after step review ({reviewed})")
        drift_found = True

sys.exit(1 if drift_found else 0)

Pairing this with a contract snapshot at the step level catches a second class of drift: API or schema changes that don't touch the feature file at all. In Behave or Pytest-BDD, you can record the JSON schema of every HTTP response your step definitions touch, commit those snapshots, and fail the build when the live schema diverges. Teams that adopted this pattern against a microservices backend reduced mean-time-to-detect contract regressions from 3 days (found in staging) to under 40 minutes (caught in the PR pipeline). Pact is the canonical tool here for consumer-driven contracts; the snapshot approach is lighter-weight when you don't control both sides of the API.

For UI-heavy suites on Playwright (v1.44+) or Cypress 13, drift often hides in locator strategies. A step written against a data-testid attribute silently survives a front-end refactor that moves the element into a shadow DOM or a React portal — Playwright's auto-waiting masks the structural change. The fix is to log the full resolved locator path to OpenTelemetry spans during test execution and alert when the resolved path changes between runs. Grafana dashboards on those spans give you a visual history of locator stability without requiring any test-framework changes. This is related to the broader problem of self-healing test mechanisms — understanding what they actually repair (locators) versus what they silently ignore (semantic drift) is critical before you adopt one.

Finally, apply a scenario-to-implementation ratio review during sprint retrospectives. If a single Gherkin step maps to more than ~50 lines of step-definition code, that step is almost certainly doing too much and has accumulated behaviour the scenario text doesn't describe. Refactor toward thinner steps and explicit sub-steps rather than hiding complexity in helper methods. For guidance on structuring a pipeline that enforces this from the start, the walkthrough in building a real Gherkin-to-code pipeline covers the scaffolding decisions that prevent over-stuffed step definitions from forming in the first place.

Three Ways Senior Engineers Accelerate Drift Without Realising It

The first is step reuse across unrelated features. Generic steps like Given I am logged in as a {role} get shared across 40 scenarios because they save keystrokes. When authentication changes — OAuth 2.0 PKCE replaces a session cookie, MFA is added — the step gets patched in place rather than versioned. Every scenario that reuses it now tests the new auth flow, even if the scenario was written to test something that predates it. The org-level cause is a shared step library treated as a utility package with no ownership model.

The second is Page Object or Screenplay layer churn without feature-file review. Engineers refactor the Page Object Model or migrate to the Screenplay pattern (a worthwhile move for complex domains) and update the step definitions to call the new abstractions — but nobody re-reads the feature files to verify the scenario text still matches what the new implementation does. The third is relying on Cucumber's --dry-run or SpecFlow's step binding validation as a correctness signal. These tools confirm that steps are matched, not that they're correct. Passing dry-run is a syntax check, not a semantic one.

Two Myths That Let Drift Compound Undetected

Myth 1: Green CI means the BDD layer is healthy. CI validates execution, not intent. A drifted scenario that passes every run is indistinguishable from a correct one in a standard pipeline report. Teams conflate test-pass rate with scenario accuracy, which is why drift compounds quietly for months. The correction is to treat scenario accuracy as a separate quality dimension — one you measure through periodic human review and contract snapshot diffing, not through pass/fail counts. This is especially acute when AI-generated step definitions are in the mix, where the divergence between generated code and actual domain behaviour can be invisible to reviewers who didn't write the original feature.

Myth 2: BDD scenarios are living documentation, so they stay current automatically. They don't. Living documentation is a goal, not a property. Gherkin is plain text; it doesn't auto-update when a product manager changes a flow in Confluence or a developer renames a domain concept. The "living" part requires an active process — at minimum, a rule that no feature file survives a sprint without a step-definition review if the underlying system changed. Without that rule, your feature files become archaeological artifacts: accurate descriptions of how the system worked when someone cared enough to write them down.

Semantic drift is a process failure dressed up as a testing problem. The instrumentation described here — changelog annotations, contract snapshots, locator telemetry — surfaces it early, but none of it substitutes for a team norm that treats step definitions as production code with an owner. If you implement the drift-check pipeline, the next metric worth tracking is the step-to-scenario churn ratio over time: a rising ratio signals that your step layer is absorbing complexity your scenarios aren't acknowledging, which is exactly where the next production surprise will come from.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.