LLMs write Gherkin and step definitions fast. Feed Claude or ChatGPT a user story and you'll have a .feature file in under a minute. What you won't get — unless you explicitly engineer for it — is scenario isolation. The model has no concept of test state lifecycle. It produces steps that look correct in isolation but share implicit context across scenarios in ways that are genuinely hard to detect until your suite starts producing false positives at 2 a.m.

The specific failure mode is context bleed: one scenario mutates shared state (a singleton service client, a module-level fixture, a browser session, a database row) and the next scenario silently inherits it. The steps pass individually in authoring context but fail non-deterministically in suite order. This is a structural problem in how LLMs generate step code, not a fluke.

By the end of this article you'll be able to identify the three most common bleed patterns in AI-generated steps, instrument your suite to surface them deterministically, and apply fixture-scoping rules that prevent the model from reintroducing them on the next generation pass.

Why AI Step Generators Produce Stateful Code by Default

When an LLM generates step definitions, it optimises for coherence within the prompt window — the current feature file and whatever system prompt you've given it. It does not reason about test execution order, fixture teardown, or the Behave/Cucumber-JVM/SpecFlow scoping model. The result is steps that initialise clients, browsers, or DB connections at module or class scope rather than scenario scope, because that's the pattern most prevalent in the training corpus (tutorials, sample repos, Stack Overflow answers).

In a well-maintained suite, this is caught in code review. In an AI-assisted workflow where the generation volume is high — which is exactly the context where teams adopt LLM tooling — review bandwidth doesn't scale with output volume. The bleed accumulates silently. It's structurally identical to the problem that background steps silently corrupt isolation when they set up state that isn't fully torn down, except here the corruption is in the step implementation layer, not the feature file layer, making it harder to grep for.

Detecting and Fixing the Three Bleed Patterns in Testing AI-Generated Steps

The three patterns appear consistently across Behave, Cucumber-JVM 7, and SpecFlow 6. Each has a detectable signature and a fix that can be encoded back into your LLM system prompt so the model stops regenerating the problem.

Pattern 1: Module-level client initialisation

The model emits a database or API client at import time. Every scenario in the module shares the same connection, including any transaction state or auth token set by a previous scenario.

# AI-generated — dangerous
import psycopg2
conn = psycopg2.connect(DSN)  # module scope, shared across all scenarios

@given("a clean accounts table")
def step_clean_accounts(context):
    conn.execute("DELETE FROM accounts")  # uses shared conn

# Fixed — scenario-scoped via Behave fixture
# environment.py
from behave import fixture, use_fixture
import psycopg2

@fixture
def db_connection(context):
    context.conn = psycopg2.connect(DSN)
    context.conn.autocommit = False
    yield context.conn
    context.conn.rollback()
    context.conn.close()

def before_scenario(context, scenario):
    use_fixture(db_connection, context)

The rollback on teardown is the critical line. Without it, a scenario that inserts a row leaves it for the next scenario's SELECT count — a classic false pass that only surfaces when you run scenarios in a different order.

Pattern 2: Browser session reuse across Playwright or Selenium 4 scenarios

LLMs consistently generate a single browser or driver object at the feature or class level. Cookies, localStorage, and authenticated sessions persist between scenarios.

# pytest-bdd + Playwright — AI-generated, broken
@pytest.fixture(scope="module")
def browser():
    with sync_playwright() as p:
        b = p.chromium.launch()
        yield b
        b.close()

# Fixed — scenario scope, fresh context per scenario
@pytest.fixture(scope="function")
def browser_context():
    with sync_playwright() as p:
        browser = p.chromium.launch()
        ctx = browser.new_context()
        yield ctx
        ctx.close()
        browser.close()

Switching from scope="module" to scope="function" in pytest-bdd increased our suite run time from 4 minutes to 11 minutes on a 200-scenario suite — a real cost. The trade-off is deterministic isolation. If that's unacceptable, run scenarios in parallel with pytest-xdist and worker-level browser pools; don't compromise scope.

Pattern 3: Shared step-definition context object mutation

In Cucumber-JVM 7, AI-generated step classes frequently use Spring or Guice injection at singleton scope. One step sets a field; a later scenario's step reads a stale value.

// Cucumber-JVM 7 — broken scope
@Component
public class OrderSteps {
    private Order lastOrder; // singleton — bleeds between scenarios

    @When("a user places an order for {string}")
    public void placeOrder(String item) {
        lastOrder = orderService.create(item);
    }

    @Then("the order total should be {double}")
    public void verifyTotal(double expected) {
        assertEquals(expected, lastOrder.getTotal()); // could be previous scenario's order
    }
}

Fix this by scoping the step class to @ScenarioScope (Cucumber-JVM's PicoContainer or Spring integration both support it). Every scenario gets a fresh instance; no field survives teardown. This is the single most effective change you can make when auditing AI-generated step code at scale.

Encoding the fix into your generation prompt

Add a constraint block to your LLM system prompt. This is part of a context-driven LLM testing build that enforces structural rules at generation time rather than review time:

SYSTEM:
You are generating Behave step definitions.
RULES (never violate):
1. Never initialise DB connections, HTTP clients, or browser objects at module scope.
2. All shared state must be attached to `context` inside a `before_scenario` hook or a @fixture.
3. Every resource acquired in a step must be released in an `after_scenario` hook.
4. Do not use global or class-level variables to pass data between steps.

This doesn't eliminate the problem entirely — the model will occasionally violate rule 4 when generating complex multi-step flows — but it reduces the bleed rate to something a targeted linting pass can catch. A grep -rn "^conn\s*=\|^driver\s*=\|^client\s*=" across your steps directory surfaces most remaining violations in under a second.

Where Senior Engineers Still Get Burned During Testing LLM-Generated Code

The first mistake is trusting green CI as proof of isolation. A suite that always runs in the same order — as most CI pipelines do — will pass even with severe bleed because the dependency direction is consistent. The fix is to randomise scenario execution order at least once per day. In Behave, there's no built-in shuffle; wrap your runner with a small script that reorders the feature list. In pytest-bdd, pytest-randomly handles it. A suite that was green for three weeks at a team we audited produced 23 failures on first randomised run — all bleed-related.

The second mistake is treating AI step generation as a one-time bootstrap. Teams generate steps, fix the obvious issues, then re-run generation when requirements change — and the model regenerates the same anti-patterns into files that were previously clean. You need a linting gate in CI (Flake8 with a custom AST rule, or a simple grep) that rejects module-scope resource initialisation regardless of how the file was authored. Treat generated code with the same static analysis discipline as hand-written code. The real costs of AI-generated tests include the maintenance overhead of exactly this re-contamination cycle.

What Most Teams Get Wrong About Testing Process Steps and Isolation

The most persistent myth is that scenario isolation is a BDD concern rather than a step implementation concern. Teams spend time auditing their .feature files for shared state and miss the fact that the isolation contract lives in fixture scoping and teardown, not in Gherkin syntax. A perfectly written Given/When/Then scenario is meaningless if the step definitions underneath it share a connection pool. The anatomy of a well-scoped step definition matters as much as the anatomy of the step itself — they are two sides of the same contract.

A second common error is assuming that parallel execution solves the bleed problem. It doesn't — it makes it worse. Parallel workers that share a database schema without row-level or schema-level partitioning will produce race conditions that look like flakiness but are actually deterministic bleed under concurrency. Isolation must be established before parallelism is introduced, not as a consequence of it. If your suite is flaky under pytest-xdist -n 4 but stable under -n 1, you almost certainly have a bleed problem, not a timing problem.

Context bleed in AI-generated steps is a structural problem with a structural fix: enforce scenario-scoped resource lifecycle at the fixture layer and encode that constraint into every generation prompt. Once isolation is deterministic, the next metric worth tracking is mean-time-to-detect on order-dependent failures — run your suite in randomised order in a nightly job and alert on any scenario whose pass rate drops below 95% across 10 runs. That signal surfaces bleed before it reaches production.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.