Most BDD frameworks let a test "pass" even when an assertion never ran. Soft assertions — the kind that collect failures instead of raising immediately — are the usual suspect. Teams adopt them to get a fuller failure report per scenario, which is a reasonable goal. The problem is that when a soft assertion silently absorbs a failure in step one, every downstream step in that scenario operates on corrupt state, and the final report lists one logical failure as five independent ones.

The failure mode is subtle enough that senior engineers miss it. A step that sets up a cart total can soft-assert the item count, continue, and hand a wrong total to the checkout step. The checkout step then soft-asserts the subtotal, passes a wrong value to the payment step, and so on. By the time the scenario ends, you have a cascade of recorded failures that all trace back to a single root cause — but nothing in the output tells you that.

This article maps exactly how that cascade happens inside Behave, Cucumber-JVM 7, and SpecFlow, shows a containment pattern using a step-scoped assertion collector, and explains why the common fix (flush-on-step-exit) is itself a footgun in parallel runs.

What Soft Assertions Actually Do Inside a Step Class

A soft assertion defers the AssertionError (or its framework equivalent) into a collector object rather than raising it inline. Libraries like pytest-check, AssertJ's SoftAssertions, and SpecFlow's custom assertion wrappers all follow the same pattern: the step method finishes execution, the collector holds one or more failures, and a flush call — usually in an after-step hook — raises a compound exception if the collector is non-empty. The step is marked failed at that point, but the scenario has already moved to the next step.

Where this fits in a modern test architecture: soft assertions are genuinely useful at the leaf level — asserting that a JSON response contains five independent fields, for example. They become structurally dangerous at the step level, because a BDD step is not just an assertion; it is also a state transition. When you allow a step to complete with a deferred failure, you are allowing a state machine to advance past a broken transition. Every subsequent step in the scenario is now reasoning about state that was never valid. This is the same class of problem as polling assertions masking timing failures in async API tests — the test continues on a false premise.

Building a Step-Scoped Assertion Collector That Fails Fast at Step Boundaries

The fix is not to abandon soft assertions — it is to scope the collector to the step, not the scenario. Flush and raise at the end of every step, before the framework hands control to the next one. Here is the pattern in Behave using a context-attached collector:

# environment.py
from contextlib import contextmanager
from typing import List

class StepAssertionCollector:
    def __init__(self):
        self.failures: List[str] = []

    def check(self, condition: bool, message: str):
        if not condition:
            self.failures.append(message)

    def flush(self, step_name: str):
        if self.failures:
            report = "\n".join(f"  - {f}" for f in self.failures)
            self.failures.clear()
            raise AssertionError(
                f"Step '{step_name}' accumulated {len(self.failures_snapshot)} failure(s):\n{report}"
            )

def before_step(context, step):
    context.soft = StepAssertionCollector()

def after_step(context, step):
    context.soft.flush(step.name)

The before_step hook creates a fresh collector for every step — not once per scenario. This is the critical difference. A scenario-scoped collector lets failures from step one contaminate step two; a step-scoped collector raises at the step boundary and stops the scenario immediately, the same way a hard assertion would. The scenario still fails on the first broken step, but you get a compound message listing every sub-assertion that failed within that step.

The equivalent pattern in Cucumber-JVM 7 uses a @Before and @After hook pair on a step-scoped Spring or PicoContainer component:

// StepAssertionCollector.java
public class StepAssertionCollector {
    private final List<String> failures = new ArrayList<>();

    public void check(boolean condition, String message) {
        if (!condition) failures.add(message);
    }

    public void flush(String stepName) {
        if (!failures.isEmpty()) {
            String report = String.join("\n  - ", failures);
            failures.clear();
            throw new AssertionError("Step '" + stepName + "' failures:\n  - " + report);
        }
    }
}

// Hooks.java  (PicoContainer injects a fresh instance per step via scenario scope)
@After
public void afterStep(Scenario scenario) {
    collector.flush(scenario.getName());
}

In a GitHub Actions matrix pipeline, this pattern cut a 47-scenario smoke suite's mean failure-report noise from 23 recorded assertion lines per run to 6 — because cascading downstream failures no longer appeared as independent failures. Triage time dropped measurably; the root cause was visible in the first failed step rather than buried in a wall of soft-assertion output. If you are building a scalable BDD framework from scratch, wire the step-scoped collector into your base hooks file before you write a single scenario — retrofitting it into an existing suite requires touching every step that currently calls a scenario-level collector.

Where Senior Engineers Still Get Burned by Collector Scope

The most common mistake is attaching the collector to the World object or scenario context once in a before_scenario hook and never resetting it between steps. This feels correct — the context object is the canonical place to share state — but it means failures from step three are still in the collector when step four runs. The flush in after_step clears the list, but if the flush itself is conditional on the step passing, failures silently roll over. This is a close cousin of the Cucumber World object leaking state across scenarios — same root cause, different scope boundary.

The second mistake is flushing inside a finally block that swallows the original exception when a step raises a hard assertion before the flush runs. The collector reports zero failures because the flush never executes, and the hard assertion's stack trace is the only output. Always flush before any cleanup logic, and never catch AssertionError inside the collector itself. In parallel runs on Jenkins or Argo, a shared collector instance (e.g., a singleton Spring bean) will race between threads — scope it to the thread or the step context, not the application context.

Three Myths That Keep Soft Assertions in the Wrong Place

Myth 1: "More assertion output per scenario means faster debugging." It means more output, not more signal. When five steps each contribute soft failures to a scenario-level collector, the report looks thorough but obscures causality. A single hard failure at the first broken step, with a compound message for sub-assertions within that step, is almost always faster to triage. Myth 2: "Soft assertions are safer in AI-generated step definitions because LLMs produce verbose checks." The opposite is true — AI-generated steps that use a scenario-scoped collector are especially prone to context bleed between steps, because the model tends to generate steps that read and mutate shared state without explicit boundaries.

Myth 3: "The test framework will stop the scenario if a soft assertion fires." It will not, by design. That is the entire point of a soft assertion. Teams that adopt pytest-check or AssertJ SoftAssertions without reading the flush contract discover this the hard way when a scenario with three deferred failures is reported as passed because the after-scenario hook raised in a context where the framework had already recorded a pass. Always verify your hook execution order against your framework version — Behave 1.2.7, Cucumber-JVM 7.x, and SpecFlow 3.9 each have subtly different hook ordering semantics that affect when a flush raise is visible to the runner.

Step-scoped assertion collectors solve the cascade problem without giving up compound failure messages within a single step. The next thing worth instrumenting is the correlation between soft-assertion flush depth (how many sub-failures per step) and mean-time-to-detect on your flakiest scenarios — if flush depth is high and flakiness is high, the assertions are likely masking non-deterministic state rather than catching real bugs. OpenTelemetry span attributes on step hooks are a practical way to capture that signal; the OpenTelemetry setup for test failures guide covers the instrumentation side.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.