Most test suites grow by accretion. A feature ships, someone adds three scenarios, and six months later you have 400 Gherkin steps that cover the happy path well and the edge cases almost not at all. The bottleneck has never been execution — Playwright parallelizes across workers, GitHub Actions scales runners — it's always been authoring. Writing a thorough scenario set for a single API endpoint can take a senior SDET the better part of an afternoon when you factor in boundary analysis, negative paths, and contract alignment.

AI-assisted test generation changes that authoring bottleneck without changing what good tests look like. The output still needs to be deterministic, maintainable, and tied to real acceptance criteria. The difference is that ChatGPT-4o, Claude 3.5 Sonnet, and Cursor's composer can produce a first-draft scenario matrix in under two minutes — leaving the engineer to do the work that actually requires judgment: pruning redundancy, asserting on the right observability signals, and wiring the cases into CI.

By the end of this article you'll have a repeatable prompt-to-pipeline workflow: structured prompts that produce usable Gherkin and Pytest, a thin Python harness that calls the OpenAI API to generate cases programmatically, and a GitHub Actions job that gates the generated suite before merge. The pattern applies equally to Behave, SpecFlow, and Cucumber-JVM 7 shops.

AI Test Generation: What It Produces and Where It Fits

AI test generation, in its practical form, is prompt-driven scenario synthesis: you supply a model with a specification artifact — an OpenAPI schema, a user story, a domain model, or existing test stubs — and the model returns structured test cases. The output format depends on your prompt contract: you can ask for raw Gherkin, Pytest parametrize tables, Playwright TypeScript fixtures, or JSON that a downstream harness renders into whatever DSL you use. The model is not executing tests; it is doing the combinatorial analysis a human would do during test design — equivalence partitioning, boundary value analysis, negative path enumeration — but faster and without the cognitive fatigue that causes engineers to stop at five scenarios when twenty are warranted.

In a modern test architecture, this sits at the test design layer, upstream of your existing frameworks. Generated cases feed into Behave feature files, Pytest modules, or Cucumber-JVM 7 feature directories exactly as hand-authored cases would. The generation step can be a local CLI invoked by a developer, a pre-commit hook, or a GitHub Actions job triggered on OpenAPI spec changes. It does not replace the framework, the runner, or the reporting layer — Grafana dashboards, OpenTelemetry traces, and Allure reports remain unchanged. It replaces the blank-page problem at authoring time.

Building the Prompt-to-Pipeline Harness

Start with a prompt contract that constrains the model's output format. Unstructured prompts produce unstructured output; if you ask "write tests for my login endpoint" you will get prose. Ask instead for a JSON array of scenario objects with fields name, given, when, then, and tags. The model respects the schema reliably with GPT-4o and Claude 3.5 Sonnet — less so with smaller models.

# generate_cases.py  (requires openai>=1.30, pydantic>=2.0)
import json, textwrap
from openai import OpenAI
from pydantic import BaseModel

class Scenario(BaseModel):
    name: str
    given: list[str]
    when: str
    then: list[str]
    tags: list[str]

SYSTEM = textwrap.dedent("""
    You are a senior SDET. Given an OpenAPI path object, return a JSON array
    of test scenarios covering: happy path, boundary values, missing required
    fields, auth failures, and upstream timeout simulation.
    Output ONLY valid JSON matching the schema: [{name, given, when, then, tags}].
""")

def generate(spec_fragment: str) -> list[Scenario]:
    client = OpenAI()
    resp = client.chat.completions.create(
        model="gpt-4o",
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user",   "content": spec_fragment},
        ],
        temperature=0.2,   # low temp = less creative, more consistent
    )
    raw = json.loads(resp.choices[0].message.content)
    return [Scenario(**s) for s in raw["scenarios"]]

temperature=0.2 is the practical sweet spot: high enough to avoid repetitive phrasing across scenarios, low enough that the model doesn't invent fields your API doesn't have. Once you have a list of Scenario objects, render them to Gherkin with a Jinja2 template — no bespoke parser needed.

# render_gherkin.py
from jinja2 import Template

TMPL = Template("""
Feature: {{ feature_name }}
{% for s in scenarios %}
  @{{ s.tags | join(' @') }}
  Scenario: {{ s.name }}
{% for g in s.given %}    Given {{ g }}
{% endfor %}    When {{ s.when }}
{% for t in s.then %}    Then {{ t }}
{% endfor %}
{% endfor %}
""")

def render(feature_name: str, scenarios) -> str:
    return TMPL.render(feature_name=feature_name, scenarios=scenarios)

Wire this into GitHub Actions so the suite regenerates whenever the OpenAPI spec changes. A team running Behave against a payments microservice used this pattern to go from 23 manually authored scenarios to 91 generated ones in a single sprint — covering 14 previously untested error codes. Regression run time on that service stayed under 4 minutes because generated scenarios are naturally atomic and parallelize cleanly across Pytest-xdist workers.

# .github/workflows/generate-tests.yml
name: Regenerate Test Cases
on:
  push:
    paths: ['openapi/**/*.yaml']

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install openai pydantic jinja2 behave
      - run: python generate_cases.py --spec openapi/payments.yaml --out features/
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      - uses: actions/upload-artifact@v4
        with:
          name: generated-features
          path: features/

For Playwright TypeScript shops, swap the Jinja2 renderer for a template that emits test.describe blocks. Cursor's composer is useful here: paste the Scenario JSON into the composer context and ask it to emit a page.spec.ts file against your existing Page Object classes. The composer respects your existing import paths if you include one representative spec file in the context window.

Where AI Generation Breaks Down in Practice

Feeding the model stale or ambiguous specs is the most common failure mode. If your OpenAPI document has description: "user object" and nothing else, the model will hallucinate field names that don't exist in your actual implementation. The fix is upstream, not in the prompt: enforce spec completeness with Spectral lint rules in CI before the generation job runs. Generated tests are only as precise as the artifact you feed them.

The second mistake is skipping human review of generated tags and test scope. Models consistently over-generate @smoke tags and under-generate @contract or @security tags because those concepts are underrepresented in the training distribution relative to functional tests. A five-minute review pass — filtering by tag in Behave or Cucumber-JVM 7 before committing — catches this. The third pitfall is running generation at too high a temperature in a shared CI environment, which produces non-deterministic feature files that cause spurious diffs on every push. Pin temperature at 0.2 and seed the run with a fixed value if your model provider supports it.

Myths That Slow Down AI-Assisted Test Authoring

Myth 1: AI-generated tests replace test design judgment. They don't. The model applies combinatorial heuristics to whatever spec you give it. It cannot know that your payments service has a known race condition on concurrent refunds, or that a specific third-party dependency returns HTTP 200 with an error body. Domain-specific edge cases still require a human who has read the incident history. Treat generated output as a first-draft checklist, not a finished suite.

Myth 2: You need a dedicated AI toolchain separate from your existing framework. You don't. The harness above is ~80 lines of Python and a YAML file. It produces standard Gherkin that Behave, Cucumber-JVM 7, and SpecFlow consume without modification. There is no new runner, no new reporting layer, no vendor lock-in beyond the OpenAI or Anthropic API call — which you can swap for a self-hosted Ollama instance running Mistral if your org has data-residency constraints. Myth 3: 100% AI-generated coverage is the goal. Coverage percentage is the wrong metric. The goal is that every scenario in the suite is asserting on a real behavior that could regress. Forty well-scoped generated scenarios beat 200 generated ones that duplicate each other across minor input permutations.

The workflow described here — spec-driven prompt, structured JSON output, Jinja2 render, GitHub Actions gate — is operational today with GPT-4o and Claude 3.5 Sonnet. If you ship it, the next metric worth tracking is scenario-to-defect ratio: how many generated scenarios have caught a real regression in the last 30 days versus how many are dead weight. That ratio tells you whether your spec quality is high enough to make generation worthwhile, and it's a more honest signal than raw scenario count.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.