Schema Registry Drift Breaks Contract Tests
Pact has been the de facto standard for consumer-driven contract testing in API and distributed system testing for nearly a decade. Most teams wire it up once, get the green badges, and move on. Then six months later a field rename in Confluent Schema Registry or AWS Glue silently invalidates a dozen consumer contracts — and nobody notices until a login flow returns a 422 in staging. The schema changed. The Pact broker didn't know.
The root cause is a gap in ownership: contract tests verify message shape at a point in time, while schema registries evolve continuously under a separate compatibility policy. When those two systems drift apart — even by a single field type promotion from int to long — your contract suite becomes a liability instead of a safety net.
By the end of this article you'll know exactly where the drift originates, how to instrument your CI pipeline to catch it before merge, and which compatibility modes in Confluent Schema Registry (CSR) and AWS Glue actually align with Pact's matching rules. A working GitHub Actions job is included.
Smarter API Test Automation — Python, Behave, VS Code, AI with GitHub Copilot & CI/CD Pipelines. Complete in a Weekend!
Why Schema Registry Drift Is a Contract Testing Problem
Contract testing in API contexts — whether you're using Pact 4.x, Spring Cloud Contract, or Pactflow — operates on a static snapshot of a message schema captured at the time the consumer writes its expectations. The Pact broker stores that snapshot as a JSON pact file. The schema registry, by contrast, stores a living Avro, Protobuf, or JSON Schema definition that producers evolve under a compatibility rule (BACKWARD, FORWARD, FULL, or NONE). These two systems share no runtime coupling by default.
Drift occurs when a schema advances to a new version that remains registry-compatible (say, adding a nullable field under BACKWARD compatibility) but violates a consumer's Pact expectation that asserted the field was absent. The registry says "valid evolution." Pact says "unexpected key." Both are correct in their own frame. The problem is architectural: a modern distributed test strategy must treat schema registry state as a first-class test input, not an external assumption. Without that coupling, your contract suite tests the past, not the present.
Coupling Schema Registry Versions to Your Pact CI Gate
The fix is a two-step CI gate: fetch the current registered schema before running Pact verification, then assert that the registered schema is structurally compatible with every active consumer pact. If the schema has advanced beyond what any consumer expects, fail fast. Here's the pattern using Python, Pact's verifier, and the Confluent Schema Registry REST API.
# schema_drift_check.py (Python 3.11+, requests 2.31, pact-python 2.x)
import requests, json, sys
from pact import Verifier
REGISTRY_URL = "https://schema-registry.internal"
SUBJECT = "user-login-value" # Avro subject for the login event
PACT_BROKER = "https://pact-broker.internal"
PROVIDER = "auth-service"
def fetch_latest_schema(subject: str) -> dict:
resp = requests.get(f"{REGISTRY_URL}/subjects/{subject}/versions/latest")
resp.raise_for_status()
return json.loads(resp.json()["schema"])
def assert_no_required_field_additions(schema: dict, baseline: dict) -> None:
"""Fail if new non-nullable fields appeared since baseline was captured."""
new_fields = {f["name"] for f in schema.get("fields", [])} - \
{f["name"] for f in baseline.get("fields", [])}
breaking = [f for f in schema.get("fields", [])
if f["name"] in new_fields and "null" not in str(f.get("type", ""))]
if breaking:
print(f"DRIFT DETECTED: required fields added {breaking}", file=sys.stderr)
sys.exit(1)
baseline = json.load(open("schemas/user-login-value.baseline.json"))
live = fetch_latest_schema(SUBJECT)
assert_no_required_field_additions(live, baseline)
verifier = Verifier(provider=PROVIDER, provider_base_url="http://localhost:8080")
verifier.verify_with_broker(broker_url=PACT_BROKER,
publish_verification_results=True,
provider_version="$GIT_SHA")
The baseline JSON is committed to the repo and updated deliberately — not automatically — so schema promotions require a conscious PR. The assert_no_required_field_additions check catches the most common drift pattern: a producer team adds a required Avro field under BACKWARD compatibility (which the registry allows, because old consumers can still deserialize), but the Pact consumer test fails because it used EachLike or exact-match on the fields array. In one internal rollout this gate caught three silent drift events in the first two weeks; CI run time added roughly 800 ms.
Wire this into GitHub Actions as a pre-verification job so Pact verification never runs against a drifted schema:
# .github/workflows/contract-tests.yml
jobs:
schema-drift-check:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.11" }
- run: pip install requests pact-python==2.2.1
- run: python schema_drift_check.py
env:
REGISTRY_URL: ${{ secrets.SCHEMA_REGISTRY_URL }}
PACT_BROKER: ${{ secrets.PACT_BROKER_URL }}
GIT_SHA: ${{ github.sha }}
pact-verify:
needs: schema-drift-check
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- run: ./gradlew pactVerify
For teams on Kafka with Confluent Schema Registry, set the subject compatibility to FULL_TRANSITIVE rather than the default BACKWARD. FULL_TRANSITIVE enforces that every schema version is both forward- and backward-compatible with all prior versions, not just the immediately preceding one. This narrows the window between what the registry permits and what Pact consumers can tolerate. Pair it with OpenTelemetry-based tracing on test failures to correlate schema version changes with consumer error spikes in Grafana — a causal link that's otherwise invisible in distributed logs.
Where Senior Engineers Still Get Burned by Drift
The first mistake is treating the Pact broker as the single source of truth for schema shape. Pact brokers store consumer expectations, not canonical schemas. Teams that skip the schema registry entirely and rely only on Pact's JSON pact files discover the gap when a producer rotates to Protobuf 3 and the Pact file still encodes Avro field-order assumptions. The fix is explicit: the baseline schema file in the repo must be the canonical reference, regenerated from the registry on every schema bump, reviewed in the same PR that updates the pact.
The second mistake is running Pact verification against a provider stub that doesn't serialize through the actual Avro codec. If your provider verification spins up a Spring Boot test context that returns plain JSON, it bypasses the Avro serializer entirely — meaning field type promotions (int → long, string → bytes) pass verification silently. Use the same Kafka producer configuration in your test context that production uses, including schema.registry.url and value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer. This is especially relevant for contract testing in API-plus-event hybrid architectures where the REST surface is tested but the async path is not.
Myths That Keep Contract Suites Fragile
Myth 1: Registry compatibility rules make contract tests redundant. BACKWARD compatibility means old consumers can read new data — it says nothing about whether your Pact consumer expects that data shape. A registry-compatible schema change can still break a consumer that used term() or like() matchers anchored to a specific field set. Registry rules and Pact matchers operate at different layers; you need both. Myth 2: Contract testing is only for REST APIs. Pact's message pact support covers async contracts over Kafka, Pulsar, and SNS. If your architecture has event-driven boundaries — and most do — those async contracts are where drift is most dangerous, because there's no synchronous HTTP 422 to surface the failure immediately. The test pyramid as a static model has no layer for async contract verification; teams that follow it literally skip this entirely.
Myth 3: The login flow is low-risk for schema drift because it's stable. Authentication events are among the highest-volume, most-consumed topics in a platform. A user-login-value subject may have 15 downstream consumers. A single field rename — user_id to userId — under a schema registry that allows NONE compatibility will silently break every one of them. Distributed system test strategy should flag high-fan-out subjects for stricter compatibility modes and mandatory drift checks in CI, not treat them as stable because they rarely change.
Schema registry drift is a tooling integration gap, not a process failure. The baseline-schema-in-repo pattern closes it with minimal overhead — under a second of CI time for most subject counts. If you implement this gate, the next measurement worth tracking is mean-time-to-detect on schema-induced consumer failures: with the gate in place, that number should collapse to zero for pre-merge changes. From there, extend the pattern to cover Protobuf descriptors and AWS Glue catalog versions using the same structural diff approach.
Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.