When the mock starts lying to you
Every fake backend eventually drifts from the real one it stands in for, and the dangerous window is after it stops being obviously wrong — the signals that your mock has become the thing you are designing against, and what to do the day you notice.
The mock was obviously a mock for the first week. It said Looking into that... and returned a hardcoded weather result, and nobody could have mistaken it for anything. That is the safe state. A fake that is visibly fake cannot mislead you.
Three weeks later it has grown. It handles four scenarios, the tool outputs look plausible, the failure script is realistic, and a real backend team has started building the actual graph. Your mock has not changed, because nothing broke. It is still what you open, still what you demo, still what every design decision gets made against. And it has quietly stopped being a model of the system and started being a system of its own.
The dangerous window is after it stops being obviously wrong
Plot the mock’s realism against time and there are three phases.
- Obviously fake. Safe. Every decision made against it is made knowing it is provisional.
- Plausibly real. Dangerous. It is good enough that you stop annotating your conclusions as provisional, and not good enough to actually be right.
- Visibly broken. Safe again, and cheap. Something throws, you fix it, and the fix tells you what changed.
The middle phase is the one to design against, and it is worth being exact about why it is worse than the third. A mock that breaks costs you an hour. A mock that is plausibly real costs you a decision, and you will not know which decision, because it was made silently and it looked the same as every correct decision you made that day.
The mechanism is not neglect. It is the opposite. The mock gets better because you keep improving it, and every improvement moves it further from “obviously a placeholder” without moving it any closer to the real system, which nobody is comparing it against.
Six signals, in the order they usually appear
These are the ones this course watches for. They are observations from building this way, not measured indicators.
- The mock has a feature the real system does not. You added a convenient field because a component wanted it. Nobody agreed to it upstream.
- Someone asks the mock a question about the product. “What happens if the tool returns nothing?” and you check the mock to answer. The mock is now the specification.
- The scripts have branching. Conditionals in a fake backend mean it has acquired behaviour, and behaviour is the thing you were supposed to be deferring.
- It has never failed in a way you did not write. Real systems fail in shapes you did not anticipate. A fake only fails in the shapes you scripted, so the absence of surprise is not evidence that the surface is robust.
- Nobody can tell you when it was last compared. Not “when was it last edited” — when was its output last put next to the real system’s output by a person.
- You would be nervous to swap it. The honest test. If pointing the surface at the real backend for ten minutes feels risky, the mock is load-bearing and the swap you designed for is no longer a swap.
Mechanisms that make divergence mechanical
Detecting drift by noticing is unreliable, and there are published approaches that do not rely on noticing. None of them was designed for a three-hour prototype, which is exactly why they are worth knowing: they show you what the rigorous version costs, so you can choose your position on the cheap end of it deliberately.
A written interface description
The OpenAPI Specification — version 3.2.0, published by the OpenAPI Initiative under the Linux Foundation — defines, in its own words, “a standard, programming language-agnostic interface description for HTTP APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code.” It is normative text from a standards body rather than a vendor page, which is a meaningfully different kind of source from the rest of this module.
If the mock and the real service both describe themselves in one document, divergence becomes a diff rather than a discovery. Tooling follows from it directly: Prism, from Stoplight, part of SmartBear, describes itself as “an open-source HTTP mock server that can emulate your API’s behavior as if you already built it,” with mock servers generated from OpenAPI documents and both requests and responses validated against the description. Stoplight documents and sells API tooling, so that page is a vendor describing its own product.
Contracts recorded by the consumer and replayed at the provider
Pact takes the opposite direction. It calls itself “a code-first consumer-driven contract testing tool,” where “the contract is generated during the execution of the automated consumer tests.” Your tests run against a mock provider, and that run produces a pact file describing every interaction. Then a separate verification step takes over: each recorded request “is sent to the provider, and the actual response it generates is compared with the minimal expected response described in the consumer test”.
That second step is the part with no cheap substitute, and it is the part a prototype cannot have. Verification needs a running provider. When the provider does not exist yet — the entire premise of this module — you can generate contracts all day and never verify one. Pact’s deliberate narrowness is worth carrying anyway: it notes that only the parts of the communication actually used by consumers ever get tested, which is the same coverage trade your scripted mock is making silently.
Do not import contract testing into a three-hour prototype
Every mechanism above is real and none of them belongs inside the clock. A prototype that carries contract verification is not a prototype, and the inclusion test from the kit module rejects it outright: it encodes a decision the next prototype should be free to skip. Know these exist so that you can say what you are trading away, and so that the day a prototype turns into a project you know what the upgrade path is called.
What to do the day you notice
The failure mode is treating discovery as a bug report and patching the mock. That restores plausibility without restoring truth, which is the dangerous phase again with a fresh commit on top. Four moves instead, in order.
- Write down what you believed. Before touching anything, record the behaviour you thought was true. This is the only moment that record is recoverable.
- Find the decisions that rested on it. Drift is only expensive through the choices it produced. List them explicitly; there are usually fewer than you fear and never zero.
- Fix the surface first, the mock second. The mock is not the deliverable. If a design assumed something false, the design is what is wrong.
- Decide whether the mock still earns its place. If the real system exists and is reachable, the mock has become optional, and keeping an optional fake around is how the next round of drift starts.
Check your recall
Answer from memory — no scrolling back.
Retrieval check
You discover the real backend returns tool errors as a normal output with an error field, not as the protocol’s error part — and your mock has always used the error part. What do you do first?
Check your answer
Write down what you believed and what you built on it, before editing anything. The mock is the cheapest thing in the room to change and the least important. What matters is which pieces of the surface were designed around a failure arriving as a stream-level event, because those are the pieces that are now wrong, and after you fix the mock you will no longer be able to tell which ones they were.
Then fix the surface, then the mock, then ask whether a mock is still the right thing to be pointing at now that a real backend exists and is reachable.
Hands on
Give the mock an expiry and a comparison
Done when: The mock has a dated freshness note in the kit manifest listing what it was last compared against, a written list of the protocol behaviours it asserts, and one recorded comparison against a real streaming backend with the divergences written down.
- Add a dated note at the top of the mock’s own file: what it imitates, when its behaviour was last checked against something real, and by whom. A mock with no date is a mock nobody can reason about.
- List the behaviours the mock asserts, not the parts it emits — “tool errors arrive as the protocol error part,” “approval requests precede tool output,” and so on. Each line is a claim about the real system that could turn out to be false.
- Run one real streaming backend behind the same surface, even a trivial one with a single real tool call. Capture its actual event sequence.
- Diff that sequence against your list and record every divergence, including the ones you decide not to fix. The unfixed ones are the honest inventory of what your prototype currently assumes.
- Write the comparison cadence into the manifest next to the kit’s freshness budget, and put the next date on your calendar. The cold-clone drill and this comparison are the same class of mechanism: a check that only exists if it is scheduled.
- Bring the divergence list into the chat. I will push hardest on any entry you marked “does not matter,” since that judgement is the one that later turns out to have been a design decision.
What this does not cover
Nothing here says which components should exist on top of the mock. Two of them are non-negotiable for an agentic surface and neither gets built under a three-hour clock, so both go into the kit once, against the mock protocol rather than against any particular backend. That is the next lesson, on wiring the trace and the gate once.
The drift discussed here is drift between your mock and a real system. Drift between your kit and the ecosystem it was built on — versions, renamed helpers, a build that no longer starts — is a different failure with a different remedy, and it is the maintenance-tax lesson in the previous module.
Read this next — primary source
Pact — How Pact worksdocs.pact.io — fetched 5 September 2026. Pact documenting its own tool; the open-source project has a commercial broker product behind it, so read the positioning with that in mind.
Read this as an existing engineering answer to the problem this lesson names, not as evidence for the lesson’s argument. Pact is the clearest published account of making mock-versus-real divergence mechanically detectable: the consumer test records what it actually asked for, and a separate verification step replays those recorded interactions against the running provider. The half worth your attention is the asymmetry it accepts on purpose — only the parts of the interaction the consumer uses ever get checked — because that is the same trade a scripted prototype mock makes implicitly and never writes down.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.