Point wrk at an MCP server and you get a clean report: every response a 200. Some of those 200s carry isError: true results, and nothing in the output says what the tool definitions cost the model on every turn. The tool is measuring requests per second. An agent runs a workflow, and what the server costs it is latency per call times calls per task, plus the tokens each response and each definition puts into its context.

You test an MCP server by chatting with it. Ask for the open bugs, get the open bugs, ship it. The model never sends limit: "ten", an empty path or a 200 KB query while you’re watching, so those paths stay dark until a real session hits one. Say it’s the limit. The handler throws, the framework wraps the exception in a 40 KB stack trace, and the model reads all of it, adjusts, and retries with the same bug in a new shape.

A customer’s checkout returns a 500. Support pastes the request ID into the ticket, and the engineer on call finds the log line: KeyError: 'tax_region' in the pricing module. They send the same request locally and get a 200. Of course they do. Their database has no customer with a null tax region, the feature flag that routes to the new tax engine is off in development, the rates service answers differently today, and the clock is a day later. The bug is a function of all of that, and the ticket contains none of it.

Your agent test passes on your laptop and fails in CI. The laptop has a token for the issue tracker’s MCP server; CI doesn’t, and shouldn’t. Even with a token, the tracker listed three open bugs yesterday and lists four today, so the agent’s summary changes and the assertion on it breaks.

Web developers dealt with this years ago. VCR (Ruby), Polly.js, Betamax and go-vcr record real HTTP responses into a file the first time a test runs, then serve them from that file on every run after. The file is called a cassette. MCP needs the same thing: a proxy that sits between an agent and its MCP servers, writes every exchange into one cassette, and plays it back later with no server, no credentials and no network. mcprec below is an illustrative name for a tool you’d have to build.

You’re about to refactor the billing endpoints. The service has a few dozen tests, mostly happy paths, and nobody trusts them to catch a changed rounding rule or a renamed field. Writing better ones by hand means reading every handler and inventing inputs. Meanwhile production receives thousands of real inputs a minute, and each one comes labelled with the response your current code gives.

Capturing that traffic is the easy part; a proxy, a packet tap or a log line with the body in it will do. The product is everything after. Ten minutes of traffic holds thousands of near-duplicate requests and a handful that exercise something different, and a tool is only useful if it can tell them apart. Cluster by endpoint, request shape and response shape. Pick representatives that cover the status codes and branches. Assert only on fields that are stable. Mock the downstream calls. What comes out is a few dozen characterization tests (Michael Feathers’ name for tests that pin down what code does today).

An API can pass every unit test in its suite and still break every client that calls it, because unit tests check that functions do what the code says they do, not that the API does what the contract says it does. That gap is where most production API incidents actually come from: a field renamed, a status code changed from 200 to 204, a required parameter quietly made optional. Testing an API well means testing at several different levels, because each one catches a different class of failure.

Unit tests verify that individual functions behave correctly in isolation. They run fast, catch regressions close to where they are introduced, and document expected behavior at the code level. They do not verify that the API endpoints exposed to consumers behave according to the documented contract, that changes in one service do not break the consumers that depend on it, or that the system behaves correctly under the conditions that production traffic creates.