Directual
Menu

Who Fixes an AI-Written Backend at 3 a.m.? We Ran the Test

November 18, 2026

We had Claude Code and Codex build the same shop backend 18 times and tried to break it. The code held. Then we asked five questions a shop owner gets at night. An agent answers them in under a minute for $0.30. A person without an agent needs SQL and the worker code, and two of the five questions have no answer at all, for anyone.

Who Fixes an AI-Written Backend at 3 a.m.? We Ran the Test

We had two coding agents build the same small shop backend 18 times, then threw production failures at it. The code held: duplicate webhooks, a dying warehouse API, a kill -9 mid-request, 30 buyers racing for the last 5 bags of coffee. That is not where AI-written backends break.

They break at 3 a.m., when the owner gets a message from a customer and has to find out what happened in code no person has ever read. We tested that too. Short version: with an agent, five night questions take under a minute and 30 cents. Without one, they take SQL, JSON operators and the worker code. And two of the five have no answer at all, for anyone, because nobody asked the agent to build the thing that would answer them.

The code is not the problem

The task was an orders API for a coffee shop: customers, products with stock, orders, a signed payment webhook, a receipt email and a shipment to an external warehouse. Claude Code (Claude Opus 5.5) and Codex (gpt-6.1-sol) each built it nine times, alone, from an empty folder. We ran three series: production conditions spelled out in the spec; no conditions, only the API docs; and no conditions with a bare sandbox that hinted at nothing.

Then we ran 14 probes against every build, and a second agent session added a required field against a database we told it was production.

What we threw at it Builds that survived
Customer B reads, lists and cancels customer A's order 18 of 18
Unsigned or wrongly signed payment webhook 18 of 18
The same webhook twice, and five copies at once 18 of 18
The warehouse drops the connection, then answers 503 18 of 18
30 buyers, 5 bags in stock 18 of 18
kill -9 while the warehouse call is in flight, then restart: exactly one shipment, one receipt 18 of 18
A new required field added on live data: no lost orders, old passwords still work 18 of 18
The warehouse always takes 12 seconds to answer 9 of 18

Every build, unprompted, used the same design a senior engineer would: a job queue in Postgres with leases and SKIP LOCKED, an idempotency key on every outbound call, conditional stock updates, a constant-time signature check, a non-destructive migration. Builds took 4 to 15 minutes. A Claude run, build plus change, cost $1.60 to $2.20.

The one failure is instructive. Agents pick a sensible default timeout of 5 or 10 seconds. A warehouse that answers in 12 never gets through. The order stays "paid" forever while the system retries in the background. One agent's code gives up after 20 attempts and writes a line to the log. Another retries indefinitely and says nothing.

So the code is fine. What is missing is a reader.

Zero human readers

The service we used for the night test is 832 lines of Node.js, written in five minutes. It even comes with a good README. Nobody has read it: not the author, who is a model; not a reviewer, because there was none. The owner becomes its first human reader at the worst possible moment.

It does not get better over time. The next feature is written by an agent too, and the one after that. The code grows; the number of people who have read it stays at zero. Every time something has to be understood, the owner calls an agent, and the agent reads everything again from scratch.

The 3 a.m. test

We kept the service running for three days of synthetic shop life: 40 customers, 125 orders, one price changed quietly in the database. Then the warehouse started answering slower than the service's timeout. The owner has the code, the settings file, the log, the database and five questions:

  1. A customer writes: "I paid two hours ago and my coffee hasn't shipped." What happened to her order?
  2. Is the system still trying to ship it? How many times has it tried, and what happens next?
  3. The warehouse says it has an outage. How do I pause shipments for an hour and keep everything else running?
  4. The webhook secret may have leaked. Where is it, and how do I change it?
  5. The price of Ethiopia Yirgacheffe went from 14.50 to 18.90. Who changed it, and when?

We gave the same questions to fresh Claude Code and Codex sessions with no context, read-only. Then we traced, step by step, what a person without an agent has to do to answer each one. We did not put a person through it this time, so the right-hand column is our breakdown, not a stopwatch.

Question Agent, fresh session What a person needs without an agent
1. Where is the order? Both correct: paid, receipt sent, every shipment attempt times out after 10 s A SQL join from the customer's email to her orders, then to a jobs table through a JSON field. The log says "job 210 attempt 8 failed" and never mentions the order or the time
2. Still trying? Both correct: attempt 8 of 20, delays doubling up to 10 minutes, then dead, then nothing The jobs table for the count; the worker code or the README for the retry rules
3. Pause shipments Both correct: there is no switch. A SQL workaround delays queued jobs, but every new payment schedules a new one Understanding the worker loop well enough to see that the switch does not exist
4. Rotate the secret Both correct: an environment variable read at start; change it at the provider and in the settings, restart, expect 401s in between Knowing where the settings live, how to restart without a stop script, and that the provider will retry
5. Who changed the price? Both correct: nobody can tell. Claude narrowed "when" to a 10-hour window from order amounts The same answer, after realising there is no audit anywhere

The agents were good. Claude Code answered all five in 57 seconds for $0.29; Codex in 47 seconds. Claude even noticed that our synthetic history did not add up: eight attempts with that backoff should take five minutes, not two hours.

Three things stand out.

The log speaks a different language from the customer. The customer has an email and "two hours ago". The log has job numbers and no timestamps. The bridge is a JSON field in a table the owner did not know existed.

The agent reads everything, every time. Both sessions read essentially the whole codebase before answering. At 832 lines that costs 30 cents. The service will not stay at 832 lines, and the next incident starts from zero again.

Two questions have no answer, for anyone. There is no switch to pause shipments, and no record of who changed a price. Not a bug: nobody asked for them. The agent built exactly the service in the spec, and the spec did not say "the owner will need to see and steer this at night". Neither would yours.

And one thing nobody saw. In our setup, as with many real APIs, the warehouse created the shipment on the very first request; the service just never got the answer back. The coffee is probably on its way. The system says it is not, and neither agent mentioned it.

What an operable backend needs

None of this is solved by better code. It is solved by things the owner can see and touch without reading code:

  • Every external call, per order. Open the order and see the receipt that went out and the shipment that is on its eighth attempt, with the error and the next retry time.
  • A pause that keeps what it missed. Stop one integration for an hour and replay what arrived in the meantime, without stopping orders or payments.
  • A log of who changed what. Data and settings, people and agents, in one feed.
  • Secrets in one place. One screen, one way to rotate.
  • A description a person can read. What the system stores, what it calls and when, readable without opening a single source file.

Ask your agent to build these and it will: it built everything else we asked for. But each one is another few hundred lines nobody reads, and they are the same few hundred lines for every backend. They belong to the platform the backend runs on, not to each app.

Method

  • Agents: Claude Code 2.1.326 with Claude Opus 5.5; Codex CLI 0.167.0 with gpt-6.1-sol, reasoning effort medium. Each worked alone, without a human, in an empty folder, on 18 November 2026.
  • Build test: 18 builds of the same spec (Node.js or Python, PostgreSQL), 14 probes each, plus a schema change on live data. Probes were checked against a reference implementation: it passes all 14, and with idempotency keys removed it fails the kill -9 probe with two shipments.
  • 3 a.m. test: one of the 18 builds, 832 lines, three days of synthetic data, one fresh session per agent, read-only.
  • Limits: a small single service and three builds per cell. Weeks of changes, several services and codebases larger than one context window are not covered. The human column is our breakdown, not a timed trial.

FAQ

Is backend code written by AI agents bad?

Not for a small, well-specified service. In our November 2026 test Claude Code and Codex built the same orders API 18 times; every build used a durable job queue, idempotency keys and race-safe stock updates, and survived duplicate webhooks, a kill -9 and a schema change on live data. Nine of 18 failed one probe: a timeout shorter than a slow dependency's answer.

What does 'zero human readers' mean for AI-generated code?

Code an agent wrote in minutes that no person has read: not the author, not a reviewer. The first human to read it is whoever is on call when something breaks. Each later change is usually made by an agent too, so the number of human readers stays at zero while the code grows.

Can an AI agent answer production questions about a backend it did not write?

Yes, for a small codebase. A fresh Claude Code session answered our five 3 a.m. questions correctly in 57 seconds for $0.29, and Codex in 47 seconds; both read essentially the whole 832-line codebase and queried the database. Neither could answer what the system does not record, such as who changed a price.

What makes a backend operable without reading its code?

Five things: every external call visible per order with its attempts and errors; a switch to pause one integration and replay what it missed; a log of who changed data and settings, people and agents alike; secrets in one place with a way to rotate them; and a description of the system a person can read without reading code.
← Back to list