I built five demo sites on the same pipeline: a watch teardown, a coffee roaster with a real cart, a boutique hotel, a fractional CFO, a food guide. Same scaffold, same audit step, same deploy.

After each one I wrote up what I learned. Two of those writeups said the same thing: running out of image budget forced a restraint I would have had to talk myself into otherwise.

Read on its own, each one sounded true. I believed it when I wrote it, both times.

Then I put all five audit scores in one column.

The two sites that ended up with imagery scored highest. The two without scored lowest.

Then it got worse, because "ran out of budget" and "ended up with no images" are not the same split and I had been treating them as one thing. Casa Vento ran out of stills at nine of eleven. It still tops the table at 8.3, because the hero film had already been made and the film was the whole page. Meridian Table hit zero before generating anything and came in at 7.9. CFO Line never had a budget problem at all. It banned images from the start as a deliberate rule, and it scored lowest of the four.

So running out did not sharpen anything. Casa Vento scored highest despite running out, because the one asset that mattered survived. The site that chose restraint on purpose scored last.

Nothing in either writeup was false. I never claimed a score I did not get. The story just did not survive being read next to its neighbours, and I had published it twice without putting these four numbers in one place.

The watch teardown is missing from that table because it never reported an audit score at all. Five writeups, four scores, and I did not notice the gap until I tried to build the column.

Why one post at a time can never catch this

Here is the part I think generalises past my five demo sites.

Coherence gets graded per document. When I sat down to write about the hotel, I had one build in front of me and one set of facts, and I produced something internally consistent. Same for the food guide. Neither writeup contained a contradiction, because the contradiction did not exist inside either one. It only existed in the gap between them.

So no amount of reviewing a single post catches it. Not a careful reread, not an editor, not a critic agent pointed at one file. The defect is structural: you are checking documents individually for a property that only shows up across the set.

This gets worse when a model does the writing. I use AI for a lot of this, and an LLM writing your fifth retro has no memory of the first four. It will assert a confident causal story every time, because asserting is cheap and checking is expensive. Five sessions, five internally tidy narratives, no mechanism anywhere forcing them to agree. If you are shipping AI-assisted status reports, test summaries, or client updates, you have this exact problem and per-item review will not find it.

The fix is not better writing. It is a check that runs across artifacts instead of inside them.

The prompt that found it

This is the whole intervention. Point a model at every writeup at once and forbid it from summarising.

You are reviewing N writeups produced by the same pipeline.
Do not summarize them. Find where they disagree.

Return:
1. REPEATED CLAIMS: sentences or ideas appearing in near-identical
   form in 3+ writeups. These are the through-lines, and they are
   also where the writing is on autopilot.
2. GENUINE TENSIONS: any place two writeups make opposite choices
   or tell opposite causal stories about the same outcome.
3. CLAIMS THE DATA CONTRADICTS: for every qualitative claim
   ("X made the work better"), find the reported numbers and say
   whether they support it, contradict it, or are silent.
4. WEAKEST PATTERN: the move all N make that reads as filler.

Quote real numbers. Do not be charitable. If a claim is
unfalsifiable as written, say so.

Point 3 is the one that did the work. Points 1, 2 and 4 are useful tidying. Point 3 is what produced the table at the top of this post, because it forced the qualitative claim and the reported number into the same sentence, which is exactly what writing them a week apart had prevented.

It also caught something I would have defended: the CFO site has zero images too, but there the decision was deliberate from the start. Same output, and I had written two completely different causal stories about how it happened. Reading the five together, you cannot tell which sites chose restraint and which had it imposed, because I wrote them all in the same satisfied voice.

If I cannot separate my own accidents from my own decisions two weeks later, a client reading the same page has no chance.

What actually broke, since you will hit these too

Four things across five builds, and none of them are the interesting part of this post. They are the price of admission. If you want the argument for keeping the system small enough to actually ship, that is a different post.

Image credits ran out twice. Casa Vento died at nine stills of eleven and two sections went type-only. Meridian Table hit zero during planning. After the first one I did not add a check, so the pipeline still had no idea what a build would cost before starting it.

Content drifted silently. Content lives in a data.json per site, and a missing field does not throw. It renders an empty section that looks deliberate. On the food guide a copy critic caught a budget breakdown whose math did not add up, plus one venue whose opening hours contradicted themselves between two sections of the same article. Both were in perfectly valid JSON.

Deploy and gating only failed in production. Host header slugs, token gates, domain attachment. All of it passed locally, because the failure mode is a 200 response with the wrong thing behind it.

And they came out looking related. One scaffold, five sites, same section rhythm. More component variation was not the answer. Forcing a different compositional treatment per site up front was.

The one number I still trust

For the hotel hero I needed a video loop with no visible seam. The trick was feeding Kling the same image as both the first and last frame, then timing a boat to exit and re-enter frame at the reset. To check it, I measured structural similarity between the first and last frame: 0.9696.

That is the only place in five writeups where I measured a claimed effect instead of asserting it. It does not prove a viewer sees the loop as unbroken, which is a different question. It does mean I checked something. Everything else is a design score, which is a model's opinion, or a Lighthouse number, which measures the shell rather than the work.

Three things I am building before the next batch

A budget check that runs before generation, not during. Count the planned assets, multiply by per asset cost, compare to balance. If the balance does not cover the full plan, stop and ask. Do not generate partway and then call the gap a style. This is one function before step one, not a monitor during.

A content contract that throws instead of rendering blank. Zod is already in the stack. Required fields missing, fail the build. Numbers that should sum get summed in the validator and throw when they do not. An empty section should never reach a browser looking like a design decision.

A build log every writeup has to cite. One JSON Lines file per build, one line per event: asset generated, budget hit, section skipped, validation error. Then "restraint sharpened this" either points at a logged line showing the credits ran out, or it does not ship.

The third one is the only one that would have caught this post's mistake. The first two prevent broken builds. The third prevents broken stories, and until this week I did not know I needed it.

What I am not claiming

Four data points cannot establish that imagery raises design scores. I am not claiming that, and if you take that away from this post I have written it badly.

I also nearly shipped this post with a second version of the same mistake. The first draft opened by saying the two sites that ran out of budget scored lowest, which is not true: one of them scored highest. A critic caught it by tracing Casa Vento through the piece. Two writeups made a claim their own numbers contradicted, and then my post about that made a claim its own table contradicted. The check is not something you run once and graduate from.

The scores came from a model, on a ten point scale, judging sites adjacent to the pipeline that built them. Treat that as a consistent instrument rather than an accurate one. The ordering across five builds is the signal. Any single number is not.

These are demo sites. The brands are fictional, no revenue moved, and a real client engagement would apply pressure none of this had. They passed the checks I set for this pipeline, which is not the same as passing an external standard.

What I am claiming is narrower and I am confident in it: I wrote a causal story twice, the numbers I already had pointed the other way, and no amount of reviewing those posts one at a time would have told me. That check has to happen across the set. It took one prompt and about ten minutes, and I had gone five builds without running it.