The framework held. I didn't

Two posts ago I rebuilt the framework I start every project on, around one idea: anything that depends on the agent remembering is not a mechanism, it is a hope. The post after that I went hunting for every place it was still trusting instead of checking, and put a number on each fix.

Then I did the thing I had been putting off. I pointed it at two projects big enough to hurt.

What "framework" means here. Not a library. Claude Code out of the box is a very capable agent with no opinions about how you work. On top of it I keep a set of rules, an agent roster, a memory store and a small CLI that checks things and refuses things. It exists to make certain mistakes structurally hard rather than merely discouraged. I call the whole pile viber, and when I say framework in this post, that is what I mean.

Games are the worst case, which is why I used them

Both test projects are games, and that was deliberate. A compiler will tell you a type is wrong. Nothing will tell you that wave three is boring. There is no schema keeping you honest about a difficulty curve. Almost every decision is a judgment call, which is exactly the shape of work where a framework either earns its keep or quietly gets in the way.

It earned some of it. Three examples, all now tests rather than anecdotes.

A rule about checking your blast radius before you change something. It was written into the part of the flow that hands work to a sub-agent. The rules also say small work should not be handed to a sub-agent. So the rule was guarding the one path the work never took, which is a very tidy way of guarding nothing.

A check that runs at the end of every single turn, reporting unfinished work, stale documents and anything that looks like a leaked credential. It had been running perfectly for months, into a stream nothing reads. A smoke alarm wired to a speaker in the garden.

A dependency scanner reporting full coverage while losing a third of its edges, because it read </span> as the start of a regular expression and ate the rest of the file. Both projects had been quietly missing call sites for weeks.

Those are the bugs I expected to find. The one that mattered came from a count.

535 records, zero documents

One project had stored 535 decisions, lessons and issues. It had written zero documents. It had 140 lessons marked open and none marked closed.

Nothing was broken. Every record was real, written the moment something was decided, correctly stored, instantly searchable. The framework did precisely its job.

535 records is not memory. It is a storage unit I am paying rent on. And 140 open lessons is not learning, it is a shelf.

Reading those two numbers is when I understood what had been happening for about six weeks. I had stopped closing loops. I had stopped writing anything down in a form I could read back. And I had kept saying yes to output because it looked reasonable, which is a different activity from checking whether it was right.

Please make a site like Facebook, don't make mistakes

That is, functionally, the prompt I had been writing. Not in those words. In effect.

When a project fits in your head, review is just reading. You spot the wrong thing because you remember what right was. Once the project outgrows your head, you carry on reviewing the same way and it stops working, without announcing that it has stopped. You read the diff, it looks fine, you approve it. At that point you are not approving the work. You are approving that it did not obviously fail.

Both games cleared that bar for weeks. Both are now larger than my ability to say whether they are any good.

The framework cannot fix this and should not try. It can tell me a sentence in a document no longer matches the code. It cannot tell me the pacing is dull, or that the feature I approved on Tuesday contradicts one I approved in June. That was always my job, and I had quietly subcontracted it to vibes.

The tool got better at producing. It did not get better at judging

Here is what I had backwards for about six months. I thought a stronger tool let me work above my level. It does the opposite. Every upgrade raises the ceiling on what I can produce and leaves the ceiling on what I can tell apart exactly where it was. The space between those two is where projects go to die, and every capability release makes it roomier.

So the interesting work stopped being model choice or rule wording. It is closing that gap on my side, on purpose, the way I would close a bug.

I am splitting it into two piles.

The soft pile is unglamorous and probably the important one. Writing a spec a stranger could build from. Knowing enough about how large systems are normally arranged to recognise a bad shape on sight, instead of building it first and finding out in month three. Being able to state what done means before the work starts, in words that can be checked. That is a writing skill, and I have been treating it as optional.

The hard pile is about where checks live. Every real defect this month had the same shape: something was wrong, nobody was looking, and the not-looking was invisible. So the question is narrow. For a given piece of work, what is the cheapest check that fails loudly when it goes wrong, and where does it have to sit so that neither I nor the agent can walk past it. Not more rules. Rules lost every round this month. Checks that run.

What actually happens next

Both games still exist. Neither is being rescued by a framework upgrade.

The next task is smaller and much less fun than shipping. I am going to write down what each game is, in a form I can read in five minutes, because right now I cannot do that for either of them. Until I can, I have no business judging anything an agent hands me, and no version of the tool will do it for me.

The framework was not the bottleneck. It was the instrument that showed me where the bottleneck was sitting.