Concept

Harness Ablation

conceptclaude-codeai-productevalsprompt-engineering
Teach me this →

Harness Ablation

Harness ablation is the practice of deleting the instructions and tools wrapped around a model — its harness — whenever a new model arrives, then restoring them one at a time only where the new model shows it needs them. Boris Cherny, who created Claude Code, described it at YC’s Startup School in 2026 as the way his team rebuilds Claude Code for each model generation.

A harness is everything between the model and the task: the system prompt, the tool definitions and their prompts, and the code that runs them. Ablation is a research term for removing a component to measure what it contributed. The two ideas combine into a claim about ageing: much of a harness exists to correct what an older model got wrong, so a better model turns those corrections from help into noise. For Opus 5, Cherny’s team deleted 80% of Claude Code’s system prompt.

It is easily confused with two neighbours. It is not a rewrite from scratch — Cherny corrects that reading directly: ‘we don’t delete the entire code base, but we do delete a lot.’ Nor is it the Bitter Lesson itself, which predicts that general methods beat hand-built structure over time; harness ablation is the maintenance routine that acts on that prediction at each model release.

The procedure

Cherny’s version has three steps.

  1. Delete. Remove the system prompt (Claude Code offers a flag for a custom one, and an undocumented ‘simple mode’ that strips every prompt, including the tools’ own).
  2. Use. Run the product, or run Claude Code on your own code base, and watch where the model does well and where it stumbles.
  3. Restore only on repetition. Add an instruction back only when the model fails at the same thing more than once. Adding one earlier is guessing, and every instruction has a cost because the model reads it on every run.

He applies the same routine to tools — ‘we unship tools all the time’ — and to users of Claude Code, who should ‘every six months delete your Claude.md. Delete your skills.’ CLAUDE.md is the file of standing instructions Claude Code loads into every session; skills are packaged procedures it can call on.

Why it works

Two observations from the talk carry the argument. Each model generation behaves differently, so ‘something that you did for one model maybe three months ago, it just might not translate at all to the next model.’ And the extra text is not free: Cherny reports that the model is ‘a little bit more intelligent without these prompts’, while conceding that a product still wants some of them to behave as users expect.

The same logic applies to Evals, only more slowly: an eval survives one to three model generations before the model saturates it and it must be replaced.

Where it stops

Cherny’s own caveat marks the limit: a product keeps the prompts that make it behave as its users want, so the procedure removes corrections, not specifications. A reasonable gloss (the wiki’s, not the talk’s) is that instructions come in two kinds. Some correct the model — ‘don’t do this, which the old model kept doing’ — and a better model makes those obsolete. Others supply facts no model can infer — a project’s conventions, a client’s constraints, a decision already taken — and those do not age with the model. Ablation tests the first kind well; for the second, deleting and waiting to see the model stumble means waiting for an error that may be costly to observe.

In the wiki