Building Products on AI Models

This is a narrative theme, with a typological spine, synthesising how to build durable products on top of foundation models — drawing on Nick Turley on ChatGPT, Boris Cherny on Claude Code, Chip Huyen on AI Engineering, Kevin Weil on OpenAI and the Future of AI Products, Aparna Chennapragada on AI Product at Microsoft, Sam Schillace on Innovation and the Future of AI Products, Dan Shipper on Every, AI-Native Company Building, and the Allocation Economy, Benedict Evans on the 1997 of AI, Task vs Job, and Where AI Value Accrues, and a dozen further sources.

The core claim

Building AI products is not building software with an AI feature. The structure is different at every level: the build loop, the unit of progress, the way you learn what to build, and where the durable moat sits.

Four claims recur across sources and form the backbone of this theme.

The model is the product, not a component. Nick Turley’s formulation at OpenAI is unambiguous: ‘There really is no distinction between the model and the product. The model is the product.’ Boris Cherny’s Claude Code architecture follows the same logic — minimal scaffolding, the product is the model, tools provided and execution sequence decided by the model. Sam Schillace at Microsoft arrives at the same point from a different direction: ‘AI isn’t a feature of your product; your product is a feature of AI.’ The platform shift is total.

Evaluations replace specifications as the unit of progress. In deterministic software, a test tells you whether a function returns the right value. In AI products, the model’s output is probabilistic and context-dependent — the ‘right’ answer is often a judgement call. Evals convert a vague sense of ‘the model seems better at X’ into a repeatable measurement. Kevin Weil’s framing is the sharpest: ‘AI is capped by the quality of its evals.’ Nick Turley discovered he had been writing evals before he knew the word existed — it was just articulating very precisely what success looked like. The unit of product progress has shifted from a feature shipped to an eval passed.

You must ship before you can know. AI product properties are emergent — they cannot be reasoned about in advance. Nick Turley’s forcing-function question, ‘Is it maximally accelerated?’, encodes this: ‘You’re going to be polishing the wrong things in this space. You won’t know what to polish until after you ship.’ Kevin Weil calls this the fundamental break with deterministic software: ‘Your database works once, it works every time. And that’s not true in this world.’ The correct operating posture is empiricism — ship, observe, iterate — not specification, then build, then ship.

The defensible moat is not the model. Foundation models are heading toward a commodity utility, as Benedict Evans on the 1997 of AI, Task vs Job, and Where AI Value Accrues argues: no winner-take-all network effects, persistent competition, no pricing power. Value moves up the stack. The durable moat in the application layer is built from proprietary data and evals, workflow depth and integration, distribution, and trust.

Evaluations as the new specification

The most operationally important difference between AI product development and conventional software development is what ‘knowing what to build’ means.

In classic software, a PM writes a specification; engineering implements it; tests confirm the implementation. In AI products, the specification is only useful once you have seen real model failures in real use. Hamel Husain and Shreya Shankar’s methodology makes this explicit: open coding traces first, axial coding to name failure modes, counting to rank them by frequency. The LLM judge prompt that results is a better PRD than one written without data, because it is grounded in what the model actually does wrong in production rather than what the team imagined it might do wrong.

Evals sit at the intersection of product and model: they are simultaneously a test suite, a training signal, and a product specification. Kevin Weil’s evals-to-fine-tuning loop at OpenAI is the practical realisation of this — identify hero use cases, write what an exemplary answer looks like, encode those as evals, fine-tune against them. The loop repeats.

Two important counterweights constrain eval enthusiasm. Howie Liu’s position is that premature evals are the wrong evals: ‘start with vibes — diverge first, then converge.’ Evals become productive once you have seen enough real use to know what to measure. Chip Huyen adds the ROI argument: the decision to invest in evals is a resource allocation question, not a moral one. High-stakes, high-scale products where AI capability is a genuine competitive differentiator need evals; lower-stakes features where failures are recoverable may not.

Empiricism as the operating principle

The empiricism claim is not a speed preference. It is a structural claim about what kind of knowledge is available before shipping.

Nick Turley at ChatGPT, Alexander Embiricos at Codex, and Boris Cherny at Claude Code all describe the same pattern: planned features do not match the use cases people actually bring. ChatGPT users discovered hundreds of uses through TikTok threads before OpenAI’s team had categorised them. Codex’s first cloud-only form had poor adoption because it was too far ahead of where developers actually worked; the 20x growth came from meeting users in the IDE they already used. Claude Code was used to analyse genomes and recover corrupted wedding photos — signals that seeded Cowork, the non-engineering product Boris was not planning to build.

Karina Nguyen on Model Training and AI Product makes the same point at the training layer: product features are taught to the model synthetically by first observing the three or four core behaviours users actually want, then building training examples from those. Form follows observed function, not anticipated function.

The implication for product teams is that the planning horizon is compressed and the feedback loop — deploy, observe, classify real failure modes, iterate — is where the real specification happens.

The model commoditisation problem and the application-layer moat

If the model is the product and models are heading toward a commodity, what does building a durable product mean?

Benedict Evans’s analysis is the structural diagnosis: frontier models show no winner-take-all dynamic, so competition persists, so pricing power erodes. Value moves up-stack. The shape to expect is cloud (AWS low-margin infrastructure), not Windows (platform lock-in). The practical corollary is Model Commoditisation: foundation-model APIs are infrastructure; the leverage is with whoever builds on top and owns distribution.

The sources suggest four defensible positions at the application layer.

Proprietary data and evals. Every domain contains knowledge the model was not trained on: company processes, proprietary workflows, subject-matter expertise specific to context. Custom evals encode what ‘good’ means for a specific use case; fine-tuning injects domain knowledge the base model lacks. Kevin Weil: ‘Most of the world’s knowledge — company processes, domain expertise, proprietary data — is not in model training sets. Models can learn anything, but they need the data.’ This data advantage compounds as the product matures.

Workflow depth and forward-deployed integration. Bret Taylor at Sierra argues the applied layer — agents embedded in customer workflows — is where durable competitive value sits, because the moat is organisational context and workflow knowledge, not model quality. The pattern echoes the Forward Deployed Engineers model at Palantir: build the product at the customer’s site, acquire the ‘domain secrets’ that neither the customer nor the platform vendor had articulated, and encode them into the system. Nabeel Qureshi’s account of how Palantir Foundry’s Ontology layer emerged from deep customer engagement is the pre-AI prototype of this. AI coding agents lower the cost of the same motion — one forward-deployed engineer can now do what previously required three.

Distribution and trust. ChatGPT had no meaningful distribution advantage from model quality alone — GPT-3.5 had been available via API for six months before launch. Making it free, wrapping it in a clean UI, and removing the login requirement were the decisive moves. At scale, distribution compounds: 700 million weekly active users means data on more use cases, better evals, better fine-tuning, and a harder-to-replicate advantage.

Interface design. Aparna Chennapragada’s NLX (natural language experience) framing names a discipline that barely exists yet: designing the invisible grammar of AI interfaces — prompts as UI constructs, editable plans, showing work, proactive follow-ups. The interface design challenge has shifted from ‘how do we surface the feature?’ to ‘how do we understand what the user actually wants?’ Products that solve the intent-inference problem have a structural advantage that pure capability does not.

Where the thin wrapper argument is honest and where it breaks down

There is a genuine tension in the ecosystem between products that are defensible applications and products that are ‘GPT wrappers’ — thin interfaces around a base model with no proprietary data, no workflow depth, and no interface innovation.

Dan Shipper explicitly defends the wrapper: ‘GPT wrappers are amazing and they’ve been much maligned for absolutely no reason.’ His argument is that the wrapper is the product if it packages the capability for a specific context — in the same way Craigslist unbundled into many verticals that were each more valuable than Craigslist itself.

Benedict Evans’s counter is structural, not moral: when foundation models commoditise, a thin wrapper has no pricing power and no advantage that the next model release cannot erase. The wrapper’s only durable advantage is distribution — which is real, but requires being distributed.

The resolution is in the definition of ‘thin’. A thin wrapper is one with no proprietary data, no custom evals, no workflow depth, and no interface innovation. A wrapper that acquires any of those four becomes something else. The question for any AI product is not ‘am I a wrapper?’ but ‘what have I built that compounds as I operate?‘

The team and organisation

AI-native product development changes the team as much as the product.

The job boundary between engineer, designer, and PM is eroding. Boris Cherny’s report from the Claude Code team: PM, engineering manager, designer, finance, and data scientist all code. Mike Krieger observes that at Anthropic, the bottleneck in a 90%-AI-written codebase has moved upstream — to deciding what to build and aligning stakeholders — and downstream — to code review, the merge queue, and shipping coherence. When code generation is unlimited, the critical path is judgement and alignment, which is precisely what product people and product-embedded researchers provide.

Dan Shipper’s allocation economy thesis generalises this: as AI agents do knowledge work, humans shift from doing to allocating — scoping tasks, evaluating outputs, giving feedback, holding vision. These are management skills, currently rare. They become universal.

Amol Avasare’s account of growth at Anthropic shows what AI-native PM work looks like operationally: automated morning dashboard reviews, anomaly surfacing before the day begins, weekly alignment scans that cross-reference Slack and OKRs, and growth hypotheses generated by Claude itself. The role has shifted from ideating tests to reviewing outputs and setting direction.

The remaining durable human contributions, across sources, converge on three: making AI comprehensible to non-power-users; strategy — deciding what to build and where to play; and opening people’s eyes to what is possible. Model capability does not automate any of these.