Reading Notes

Katie Dill on Design Quality, the ROI of Design, and Operationalising Excellence

Source: Katie Dill on Design Quality, the ROI of Design, and Operationalising Excellence

Notes — Katie Dill on Design Quality, the ROI of Design, and Operationalising Excellence

Four questions [Adler frame]

Q1. What is it about? A practitioner account of how to make the case for design investment, how to operationalise quality at scale, and how to build design organisations that survive hypergrowth. The episode moves from philosophy (why beauty matters in B2B) through measurement (the 10.5% checkout revenue lift) to specific systems (the 15 critical journeys programme, friction logs, PQR calibration) to leadership (the trust story, the performance = potential minus interference formula).

Q2. How is it argued? Primarily through Dill’s own experience across three hypergrowth companies. The Airbnb trust intervention story grounds the leadership section in failure. The Stripe checkout data (10.5% revenue lift, 99% of top e-commerce sites have checkout errors) grounds the ROI argument in evidence. The five levels of quality and the journeys programme are structured frameworks, not ad-hoc observations.

Q3. Is it true? The checkout data is plausible and consistent with CRO industry experience. The “beauty increases trust” mechanism is well-supported in psychology (aesthetics-usability effect, Kurosu & Kashimura 1995). The performance = potential − interference formula is a useful heuristic though not a precise measurement tool. The claim that beauty and functionality are not opposites is solid conceptually; the harder empirical question is how to weigh beauty investment against feature velocity in specific contexts.

Q4. What of it? Product and engineering leaders who discount design because it is “hard to measure” have a direct counterargument here: the checkout revenue example shows measurable impact. More usefully, the journeys programme and PQR calibration offer a replicable system that does not require exceptional individual taste — it embeds quality accountability into the management process.

Glossary

Critical journeys — the 15 most important user flows at Stripe, each assigned to a cross-functional owner responsible for monitoring quality. Not a comprehensive list; a tractable set that covers the highest-impact paths.

Friction log — a stream-of-consciousness record kept while walking through a product as a user would. Captures both friction and delight. Filed with severity tags. Katie Dill’s team uses templates with structured scoring rubrics. See Friction Log.

Walk the store — Dill’s term for the act of going through a product journey as a user, without special access or insider knowledge. The phrase comes from retail practice (a store manager walking the floor). At Stripe: quarterly by critical-journey owners, more informally and frequently by Dill and David Singleton.

Product Quality Review (PQR) — the calibration meeting where critical-journey owners present their friction-log findings and summary colour scores, and senior leaders debate whether the scores are correctly calibrated. Analogous to a performance-review calibration session.

Performance = potential − interference — a leadership formula Dill attributes to her Airbnb years. Leaders improve performance by (a) hiring and developing high-potential people and (b) removing barriers (process friction, unclear priorities, poor tooling, misalignment) that prevent potential from converting to output.

Meticulous craft — one of Stripe’s stated operating principles. Applied across all functions — engineering, design, support, internal tools. The principle signals that quality is not solely a design-team concern.

Key passages

On beauty and trust in B2B [§ ROI of design section] “Things that are more beautiful, increase trust. You see that we’ve put painstaking detail into this, and we care about the details of how something works, and that gives you assurance that we care about other details that you can’t see too.” — The B2B argument for beauty investment rests on this trust mechanism, not on aesthetic preference.

On the checkout revenue lift [§ ROI section] Stripe found that 99% of top e-commerce sites have checkout errors that hinder conversion. Quality improvements to the checkout experience produced a 10.5% increase in business revenue moving from an older to a newer checkout form. Individual quality decisions are hard to attribute; the cumulative result is material.

On levels of quality [§ What is great design section] Dill describes at least four levels: (1) the thing works; (2) it works error-free and well-rounded; (3) it exceeds expectations; (4) it does something the user wasn’t even seeking. She argues prioritisation debates about quality should be anchored to user expectations, not abstract craft standards.

On why quality regresses [§ Journeys programme section] Products shipped at high quality degrade because (a) other features ship around them, making older parts look worse by comparison, and (b) focused teams lose sight of the full journey, optimising their piece while neglecting cross-boundary seams.

On the trust story [§ Opening section] Dill came into Airbnb’s design team with an agenda, “ready to go,” and within a month received a formal intervention from five members of the team who read from prepared papers listing what she was doing wrong. The theme: she had not earned trust. She did not respond defensively; she listened, adjusted her approach, and brought the team’s engagement scores to the highest in the company within a couple of months.

Connections to other wiki pages