Andrew Hunter on Performance Engineering on Hard Mode

Andrew Hunter — performance engineer at Jane Street — joins host Ron Minsky to contrast the abundant, scale-driven optimisation work at Google with the ‘hard mode’ problem of extracting latency from trading systems, and to argue that the real foundation of the craft is measurement, mechanical sympathy, and disciplined prioritisation.

Key ideas

1. Easy mode versus hard mode performance. At hyperscalers such as Google, the sheer volume of compute means any improvement to shared infrastructure — memory allocation, logging, serialisation — pays for itself instantly; Hunter calls this ‘easy mode’ because hotspots are obvious and half-percentage-point wins translate to real money. At Jane Street the scale is smaller and trading systems spend 95–99% of their time deliberately idle (waiting on the network in a busy-loop), so the average CPU cycle is nearly worthless. What matters instead is what happens in the brief, critical windows when a market event arrives: latency, not throughput.

2. Profiling versus tracing: two fundamentally different instruments. A sampling profiler (such as Linux’s perf) interrupts the programme every hundred or so microseconds and records where it is — like taking a random photograph to build a statistical portrait of the subject. This reveals what the system does overall, but tells you nothing about when specific events occurred relative to one another. A tracer such as Magic Trace (Jane Street’s own tool, built on Intel Processor Trace — a hardware feature that logs every branch in a ring buffer in silicon) captures the exact sequence of function calls over the last two or three milliseconds. Hunter illustrates the difference: a profiler showed a perfectly cheap ‘send order’ function, but the tracer revealed the order was dispatched to a low-priority queue and actually sent 200 microseconds later — a misconfiguration invisible to statistical sampling.

3. Mechanical sympathy: coding with the hardware in mind. The phrase — Hunter credits it to a racing driver — means having an unconscious feel for what the CPU actually does when it executes your code. Every computer is, at bottom, one large array of memory and integer arithmetic; high-level languages impose abstractions on top, but the hardware has only one model of reality. A skilled performance engineer looks at code and instinctively knows whether it will cause cache misses (expensive trips to slow memory), branch mispredictions (the CPU guessing the wrong path through an if-statement and having to backtrack), or unnecessary heap allocations (requests to the memory manager that stall execution). OCaml’s ‘boxy’ representation — pointers to pointers rather than flat memory layouts — raises the cognitive cost of achieving this sympathy, but does not make it impossible.

4. The single most important performance optimisation is doing less work. Both Hunter and Minsky return to this repeatedly: before tuning a function, ask whether it needs to be called at all, whether the data could be pre-processed upstream, or whether the architecture can be restructured to avoid the path entirely. Hunter estimates that the hard part of a two-line change that makes a system 10% faster is the three weeks of investigation proving the change is safe. Micro-optimising a loop body — squeezing better code generation from the compiler — is the least important category; avoiding the loop entirely is the most.

5. The OODA loop as a mental model for iteration speed. Hunter is preoccupied with the speed of the feedback cycle. He invokes the OODA loop (Observe, Orient, Decide, Act) — a framework from 1950s US Air Force colonel John Boyd — to argue that whoever closes the loop fastest gains control of any dynamic system. In performance engineering this means: a testbed that tells you in ten minutes whether a change is faster beats one that requires two days in production. The same principle applies to trading strategy research (an hour of simulation versus a day), and Hunter extends it to whiskey distillers using rapid-ageing technology — heretical to purists, but giving twelve iterations in the time a traditional distillery completes one.

Content

The texture of ‘easy mode’ versus ‘hard mode’

Hunter spent his career before Jane Street at Google, where the scale of a hyperscaler creates what he calls a ‘target-rich environment.’ A paper he returns to, Profiling a Warehouse-Scale Computer, coined the term ‘data centre tax’ for the 10–20% of cycles every application spends on shared infrastructure — logging, serialisation, memory allocation — regardless of what the business logic does. Optimise any of those and you improve every service simultaneously; even a half-percentage-point gain is worth real money at millions of machines.

Jane Street does not operate at that scale. More importantly, its latency-sensitive trading systems are architecturally idle by design: they spin in a hard busy-loop on the CPU, polling the network for incoming packets and doing nothing most of the time. Point a profiler at such a system and it reports 99% of time in ‘doing nothing.’ The question is not what the system does on average but what it does in the precise nanosecond a market event arrives. That reframes performance entirely: from throughput and aggregate statistics to tail latency and event-level sequencing.

Profiling tools: perf, memory profilers, and their limits

Hunter treats profiling as primarily a measurement problem. A sampling profiler like perf uses hardware interrupt counters — not just elapsed cycles but also L2 cache misses, branch mispredictions, or any other architectural event — to snapshot the call stack at regular intervals. The resulting flame graph (or the directed-acyclic-graph view of pprof, which Hunter prefers) shows where time is spent in aggregate.

Hunter and Minsky spend time on why pprof beats flame graphs for certain problems. Flame graphs — columns of function names whose width is proportional to time — are intuitive at a glance, but they miss what Hunter calls ‘join points’: dispersed small costs that each look negligible but converge on a single function. pprof draws the call graph as a DAG with arrow weights, making the convergence visible. In C++ the canonical join point is malloc; every bit of business logic allocates a little, and the aggregate — invisible in a flame graph — can dominate.

Memory allocation profilers add a complementary dimension. OCaml’s allocator (and various C++ ones) can record a profile of where allocation is occurring, distinct from the cycle profile. In high-performance OCaml work, the goal is sometimes to reduce heap allocation to zero on the hot path; an allocation profile locates the remaining sources.

Magic Trace: retrospective hardware tracing

The core gap in sampling tools is that they give statistical portraits, not event timelines. Intel Processor Trace — a feature baked into Intel silicon, not software-emulable — maintains a ring buffer of every branch the CPU takes, compressed to roughly one bit per branch. The buffer holds a few milliseconds of history and can be read at any moment without stopping the programme between samples. Jane Street built Magic Trace on top of this facility to make it accessible: a single command captures a snapshot of what the programme was doing in the preceding two to three milliseconds, producing an exact function-call trace visualised as a timeline rather than a histogram.

Hunter notes the tool’s practical superiority over perf for certain classes of problem: a profiler reporting ‘40% of time in send_order’ cannot distinguish one call that takes forever from a thousand cheap calls in a tight loop. Magic Trace shows the difference immediately — the tight loop appears as a dense tower of repeated calls on the timeline. It also surfaces sequencing problems that have no footprint in a profiler at all: the misconfigured queue delay he describes looked perfectly fast in isolation but was catastrophically slow in context.

The overhead is not zero (5–15% of programme performance with Processor Trace enabled), so Magic Trace is activated on demand rather than left running. Hunter notes wistfully that hyperscalers do run always-on sampling profiles across entire fleets, giving them a global hotspot view; that luxury is less valuable but still desirable at Jane Street’s scale.

Visualisation as a first-class discipline

Hunter estimates that a substantial fraction of his day-to-day work is looking at or building visualisations. The standard tools — flame graphs, pprof DAGs, Magic Trace timelines — each reveal different structure in the same underlying data. A persistent frustration is that expert tooling like perf was built by and for people who have already memorised esoteric flag semantics; it has no obvious defaults, and the knowledge to use it well is primarily acquired through repetition rather than documentation. Magic Trace was a deliberate attempt to do better: one obvious flag, sensible defaults, a usable result without deep prior knowledge.

OCaml as a performance engineering challenge

Jane Street writes virtually all of its systems in OCaml, a garbage-collected functional language. Hunter identifies three layers of performance friction.

The first — code generation quality — he dismisses as real but minor. OCaml’s compiler has had thirty fewer years of optimisation investment than clang, and it shows in register spills and suboptimal loop generation, but this is rarely what determines system performance.

The second — garbage collection costs — is more concrete. OCaml’s GC must scan the heap at runtime, which forces any array to be initialised (null pointers rather than raw uninitialised memory) and makes uninitialized data dangerous. The costs are real in specific scenarios but manageable.

The third — memory layout — is Hunter’s deepest concern. OCaml’s default representation is ‘boxy’: composite types are arrays of pointers to heap-allocated sub-objects rather than flat structs. Where C++ would splat fields inline, OCaml indirects through a pointer, increasing cache footprint and reducing spatial locality. The language does not easily express the programmer’s mechanical sympathy. Workarounds include DSLs (Jane Street has packet-parsing DSLs that generate flat, tightly packed memory representations from high-level descriptions), a ‘zero-alloc dialect’ that avoids the GC entirely on hot paths, and thin C stubs for performance-critical library calls (C FFI at Jane Street costs roughly three to four nanoseconds, versus three to four hundred in Java’s JNI).

Hunter is genuinely excited about upcoming compiler work to give programmers explicit control over OCaml type layout — what he calls potentially ‘the biggest change that maybe will ever happen in the compiler.‘

Hardware, FPGAs, and the architecture of latency

At the physical limit, a software system on an ordinary computer must traverse the PCIe bus twice per packet — roughly 400 nanoseconds each way — making it nearly impossible to achieve sub-microsecond round-trip latency in software. An FPGA wired directly to a network card can turn a packet in under a hundred nanoseconds. Hunter’s view is not that CPUs are therefore irrelevant but that hardware handles simple, speed-critical decisions while software handles the complex logic that hardware cannot express. The art is partitioning: send the simplest fastest decisions to hardware, keep the complicated business logic in software, and make the software fast on its own timescale — which might be tens of microseconds rather than tens of nanoseconds, but still rewards all the same disciplines.

He extends the latency argument to human-interactive systems: a trading display that freezes for five seconds while a trader is watching prices imposes real costs, and a historical research system that returns in one hour rather than one day changes the quality of the intellectual work. Latency matters at every timescale from nanoseconds to days.

Related

• Andrew Hunter — speaker
• Ron Minsky — host
• The Future of Software Engineering — theme; the episode’s argument that mechanical sympathy and measurement discipline define expert software work sits squarely in this theme