Speaker

Jacob Baskin

Jacob Baskin

Software engineer at Jane Street, the quantitative trading firm, where he built Superstore — the firm’s from-scratch analytical data warehouse — and now works on The Hive, its large-scale compute cluster.

Baskin’s path is a study in the same problem seen from different angles. A Brown thesis on mechanism design — the economics of designing rules so that participants do best by being honest — led to Google in 2008, where he spent years on AdExchange, the real-time two-sided market Google built on the DoubleClick acquisition to show ads across the web. A startup followed, helping cities decode and manage the accreted rules of their curbs — a problem he loved and could not turn into a business. Wanting economic thinking, New York, and the company of people he respected (many of them writing OCaml), he joined Jane Street, announced at his team-match that he would build a distributed analytical database, and did.

Core positions

The lesson Baskin took from mechanism design into industry is that incentive-compatibility — the property theorists prize — is a smaller part of the problem than it appears. What dominates in practice is whether people can understand the mechanism, and whether it is simple enough to reason about. A system that hands users results they cannot explain has failed, however elegant its proof.

He designs databases around the trade-offs a specific user base actually needs rather than the full generality a standard offers. Superstore’s defining choices — atomic writes within a single table but not across tables, asynchronous writes that commit before they are readable, no single-row updates, no direct access to the underlying Parquet files — are each a deliberate surrender of generality in exchange for scale, isolation, and the freedom to keep the database a ‘Wild West’ that hundreds of users can throw anything into without breaking it for each other. Being in-house is central to this: because every user is a colleague, the team can teach the model and mediate every access, which is what makes access control, usage logging, and schema evolution possible at all.

On distributed systems he inherits the Google doctrine of containing the hard part and keeping as few components stateful as possible, but he is willing to break it when the numbers say so: replacing a consensus-based metadata store (CockroachDB) with a single-sequencer state-machine-replication system (Aria) trades a distributed design’s high availability for the single-table write throughput the workload actually needs — on the reasoning that a lone, well-tended point of failure costs about a minute every few years, and that real downtime is dominated by software bugs regardless.

In the wiki