Sylvain Gugger

Machine-learning engineer at Jane Street, focused on training performance: profiling models, eliminating GPU bottlenecks, and building custom infrastructure for financial time-series data. Before Jane Street, he was a core maintainer of Hugging Face’s Transformers library and created Hugging Face Accelerate — a library that removes boilerplate from distributed training while keeping researchers in explicit control of where data lives and how it moves. Earlier still, he worked at fast.ai alongside Jeremy Howard, co-authoring Deep Learning for Coders with fastai and PyTorch and contributing to the DAWNBench competition entry that temporarily held first place before Google’s last-minute release of public TPUs.

Gugger came to machine learning from mathematics education — a decade teaching first-year university mathematics in France — and carries that background into his current role running ML training boot camps for traders and researchers at Jane Street who are new to the field.

Core positions

Gugger’s recurring theme is that ML performance is opaque by design: the same Python line can be fast or catastrophically slow depending on whether it triggers a hidden synchronisation between the CPU and GPU, and the only reliable way to know is to profile. He advocates explicit control — as in Accelerate — over black-box abstraction, for the same reason he prefers PyTorch over compiled frameworks: when something goes wrong, you can see and fix it. On the research side, he is direct that machine learning remains poorly understood: no one has a satisfying theoretical account of why neural networks generalise to new data, which means training is still closer to cooking than engineering.

In the wiki

• Sylvain Gugger on The Uncertain Art of Accelerating ML Models — Signals and Threads, 14 October 2024: GPU performance engineering, the ML programming-language ecosystem, and the peculiarities of training on financial data at Jane Street