Chris Lattner on Why Machine Learning Needs a New Programming Language

Chris Lattner — creator of LLVM, Swift, and Mojo — joins Ron Minsky on Signals and Threads to explain why the fragmented landscape of GPU and AI accelerator hardware demands a new programming language, and why every existing tool either sacrifices portability or leaves performance on the table.

Key ideas

1. AI hardware is fragmented in a way that existing languages cannot fix. Every GPU and accelerator vendor — Nvidia, AMD, Google — has built its own software stack (CUDA, ROCm, XLA) because there is no shared foundation analogous to LLVM for CPUs. Think of LLVM as a universal translator that lets a compiler produce code for many CPU chips; no equivalent exists for GPUs. The result is that software written for one vendor’s hardware cannot easily run on another’s, and the costs of this incompatibility compound as new chips arrive every year.

2. The ‘sufficiently smart compiler’ trap keeps failing. One intuitive response to hardware fragmentation is to build a clever compiler that automatically transforms high-level Python or PyTorch code into fast GPU instructions — a ‘magic box.’ Lattner argues this approach repeatedly breaks down: the compiler works on demos and simple cases, but any change to the input code risks destroying performance or crashing entirely. The alternative Mojo pursues is to give programmers explicit control — rather than hiding complexity behind magic, expose it through a clean, expressive language where the programmer decides what happens.

3. Metaprogramming is the key to portability without a performance penalty. Metaprogramming means writing code that runs at compile time (when you build the programme) rather than at run time (when you execute it), generating specialised, stripped-down machine instructions for each target chip. Think of it as the difference between a law firm producing a bespoke contract for each jurisdiction versus one generic contract full of hedges that works adequately everywhere. Mojo unifies compile-time and run-time code into a single language — unlike C++ templates, which are notoriously hard to debug — so engineers can build powerful hardware-specific libraries without a separate, cryptic meta-language.

4. Mojo is Pythonic by design, but typed and fast. Python dominates the AI/ML world; Mojo preserves Python’s syntax and feel while adding a proper type system — the set of rules a compiler uses to catch errors before the programme runs — and the performance characteristics of a systems language. A Mojo programme can run the same code on a CPU or a GPU without rewriting it; the compiler handles specialisation. Lattner compares this to Java’s portability story, but with full hardware performance rather than Java’s managed-runtime compromise.

5. Modular’s business is not selling a language — it is fixing the infrastructure. Lattner is explicit that building a programming language is commercially perilous; Modular gives Mojo away free. The revenue model targets the enterprises running hundreds or thousands of GPUs in production: those teams need a unified software platform that works reliably across hardware vendors, rather than the current patchwork of incompatible stacks. Mojo is the foundation that makes a portable, high-performance runtime (MAX) and cluster management layer (Mammoth) possible.

Content

From LLVM to MLIR: a career at the hardware-software boundary

Lattner traces his career as a series of upward moves from deep infrastructure toward higher-level problems, each step enabled by the layer below. LLVM — begun as a university project in 2000 and adopted at Apple from 2005 — unified compiler technology for the CPU era: it provided a common back-end that languages like Rust, Swift, and Julia could all build on. Having established that foundation, Lattner moved up to tackle C itself, replacing the existing C/C++ compiler with Clang, and then up again to design Swift as a modern replacement for Objective-C.

When AI hardware arrived at scale — GPUs, tensor processing units, field-programmable gate arrays, application-specific integrated circuits — LLVM did not carry over. Lattner built MLIR at Google as a next-generation compiler infrastructure suited to this new compute class. MLIR is now embedded in nearly every major AI software stack, including Nvidia’s and Google’s. But MLIR is infrastructure, not a programming surface. The fragmentation problem persisted one level above it.

Why the hardware fragmentation problem is structural

The structural diagnosis is precise: hardware companies each build their own software stack because they have to, and because nobody else is incentivised to do the common work. Nvidia has a 20-year head-start with CUDA. AMD built ROCm largely as a CUDA clone. Google’s XLA targets TPUs. High-level frameworks like PyTorch or JAX sit on top of these incompatible foundations, which is why cross-hardware support is always incomplete and always lagging behind new silicon.

Ron Minsky observes that even within a single vendor’s line, the hardware changes materially between generations: Nvidia’s Volta, Ampere, Hopper, and Blackwell architectures each alter the performance characteristics of tensor cores (the dedicated silicon for matrix multiplication that modern AI workloads depend on), and Blackwell broke compatibility with some Hopper kernels outright. Writing code that performs well across this space is the problem Mojo was designed for.

Metaprogramming: one language for compile time and run time

The episode’s technical centrepiece is metaprogramming. A conventional GPU library like CUDA or Triton requires the programmer to write manually tuned ‘kernels’ — small, intensely optimised programs that run on the GPU — for each hardware variant. Mojo’s alternative is a metaprogramming system that lets engineers write a single generic algorithm (say, matrix multiplication) and have the compiler automatically specialise it for the target chip at build time.

C++ has a similar concept — templates — but Lattner and Minsky agree that C++ templates are effectively a second language bolted onto the first, with impenetrable error messages, catastrophic compile times, and no meaningful type-checking until the template is instantiated. Mojo’s insight is that compile-time and run-time computation are the same problem executed at different moments, and should use the same syntax, the same type system, and the same debugger. The result is what Lattner calls LayoutTensor: a library abstraction that declaratively describes how tensor data is laid out in GPU memory and accessed by the hardware, parameterised over all the vendor and generation differences, with all of that complexity resolved at compile time so there is no run-time overhead.

Traits, type-checking, and the modularity problem

A second design pattern that distinguishes Mojo from C++ is its trait system — analogous to protocols in Swift, traits in Rust, or type classes in Haskell. A trait is an interface: a named set of behaviours (operations an object must support) that a type can promise to implement. Writing generic code against a trait means the compiler can check the generic code in isolation, before it is ever instantiated with a concrete type.

The contrast with C++ templates is sharp. In C++, the compiler cannot check a template until it sees a concrete instantiation; errors propagate through layers of expansion and produce error messages that span hundreds of lines. Mojo checks generic code against its trait at parse time, so errors are caught early, reported clearly, and confined to the level where they actually occur. This also dramatically reduces compile times, because the compiler does not need to regenerate identical code for every concrete type.

Portability: the Mojo package model

Mojo’s portability story borrows the best element of Java — a portable intermediate representation that delays final machine-code generation until the deployment target is known — while discarding Java’s managed runtime. When a developer builds a Mojo package (as opposed to a standalone executable), the compiler produces a representation that retains the full generic type system and metaprogramming annotations. When that package is later deployed to an AMD GPU or an Nvidia Blackwell card, the final specialisation happens then.

Lattner contrasts this with Nvidia’s PTX (Portable Thread Execution), an intermediate bytecode that also defers final compilation to the driver. PTX’s type system is thin — essentially registers and pointers — so the programmer has no control over how specialisation proceeds; the driver does whatever it does. Mojo packages retain the programmer’s high-level type information and metaprogramming instructions, so the final compilation is an execution of the programmer’s decisions, not an opaque driver heuristic.

Language design philosophy: go and the 80-20 rule

On managing complexity, Lattner cites Go as an underappreciated model. Go was mocked for launching without generics, but it held its design discipline through the criticism, added generics deliberately in Go 2, and has maintained what Lattner describes as an 80-20 rule: 80% of features for 20% of the complexity. The 81st percentile of features, he notes, often costs a disproportionate amount of complexity — a principle he is trying to apply at Modular, where the core team is intentionally small and the design process is deliberately closed (unlike Swift’s open proposal process, which he credits with producing good ideas and feature bloat in roughly equal measure).

On backwards compatibility, Lattner favours semantic versioning — 1.0, 2.0, 3.0 — with the ability to link packages across versions, analogous to C++‘s ABI stability. The Python 2-to-3 migration, which took 15 years because the entire package ecosystem had to convert simultaneously, is the anti-pattern he wants to avoid. Strong types, he agrees with Minsky, are the mechanism that makes AI-assisted migrations tractable: you can ask a coding agent to chase down all the type errors produced by a change and fix them, whereas the same exercise in Python requires exhaustive runtime testing.

AI coding and the future of Mojo

Lattner makes a practical point about AI coding tools that cuts against the conventional wisdom that nobody adopts new languages. Mojo’s public repository of hundreds of thousands of lines of GPU kernel code gives models like Claude a large, indexable corpus; developers who configure their coding tools to index that repository get working Mojo code from hackathon-day-one without formal training in the language. The ‘nobody will learn a new language’ assumption, he argues, was true in 2015 and is simply false in 2025.

Looking forward, Lattner sketches a roadmap: Mojo 1.0 approaching; runtime traits (existentials) to be added, making Mojo a credible Rust alternative for application-level programming; classes to follow, which will make the language feel like Python 4 to Python programmers; and, over a longer horizon, enough CPython compatibility to be perceived as a Python superset. Each milestone is conditional on the market pull from the existing short-term use cases — performance code on CPUs, GPU kernels, and fast Python extensions — rather than a speculative long-run bet.

Related

• Chris Lattner — speaker
• Ron Minsky — host
• The Future of Software Engineering — theme; Mojo’s approach to AI-assisted language adoption and explicit-control programming speaks directly to how software engineering is changing
• Software 1.0, 2.0, 3.0 — concept; Lattner’s framing of Python-plus-C++ as the de facto architecture of modern ML systems, and Mojo as a unification, maps onto Karpathy’s typology
• Vibe Coding — concept; Lattner explicitly positions AI coding tools as a forcing function that dissolves the language-adoption barrier