Concept

Generalization Gap

conceptaigeneralisationlearningagi

Generalization Gap

Ilya Sutskever‘s name for what he calls the most fundamental unsolved problem in AI: ‘these models somehow just generalise dramatically worse than people. It’s a very fundamental thing.’ Models post superhuman scores on hard benchmarks yet fail trivially in the real world — and need vastly more data to learn in the first place.

Two faces of the gap

  • Sample efficiency. A model needs an enormous amount of data; a human learns from a tiny fraction. A five-year-old recognises cars well enough to (later) drive, on very low-diversity data; a teenager drives competently after ~10 hours.
  • Robustness and teachability. A mentored human picks up a way of thinking without verifiable rewards or a bespoke curriculum; a model needs schlepful, hand-built training. ‘The robustness of people is really staggering.‘

Why it is not just evolution

One might attribute human learning to evolutionary priors — plausible for vision, hearing, and locomotion, which mattered to ancestors for hundreds of millions of years. But humans are reliable and robust in maths and coding, domains far too recent for such a prior. That, Sutskever argues, ‘is more an indication that people might have just better machine learning, period’ — a fundamental learning principle the models lack, not a stack of evolved tricks.

Suspected ingredients

  • Value functions. Humans carry an internal estimate of how well they are doing (the teenage driver self-corrects with no external teacher). Today’s RL often learns only from a final graded result; a value function would let credit propagate mid-trajectory. Sutskever ties the human value function to emotions — a simple but extremely robust, evolution-hardcoded signal.
  • Eval-shaped training. Part of the appearance of the gap is organisational: RL environments built to resemble the benchmarks inflate eval scores while real-world performance lags (see Evals, Jagged Intelligence).
  • An undisclosed principle. Sutskever says he has ‘a lot of opinions’ on how humans do it but won’t share them; he flags one possible blocker — human neurons may do more computation than assumed.

The fix, he argues, is not simply more diverse RL environments (more 10,000-hour competitive programmers) but a different approach that achieves reliable generalisation from limited experience. Closing this gap is, in his telling, what separates today’s systems from real superintelligence — and reliable generalisation may even be the same problem as alignment (‘your ability to learn human values is fragile… are these not all instances of unreliable generalisation?’).

See also