Research Taste

How a researcher decides which directions are worth pursuing, before the evidence is in. Ilya Sutskever — co-author of AlexNet, GPT-3, and much of the modern paradigm — describes his own as an aesthetic: ‘thinking about how people are, but thinking correctly.‘

An aesthetic of how AI should be

Taste, for Sutskever, is a top-down sense of what is fundamental, drawn from correct inspiration by the brain. The artificial neuron is a good idea because the brain has many neurons (the cortical folds ‘probably don’t matter’, but the neurons clearly do); distributed representations, and learning from experience, follow the same logic. The screening test is whether an idea has ‘beauty, simplicity, elegance, correct inspiration from the brain’ — all present at once. ‘Ugliness, there’s no room for ugliness.‘

Why taste matters: surviving contradicting data

The practical payoff is conviction under failure. A top-down belief ‘is the thing that sustains you when the experiments contradict you’. Because a contradicting result might merely be a bug, you need an independent reason to believe — ‘something like this has to work, therefore we’ve got to keep going’ — to decide whether to keep debugging or abandon the direction. Trusting the data alone leaves you unable to tell a wrong idea from a buggy implementation of a right one.

Related judgements from the same conversation: prefer diversity of thought over copies (‘a million Ilyas’ hit diminishing returns; ‘you want people who think differently’); and beware how single words like scaling, AGI, and pre-training quietly dictate what a whole field works on (see Age of Research).

See also

• Ilya Sutskever on the Age of Research, Continual Learning, and Alignment — source episode
• Ilya Sutskever — speaker
• Age of Research — taste matters most when the field is idea-bottlenecked
• Product Taste — the product-side analogue of cultivated judgement