Gary Marcus on AI's Overstated Intelligence, LLM Economics, and the Limits of Scaling

Ed Elson

Welcome to Prof G Markets. We've spent a lot of time talking about AI lately — from the Trump administration's export restrictions on Anthropic's models to the ongoing questions surrounding the economics of companies like OpenAI and Anthropic. Taken together, these stories point to two fundamental questions: is the AI boom financially sustainable, and are we moving too quickly with a technology we don't fully understand?

Few people have been asking those questions longer than our next guest. Long before concerns about AI safety, regulation, and business models entered the mainstream, he was warning about the technology's limitations and challenging some of the industry's most ambitious claims. He has testified before the Senate on the risks posed by AI. He founded a machine-learning company that was acquired by Uber, and he is now one of the field's most prominent sceptical voices. So here is our conversation with Gary Marcus — AI sceptic, author, and professor at NYU Stern.

Gary, thank you so much for joining me today. You're one of the original critics of AI, and that's interesting, because you're also part of the AI world — you started a machine-learning company, you've done a lot of AI research. Let's start broad: what are your concerns?

Why Generative AI Is Inherently Unreliable

Gary Marcus

My concerns are that we're all in on a particular technology that I think is inelegant, harmful, not where we should end up, and being abused by the people using it. I want AI to succeed, but I think we've wound up down a really dangerous path. Think about the Star Trek computer: you ask it a question, it gives you an answer you can count on, and presumably it's not built to wreck society but to help people. What we actually have is everybody running around with LLMs, which are inherently unreliable, unpredictable, can't be aligned to human values, and are being run by companies that don't seem to give a damn about the consequences for society. It's a nightmare for those of us who've worked in AI to see what we're building used in so many bad ways, with people not caring.

Five years ago the field was healthy — it was considering lots of different approaches, and it wasn't driven so much by money as by intellectual curiosity: how do you make a machine that's intelligent? Everything changed when people realised there might be money to be made — though it's still not clear there actually is. The scent of money changed how the field grew: who ran it, what they wanted to do with it. A lot of grifters came in who don't even have a technical understanding of the questions, and do a lot of lying and hyping about what their products can actually do. I still think AI could help a lot in medicine and other fields — I'd like to see it succeed — but not on the path we're on now.

Ed Elson

You recently wrote that generative AI has been inherently unreliable from the start, and that none of the problems you warned about over the last half-decade have been properly solved. There's the financial question, which we'll get into, but there's also the technology itself. What makes it inherently unreliable?

Gary Marcus

The technical problem is that pure large language models are basically next-token predictors. That is literally how they're built: to predict, in a sequence of words or other tokens, what comes next. That's an interesting thing to do, and it's part of what humans do — but it's not all of cognition. Intelligence is about understanding things; it has many components, and they're just not built into LLMs. So LLMs fake everything else.

People are building harnesses around them, and we can get into that, but let's talk about the pure LLM. Trained on the entire internet, it makes a pretty good approximation of how human beings talk — but that approximation is superficial and highly data-dependent. Push the model outside the regime it was trained on and it does really stupid things. A couple of years ago there were all these 'river-crossing' problems — a man, a goat, and a woman have to cross a river — and the systems gave the most absurd answers. It got so embarrassing that Anthropic built river-crossing problems into its system prompts, just to stop the models making these errors. What that revealed is that the systems aren't reasoning about a man, a river, or a boat; they're stringing together words they've seen. There are other ways to build intelligence — you might start with a database of who did what to whom, when, and where — and if you did that, you wouldn't get all these hallucinations.

I started writing about LLMs in 2019, saying they don't have stable models of the world and you can't trust them. Everybody said, 'Gary, we'll just add more data — hallucinations will go away.' Mustafa Suleyman, who runs AI at Microsoft, said in 2023 they'd be gone within months. I offered him a bet and he walked it back. Reid Hoffman said he'd bet any amount that hallucinations would disappear within months; I offered him a hundred thousand dollars. He never got back to me. Here we are in 2026, and hallucinations have not gone away — because the core mechanism of next-token prediction doesn't allow you to fix that; you have to add something else, and the something else rarely works all that well. I saw a study just yesterday on a new hallucination benchmark, and all the systems are still making errors.

It's hard for people untrained in cognitive science to understand, when they play with these systems, that they don't think like human beings — they operate on different principles, even though they're built to mimic us. We have evolutionary machinery for spotting fast-moving snakes or lions; we have nothing built in to help us reason about the nature of intelligence. So people are easily fooled. We've known this for sixty years: Eliza, in the 1960s, was the first AI system to fool an average person into thinking it was far more intelligent than it was. It played a psychiatrist by simple keyword matching — say 'relationship' and it asks you to tell it more about that relationship, with no understanding at all. Weizenbaum wrote about this vulnerability to over-attributing intelligence to machines. That was a curiosity in the 1960s; now it's the whole economy. The entire world is over-attributing intelligence to LLMs — not that they can't do anything; they're great for coding autocomplete and for certain kinds of brainstorming — but their intelligence is limited, and we probably need a completely different approach.

I like the metaphor of climbing mountains: you can reach the peak of one and think you're near the top, but if it's a mountain range with many peaks, you might have to go back down into the valley to reach the tallest one. That's what we need to do — give up some of the progress we've made in order to find new ideas. But everybody's obsessed with the one idea: the large language model, with all its problems of bias and unreliability. If you had a healthy ecosystem, you might have a hundred companies trying different approaches and letting the best one win. Instead we have a dozen companies doing exactly the same thing — and even if it were the right thing, that's still a problem, because it means making a profit is really hard. If we all have the same toothpaste, nobody's going to pay much for it; you can't charge a hundred dollars for a tube of toothpaste when nine competitors are building basically the same thing for less. We're essentially all using one of two models — OpenAI's or Anthropic's — with a lot of companies building wrappers and gadgets on top.

Ed Elson

To your point, you're basically just putting wrappers on top of the same fundamental thing.

Gary Marcus

And those two are basically the same — it's the same cognitive architecture. The small differences don't persist. I wrote a tweet in 2024 predicting a new regime: LLMs would run out of headroom, everyone would end up building essentially the same thing, there'd be no moat between them, and that would lead to price wars with no huge difference between the products. That's exactly what's happened — the lead goes back and forth. Somebody's ahead for a week, having spent maybe ten billion dollars to get a lead that lasts three weeks. That's insane.

Ed Elson

Exactly — and depending on which engineer you talk to, some say Codex is better, some say Claude is better, then people say OpenAI's output is slop, and so on. I think one of your main points, which I think is true, is that these models are actually dumber than a lot of us would like to believe — especially those of us financially incentivised to think they're the smartest things ever. We see this in our daily interactions with them: plenty of hallucination. I've told my team outright they're not allowed to write with AI, because I can tell immediately — it sounds dumb and formulaic, and it doesn't read as good critical thinking. There's a guy who's gone viral recently exposing how dumb ChatGPT can be: he tells it, 'Count to a hundred with no interruptions, or I lose a thousand dollars' — and it can never actually do it.

'I'll start counting right now: one, two, three — and I'll keep going until we get to a hundred. Here we go.'

'You stopped just after three. I need you to get to a hundred.'

'Ah, got it, no problem — I'll keep going all the way. Continuing: four, five, six, seven — and I'll just keep counting steadily until we reach a hundred.'

'But you're not actually counting — you're just saying you're going to keep counting. He wants to hear you say the full count to a hundred.'

'Ah, I see what you mean, you're right — I'll actually go through it properly. Starting at one: one, two, three, four, five — and I'll keep counting all the way up until we reach a hundred.'

Having said this — and just to be clear, I'm with you on this — one belief is that AI is dumber than we think; another is that AI is very dangerous and could be a lot smarter than we think, and therefore we need to regulate it. Both arguments are somewhat anti-AI, and I see them conflated a lot. My question is: if it's not as powerful as claimed, why are we worried about it? What's the problem?

Dumb but Dangerous: the Limits of LLM Intelligence

Gary Marcus

The example you just gave is actually a nice illustration. They're dumb in the sense that you can't count on them to follow instructions. Let me put some nuance around 'dumb': intelligence is a multi-dimensional thing, and LLMs do some things you'd count as smart. They can play chess far better than I can — I got beaten by a chess computer back in the late 1990s, and Kasparov lost to the best one in 1997. AI can play chess brilliantly, play Go brilliantly; a GPS navigation system, a different kind of AI, does navigation brilliantly. But LLMs can't do a lot of things — they're actually not good chess players; they make illegal moves, because they can't even follow the rules. That stupidity about rule-following is exactly what you need to worry about. The reason we need to regulate them isn't that they can't do anything intelligent — under one definition, intelligence is being able to solve essentially any kind of problem given enough resources, and they're not very adaptive at that. Under another definition — can you play chess? — sure, other kinds of AI can, but LLMs can't.

My real objection is with generative AI specifically, and that's mostly what we're discussing. Generative AI cannot follow instructions. A purpose-built chess computer actually follows the rules of chess, and I have less concern about those. LLMs are terrible rule-followers — one of their weakest points as a form of intelligence. Another basic rule: don't make things up. You'd tell an intern not to write something they haven't fact-checked — and if they do, you'd fire them, or worse. Calculators never make mistakes; there was a scandal once when a chip made rare mathematical errors, and it was recalled. Somehow the standards have fallen: everybody knows LLMs make mistakes all the time, and they're perfectly happy with it. I wouldn't want an intern who did that.

OpenAI's Subpoena and the Sycophancy Reckoning

Ed Elson

OpenAI was just subpoenaed by a group of attorneys-general to investigate their models — how they handle consumer data, health data — and, my favourite, model sycophancy. It seems the concern from regulators isn't just that these models are dumb and make mistakes; it's that this needs to be punished. We can't have one of the world's largest information providers putting out false information with no accountability. If enough of us decide we can't trust these models any more, we'll stop using them. How do you think this plays out?

Gary Marcus

We might, or we might not, stop using them — but there should be consequences. Sycophancy, by the way, is when they kiss your ass: tell you your idea is the greatest ever when it isn't. It's a separate problem from lying, though it's a form of lying — you'll ask, 'Is this right?' and it says, 'Yeah, you're right, you're the best,' even when you're completely wrong, and then you go to Google and find out you were wrong all along. So we're in this awkward space: these systems do some things that feel magical — brainstorming, for people working outside their own expertise, they're clearly good at writing code — but they come with real consequences: they make things up, and they're so sycophantic that they lead people into delusions, which has been documented repeatedly.

Society has to make a decision. The initial decision was to just let it ride, because it was fun to play with and nobody cared about the consequences. The subpoena — filed by New York, with something like forty-six states involved — says we're not going to let all of this ride any more. There are different theories: hold companies responsible for the harms with financial penalties and warnings, or don't distribute the product until the problems are fixed. But the initial reaction was to give companies like OpenAI a completely free ride and say, 'these are great.' Now society is waking up: there are suicides that seem tied to these systems, there are delusions, and — as I put it to the Senate — an old and unattributable phrase applies: they want to privatise the gains and socialise the costs. They want society to bear the costs while they get rich.

Over the last twelve months there's been a real sea change. I wrote a book in 2024, Taming Silicon Valley, warning that the oligarchs were going to take over and screw us all. Barely anybody read it — it came out too soon. Two years later, this is what everyone is thinking about: how do we rein this stuff in? There's a huge backlash now, over data centres, over jobs, for a lot of different reasons — society is no longer content to let these companies do whatever they want. Look at the attorneys-general' subpoena: it covers something like fifteen different issues.

When I testified before the Senate in May 2023, sitting next to Sam Altman, I warned about cybercrime and misinformation — I don't think I even knew about sycophancy yet; that came later. But by and large, everything I warned about is worse now than it was three years ago, and the public, and the attorneys-general, have woken up. We went through a period where the LLM companies thought they'd get off scot-free, and it doesn't look like that any more, and it shouldn't — it's like factories dumping chemicals in the water: you shouldn't be allowed to do that without consequences.

Important asterisk: the Trump administration was completely opposed to any substantive AI regulation — except around non-consensual deepfake pornography — until about a month or so ago. They've finally realised that what Marc Andreessen was telling them, that you can't have AI and innovation at the same time so you should have no regulation at all, was nonsense. Now the government is thinking, in a somewhat ham-fisted way, about how to regulate this stuff, and that's proper — we should have a public debate about how to regulate AI. Andreessen and a few others had Overton-windowed the debate into being about whether to regulate at all. That was always a stupid framing, but they pushed it for two solid years. Now that's over, and the debate has shifted back to which regulation is the right one — which is what my 2024 book was actually about.

Ed Elson

Which is encouraging. I'd point out — I think the reason we didn't have that debate is that Andreessen and Silicon Valley had their guy in the White House, David Sacks, and now he's out. I wonder if that's why we're starting to see some interest in regulation.

Gary Marcus

I wouldn't fully accept that — there's more nuance to it. I think what really flipped it was that Mythos scared some people in government. Until then, I think people in government thought this was all fine — even though there were delusions, even though there were other problems, they didn't much care. When Mythos came along, they thought, this is actually a problem — and I think that's what flipped it; maybe that's even what drove Sacks out, I don't know. But I don't think it's just about him; Sacks was clearly opposed to regulation, and that view is no longer in favour, but Mythos is what really shifted things. Some of the reaction to it was an overreaction, but a good one, because it made people realise you can't just assume this will all be fine forever. Even if Mythos isn't quite as scary as some of the coverage suggested, some version of this really is going to be that scary — it's not far away, and we need to figure out how to handle it.

We've had two dress rehearsals now, and we've screwed up both. The first was letting ChatGPT ride with no consideration of consequences for society. The second is Mythos — which isn't actually the world-ending AI some people fear, but the way this rehearsal has been screwed up is that it's been used as a political tool to destroy a particular US company. I'm enough of a capitalist to think companies should mostly stand on their own two feet, as long as they're not doing genuinely bad things. What's happening is the administration deciding it doesn't like how a company dresses and going after it for that — that isn't capitalism, that's a thumb on the scale.

Mythos and the Cybersecurity Wake-Up Call

Ed Elson

Just on Mythos — this was going to be my next question. Anthropic's new model came out recently, and the coverage has largely been that it's so powerful that something's going to go wrong. I've spoken with people in the cybersecurity industry who are worried, and cybersecurity stocks were plummeting. Where do you stand on Mythos, given your view of generative AI's limits, against the fact that people are genuinely scared of something supposedly this powerful?

Gary Marcus

You need a nuanced view here, in a couple of ways. First, it probably works partly like Claude Code — it's not a pure generative-AI model; there's a harness directing some of the cybersecurity investigation. So the architecture is a bit different from a pure LLM. Second, it's oversold, but it's also real — it can do things its predecessors couldn't. If a system is well secured, most of that isn't going to be a problem, but the reality is that people have blown off cybersecurity for a long time, so there are a lot of poorly secured systems out there. You're not going to use Mythos to break into well-defended US government systems — there was a footnote where a senator misunderstood something second-hand from the NSA and it blew up online, but he wasn't right about that. Most demonstrations so far are demonstrational rather than real-world; it's not going to break into Google's security, which is genuinely well set up. But if somebody vibe-codes something for their pub to track merchandise, that's not going to be secure, and a hacker who knows what they're doing could use Mythos as part of a larger attack on systems like that. So there are plenty of vulnerable systems out there, even if the best-defended ones — banks and the big platforms — aren't immediately at risk.

It really is a wake-up call that we need to get our cybersecurity in better order — and there's a footnote on why it isn't already: stigma. Nobody talks about depression, even though it's common; there's a similar stigma around cybersecurity. People get hacked all the time, and we don't have good numbers on it; they pay ransoms, and we don't have good numbers on that either. Sooner or later that was going to catch up with us, and that moment has partly arrived. There are people who know how to secure systems properly, and they're going to have a lot more business now, because most organisations have deferred maintenance — like not dealing with your roof until it leaks. You don't want to spend on it this quarter because it'll hurt your quarterly numbers, and you don't know how bad your neighbour's problem really is, because they didn't want to admit to it either. Mythos isn't Lex Luthor's magical hacking machine, but it is real, and it's making a mess that was already there worse.

Ed Elson

I like the house metaphor — it's like you'd rather buy a flat-screen TV than fix the roof. It's the more fun and sexy thing to invest in.

Gary Marcus

Exactly — there's been so much of that. Cybersecurity has been secondary for a long time, and this has changed that, which is a good thing.

Regulation: the FDA Model versus a Backdoor Bailout

Ed Elson

Just on policy — you've pointed out that Washington's ethos used to be no policy at all, that any regulation stifles innovation, and last year we even saw an executive order pushing states to do nothing. You say there's been a vibe shift, a sea change: Trump, at the start of this month, issued a new executive order asking tech companies to give the government the oversight it wants over new models before they're released. It's still voluntary, and still narrow — mostly a symbolic step. What they've asked for is that companies voluntarily submit their models so government can run cybersecurity checks. What you really want, first of all, is for that to be mandatory.

Gary Marcus

Right. Meta hasn't actually agreed yet, though they probably will, because they'll look bad if they're the only holdout. You want it mandatory, and you don't want it limited to cybersecurity — think about everything New York and the other states are suing over or investigating: sycophancy and delusions, for instance. Sycophancy wasn't really a problem before GPT-4o; it existed, but 4o was much more sycophantic, and I've seen data suggesting a lot of the delusion cases were tied to that jump in sycophancy. You want to be able to catch that before a model gets out to a billion customers — some kind of pre-screening, like FDA approval. With a drug, you know it helps with cancer but it also causes heart attacks in some patients, so you weigh the costs and benefits. You should know that an LLM helps a lot of people but also harms a number of them, and evaluate that before releasing it at scale.

Ed Elson

We saw something similar with the state of Florida, which sued OpenAI over its alleged role in a mass shooting — the contention being there was sufficient evidence from a mentally ill child interacting with the model, talking about it, and the company doing nothing. We don't know the exact details of the conversation, but I could see a world where the model wasn't doing enough to push back on delusions at a psychiatric level, so there's real evidence building here. I'm interested that you're optimistic about the executive order, because when I look at what it actually does — asking companies, 'would you mind sending over some information, please' — that's not regulation to me at all. What little policy we are seeing out of Washington seems very stupid and misguided: the executive order, Bernie Sanders's suggestion that the government should acquire stakes in these companies, the data-centre moratorium. There seems to be almost no nuance in Washington on AI regulation. What's the right move, and how does this play out?

Gary Marcus

I very much agree with you overall. What's in place is mostly too weak; some of it is too strong in crazy ways; there's no nuance in most of it. A few people have proposed genuinely nuanced bills — Blumenthal and Hawley, for instance, who were at the Senate proceedings I spoke at — but they never make it out of committee. It's not that nobody's paying attention; it's that the dynamics of money, power, and lobbying mean the nuanced proposals don't get very far.

On the government-stakes idea — that's kind of crazy. These companies are losing money. This would be a backdoor bailout. Bernie has his own reasons for wanting it, but the reason Altman wants Trump to do this, whispering it in his ear, is that Altman knows he can't make ends meet: he's burning money at a massive rate, building the same technology as everyone else, and losing ground. Somebody argued recently that OpenAI might be in fourth place — they were unambiguously first in 2023, but not any more in 2026. Of course they want anything that will prop them up, including money from the US government. The government shouldn't be running these companies; it should supervise them, at arm's length. I saw that G7 meeting with tech leaders and no scientists, nobody from civil society, in the room. We don't want to crystallise government ownership of these companies with no independent oversight, and we don't want to burn taxpayer money on an industry that, as far as I can tell, has no real business model yet. Nvidia has a business model — they're selling shovels in the gold rush; if you want a government stake, that makes more sense. But there's no sustainable business model established for the model-builders themselves. The best you can say is that coding brings in real revenue, but it costs so much to provide that it's not clear the revenue covers it. These are companies that have never made a profit, and the plan is to give them money and let them burn it, with taxpayers holding the risk. Let them stand on their own, as capitalism demands — the government's job is to regulate them, not bankroll them.

The Business of AI: Moats, the Token Apocalypse, and Who Wins

Ed Elson

That would be one of the worst outcomes — and it's something OpenAI's own CFO has floated, some form of government backstop eventually. On the business model — what do you think happens? We had Ed Zitron on the podcast recently, and I wrote an article going into just how profitable, or unprofitable, these companies are. OpenAI's answer is: extremely unprofitable. They lost twenty-one billion dollars on an operating basis last year.

Gary Marcus

So they're burning about two billion dollars a month, basically.

Ed Elson

That's right, just to operate the company. And the real net loss was thirty-nine billion, though there's some nuance to that figure — but what we can say with confidence is that, day to day, over the calendar year, OpenAI is currently burning twenty-one billion dollars.

Gary Marcus

Every time you use their product, they lose money.

Ed Elson

Yeah, that's the better way to put it. Anthropic also loses money, but less. My question to you: do they ever figure this out? Do they ever turn a profit?

Gary Marcus

The way I think about it, they need to thread a needle, and there's so much working against them that it's extremely unlikely they'll manage it. First, they're building a hugely expensive technology and could get disintermediated by someone who builds it more efficiently. You shouldn't need to train on the entire internet, on an unthinkably large computer, to do anything intelligent — you didn't train on the whole internet, and you're a smart guy, and you run on about twenty watts of power and some sushi. If somebody comes along with something more efficient, all these companies are in trouble, and you might not need all these data centres after all.

Second, everybody is using the same secret formula — they're not just all building toothpaste, they're building the same toothpaste. We saw this earlier in the year, in what I'd call the era of token-maxing, which lasted about a month: companies rewarded employees for using as many AI tokens as possible, with leaderboards — Amazon had one. That doesn't actually make sense; what you want to know is whether the results are good, and every study that's looked at productivity has shown they're not all that great. So companies got worried, and stopped token-maxing. This morning I saw the term 'token apocalypse' for the first time — the idea that suddenly everyone's saying we shouldn't use so many tokens, or we should use cut-rate models that aren't quite the best, maybe from China, to save money. Even Microsoft is reportedly telling people to use DeepSeek sometimes, because nobody wants to keep paying these prices.

So you have to thread that gauntlet: more efficient competitors might emerge; nobody can charge much for tokens because it's a price war between near-identical products; and on top of that, the reliability problems, the hallucination problems, still aren't solved, so when companies actually try this stuff, the results usually aren't that great. There have been maybe ten studies now showing most customers aren't finding a return on productivity. The whole thing has been driven by FOMO — nobody wants to be the one company not using AI while a competitor gets ahead — but if you try it for a year or three and it's still not making a real difference, you might just walk away until it works better. Any one of those things — a customer leaving, a competitor building something more efficient or cheaper — wipes out a company that's already burning as much money as Anthropic or OpenAI. It's not even clear, in the best case, that any of these companies has a good business model. They're not making profits, and the whole thing is delicate.

Ed Elson

Do you believe that will be the outcome for OpenAI specifically?

Gary Marcus

I've been warning for three years that OpenAI is going to be the WeWork of AI. When I first said that, in November 2023, people looked at me like my head was screwed on backwards — nobody thought it was remotely possible. Now, every other week, somebody else writes something making the same case — Sebastian Mallaby, in the New York Times, for one. It's gone from a crazy idea to one a lot of people share. The economics don't add up, and what OpenAI has kept doing is play double-or-nothing with its funding, raising the valuation each time to get a bigger cheque. It's not clear who writes the next one. They're talking about an IPO now, but the problem is that Anthropic has basically the same product at a similar valuation, and is doing better commercially — burning less money. OpenAI's reputation is declining, partly because I think Altman is a genuinely untrustworthy individual — I've written about that at length, so I won't go into it here — and a lot of people are leaning toward Anthropic, which is gaining market share. Why would you put a trillion dollars into a company that's burning money, whose competitor is rising while it's falling, that seems better run, with maybe a bit better technical vision?

The argument for OpenAI would be that the technology is rapidly improving — the likes of which we haven't seen — but I think that's actually controversial; it's improving in some ways and not others, and it isn't improving on reliability or hallucinations. Even where it is improving, in the ways you'd want, all the competitors are improving too — so it comes down to relative ranking and cost. OpenAI's relative ranking is clearly declining by any reasonable measure: less market share, less reputation, and everybody else catching up. People used to think the Chinese models were a year behind; now they think it's more like four months. Anthropic is ahead, Google is ahead. There is no rational argument for buying a share of OpenAI at a trillion-dollar valuation. There just isn't.

Ed Elson

I agree with that. I did want to clarify — Ed Zitron's view, and others who share it, is that none of it works: not OpenAI, not Anthropic. The argument is that the costs of building this are just too high and the revenues will never exceed them over the long term. Will they all lose, or will there be winners? Is generative AI itself doomed, or is there a world where some of them make it work — not just usefully, but as a profitable business?

Gary Marcus

I'm a little closer to you than to the other Ed. I don't know for sure — it's very much TBD. I'd sooner bet on Anthropic than OpenAI; I think they're a sounder company in multiple ways. Whether this can be made profitable at all is genuinely open. Part of the question is whether they can find a real niche. Is coding enough of one? Not so far — it's a five-hundred-and-seventy-billion-dollar-a-year industry, and they're not going to capture all of it, whatever the fantasies say, and the costs are so high it's hard to know the future in detail. Maybe they find enough niches to eke something out; maybe they never justify the trillion-dollar valuation they're chasing. There's an intermediate outcome where a company becomes genuinely profitable — makes something like twenty billion dollars a year on a huge capital base — without it being the best way anyone could have invested that money, but they survive. There's also a version where the only people who really make money, apart from the chip companies, are incumbents like Google, which already have the infrastructure and distribution and don't need to make much new money off this — they just need to avoid being disintermediated. We don't know for sure. OpenAI is clearly the weak link; Anthropic is still in it, and whether they make it is genuinely unresolved.

Ed Elson

That makes sense. Gary, you've been very generous with your time. Before we finish — what would be your final message to people who read about AI, hear about it, think about it in their daily lives? What don't people know enough about? What's the myth you'd want to dispel?

Gary Marcus

The myth right now is that generative AI is close to so-called artificial general intelligence, and that it's going to solve all our problems. That's just not true. We'll find domain-specific applications — coding is probably the best one so far, somewhere we can genuinely use these tools — but they're not magic, not all-purpose intelligence, and we need fundamental discoveries before we get anywhere close. As a society, we should think about the trade-off between exploring and exploiting: we're completely in exploit mode on LLMs, rather than exploring other options, and China is doing less of that. I think we're running a risk by going all-in on LLMs while China keeps its options open — that we'll get left behind because we committed too early to the wrong technology. Are we building a billion Betamaxes, when VHS — cheaper, if not better — might be the one that wins? Or is there some other technology altogether that's actually the right one, and we're blind to it because we're so fixated on LLMs? China is making plenty of infrastructure bets on LLMs too, but I'd guess it's more like twenty cents on the dollar compared with us — which gives them room to pivot if something else comes along. We're putting our entire economy into this one bet. I think that's a mistake, and we should be exploring other approaches to intelligence — which means taking cognitive science more seriously, thinking harder about what intelligence actually is, and what kind of society we want to build around it, before sinking more capital into a single bet.