Mark Zuckerberg on the Metaverse, Codec Avatars, and the Future of Presence

Lex Fridman and Mark Zuckerberg conduct this entire conversation as photorealistic Codec Avatars inside Meta’s VR headsets — the interview doubles as the demo. Zuckerberg uses the format to argue that Quest 3, embodied AI personas, and Codec Avatars are converging on a single thesis: the ‘real world’ is becoming an inseparable blend of the physical and the digital.

Key ideas

1. Codec Avatars encode expression, not video. Meta scans a person’s face once into a compressed per-person model, then reconstructs their live expression from headset sensors and transmits only the encoded parameters — photorealistic, and more bandwidth-efficient than streaming 3D video.
2. Quest 3 makes mixed reality mainstream. Launched at $500 against Quest Pro’s $1,500, with better passthrough cameras and a stronger Qualcomm chipset, Quest 3 overlays digital objects on the physical room rather than replacing it — Zuckerberg frames it as this generation’s defining capability shift.
3. Presence, not resolution, is the product. Zuckerberg’s stated goal is not visual fidelity for its own sake but the feeling of being physically together with someone who is not — the philosophical claim underneath the whole Metaverse pitch.
4. AI personas are meant to be many, not one. Meta’s strategy is deliberately plural — an assistant (Meta AI), business-facing agents, and entertainment personas (a Snoop Dogg-voiced dungeon master) — rather than a single ‘super intelligence’; a creator’s own faithful AI self is described as a harder, later problem.
5. Llama 2’s open-sourcing was a considered bet. Zuckerberg defends releasing Llama 2 openly as a rigorously assessed judgement that the value outweighed the risk, with a Llama 3 successor acknowledged but undated and subject to the same red-teaming process.

Content

Codec Avatars

The technology underpinning the episode’s format: both speakers had earlier undergone a lengthy scan capturing a wide range of facial expressions, which Meta compresses into a per-person model. A headset’s internal sensors read the wearer’s live expression and send only the encoded parameters over the connection, rather than a video stream — cheaper on bandwidth and, according to Zuckerberg, past the uncanny valley because it preserves asymmetries, wrinkles, and micro-expressions in the eyes rather than smoothing them away. The near-term bottleneck is capture time: producing a scan currently takes hours, and Meta’s stated goal is a two-to-three-minute phone-based scan of comparable quality. Zuckerberg is explicit that photorealistic and cartoon-style (‘expressive’) avatars will likely coexist — he does not commit to photorealism as the end state, only as the more persuasive on-ramp while people adjust to the format.

Quest 3 and mixed reality

Quest 3 ships this same autumn at a third of Quest Pro’s price, with a newer Qualcomm chipset and higher-resolution passthrough cameras that Zuckerberg says make its mixed-reality experience better than the more expensive Pro line. He frames each headset generation as a step-change — unlike annual smartphone refreshes — with mixed reality (overlaying holograms and objects on a camera view of the real room) as this generation’s addition. Part of the pitch is comfort as much as ambition: seeing your real furniture while playing a VR shooter, or exercising, removes the disorientation and physical risk of fully immersive VR.

AI in the Metaverse

Zuckerberg lays out a deliberately plural AI strategy: a plain assistant (Meta AI) for factual help across Meta’s apps, business-facing conversational agents for small businesses, and entertainment personas — including licensed celebrity voices, such as Snoop Dogg playing a dungeon master in a text-adventure AI that Zuckerberg says tested best internally. A further-out and harder goal is letting a creator (Fridman is offered as the example) build a faithful AI version of themselves that fans can interact with while the creator is still alive — harder than a fictional persona because it must stay reliably true to a real person’s views and voice. Llama 2 underpins this build-out; Zuckerberg defends its open release as a rigorously red-teamed judgement call, and confirms a Llama 3-class model is in training without committing to a release date or an open-source decision in advance.

Presence and the nature of reality

The conversation’s philosophical throughline is Zuckerberg’s claim that the ‘real world’ is not the physical world alone but the combination of physical and digital experience — and that as more of life’s content becomes photorealistic and immersive, that combination will feel as natural as the shift from text to photos to video already has on social platforms. He extends the logic to remote work (feeling physically present without travel) and, more cautiously, to grief — interacting with a deceased loved one’s digital presence — while explicitly flagging that question as ethically unresolved and outside his own expertise. On what should be permitted in virtual spaces, his line is harm-based rather than simulation-based: virtual violence among consenting friends is treated as harmless play, while anonymity-driven toxic behaviour is the harm he says still needs solving.

See also

• Mark Zuckerberg — guest
• Lex Fridman — host
• Yann LeCun on Meta AI, LLMs, and the Path to AGI — companion Meta/AI episode on the same show