Google’s Genie 4 world model adds audio, more realistic worlds

Google DeepMind released Genie 4, its newest world model, in research preview on Tuesday. TechCrunch's report says the model generates the most photorealistic and interactive worlds yet, with mid-simulation prompts and real-time audio: music, ambient noises, and interactive dialogue.

This piece is for builders in games, interactive learning, robotics, and film prototyping. It covers Genie 4 versus Genie 3, the Gemini 4 Argon loop, trusted-tester access, and the coming rollout to Project Genie, the Gemini API, and Google AI Studio.

What Google’s Genie 4 world model shipped

During a run, users type text prompts and make freeform edits: change the weather, change the camera angle in real time, or add new people or objects. Real-time audio matches the simulated world and includes music, ambient noises, and interactive dialogue. Genie 4 is far more photorealistic than Genie 3, with better physics and more human-like characters. It keeps a consistent scene for up to two minutes, double the predecessor’s limit. It generates the most photorealistic and interactive worlds yet.

The release comes a couple of months after AI video startup Runway released its second world model, GWM Worlds 2, a playable model that responds dynamically to user inputs and includes audio generation. The world model industry is beginning to consolidate. Fei-Fei Li’s World Labs is being acquired by AMD, Amazon is in talks to buy Decart, and Rivian just acquired Third Dimension for in-house self-driving testing. Genie 4 is Google’s latest move to stay ahead in the crowded field.

What changed for builders in Google’s Genie 4 world model

Google released its predecessor, Genie 3, as a research preview a little over a year ago. Genie 4 is far more photorealistic, features audio, and adds freeform edits in real time during a run. Scene consistency is up to two minutes, double Genie 3’s limit, so the prior cap was one minute. Gemini, Google’s AI model, is more closely integrated and acts as the director: it plans steps and checks frames while Genie 4 creates the visuals for the world.

In Gemini Robotics research, a Gemini 4 Argon model broke down tasks into steps that Genie 4 simulated in real time, a possible way to test robot scenarios before the real world. Tejas Kulkarni, research director at Google, said during a briefing, “What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction.”

How to install or upgrade Google’s Genie 4 world model

Genie 4 is available now as a research preview for trusted testers, the people who get first access to Google cutting-edge products. Those testers are pointing to a possible new use: prototyping films. Gaming, interactive learning, and robotics remain the commercial cases pitched for world models. Diego Rivas, a Google DeepMind engineer, said the real-time aspect is most valuable where keeping a state of flow in the creativity space becomes useful. He named the ideation and storyboarding phases as examples.

In the coming weeks the model rolls out to Google AI Ultra subscribers in Project Genie, Google’s AI tool for generating interactive worlds from text or image prompts. Early access partners of the Gemini API and Google AI Studio are in that rollout. The preview is the access available now, before that subscriber and partner rollout. Rivas said it is still early days for Genie 4 and that it is not yet clear what the end-to-end product experience will be.

Gotchas and compatibility in Google’s Genie 4 world model

The team also acknowledged that Genie 4 still has gaps. Details and objects can drift the longer a simulation runs. It can maintain a consistent scene for up to two minutes, double Genie 3’s one-minute limit. They still haven’t quite figured out how to ensure the audio will be accurate, and they still haven’t quite figured out how to ensure any in-world text will be legible in the simulated scene. That audio is real-time music, ambient noises, and interactive dialogue.

Rivas added that users could, eventually, rely on Gemini Omni, a multimodal large language model that can generate video from images, audio, and text, to turn those prototypes into creative assets. Those capabilities, he says, are still a ways off. Google also offers Veo, its AI video generation model that makes high-definition clips with built-in synchronized audio from text, image, or video prompts. Veo has been growing in popularity among filmmakers. Veo is Google’s other video model, not Genie 4.

What to watch after Google’s Genie 4 world model

One speculative example of interactive learning would be entering a painting of a historical scene and chatting to the painting’s subjects about the event portrayed. Testers point to prototyping films. During a press briefing Rivas said, “The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful,” and, “For example, in the ideation phase, in the storyboarding phase — anytime you’re kind of in need of rapid ideation.”

Kulkarni described hybrid models in which Gemini writes code and text while Genie ingests that input in real time and simulates reality at different levels of abstraction, the pattern a Gemini 4 Argon model used in the robotics experiment. Runway’s GWM Worlds 2 arrived a couple of months earlier, playable with audio. AMD is acquiring Fei-Fei Li’s World Labs, Amazon is in talks to buy Decart, and Rivian just acquired Third Dimension to bring simulation capabilities in-house for self-driving testing.