A definition and roadmap for world models
An attempt to pin down what a world model actually is and what would count as progress, at a point where the term is being applied to very different systems by different labs.
Useful precisely because "world model" has become a marketing term. If you are being sold one, this is the vocabulary to ask questions with — persistence, controllability, and whether the thing holds together when you move the camera.
The term world model now covers a video generator, a navigable 3D scene and a robotics planner at the same time, and this paper is an attempt to stop that drift. Written as a perspective article by the Physical Intelligence Team at Shanghai AI Laboratory, it offers a formal definition, a taxonomy and a staged roadmap. For production people the value is not the formalism but the distinctions it forces, because those distinctions map closely onto what breaks when a generated environment is put in front of a camera.
The definition is deliberately austere: a world model is a compressed model of how the physical world changes state, built under finite compute. From that the authors derive three properties: handling every perceptual modality, coping with signals sampled at different rates, and modelling the world from a local, partial point of view. More useful in practice is the functional split they adapt from Fei-Fei Li. A renderer produces plausible images or video; a simulator carries state forward under intervention, preserving geometry, object persistence and physical constraints; a planner chooses actions against a goal. The paper places Sora and Seedance in the renderer category, describes Genie 3 as observation-level but reaching toward 3D, and puts World Labs’ Marble in the structured 3D camp, with the explicit caution that collision meshes are not the same thing as a learned model of physics.
That renderer-versus-simulator line is the question a previs or virtual production team should ask of any tool sold as a world model. A renderer can give a convincing shot; a simulator is what lets a camera move, a prop shift, or a second setup come back to the same space and find it unchanged. The paper names the failure mode directly: small errors in an autoregressive rollout compound until the world drifts. It is the same problem behind the addressable-memory work in August, and behind Runway’s own stated limits for GWM Worlds 2, where detail degrades on fast camera turns and long-term memory is imperfect. Genie 4’s two-minute consistency window, reported in October, is progress measured on exactly this axis.
The roadmap will also read familiarly to anyone from VFX. The authors observe that most systems keep three separate definitions of the world, appearance primitives such as Gaussian splats for rendering, meshes or particles for simulation, grids or object slots for planning, and that translating between them is lossy and ad hoc. Their proposal is one physical state with many decoders. Pipelines already pay that translation tax every time a scanned set is rebuilt for simulation or relit for a new plate, which is why research like LightBridge’s single-pass splat relighting matters beyond the lab.
The caveats are significant. The paper’s centre of gravity is robotics and physical AI, not filmmaking, and its definition is stricter than the one most vendors use; by its standard, many products marketed as world models are renderers. It also admits that evaluation is fragmented, listing benchmarks such as PhyWorldBench alongside visual-quality metrics, and that no current metric captures physical correctness well. With AMD agreeing in September to buy World Labs for $8.2 billion, money is moving faster than definitions. Asking a vendor which of the three roles its system actually performs is a cheap way to find out what is being bought.