Trained to Imagine
Rendering
Creative Intelligence
AI Research / Breakthroughs

Phantom: physics-infused video generation

Source: arXiv ·

A method that jointly models visual appearance and latent physical dynamics during generation, rather than treating physical plausibility as something that emerges by accident from enough training data.

TTI Takeaway

Physics is where generated footage still gives itself away — cloth, water, hair, and anything with weight. Approaches that model dynamics explicitly rather than hoping for them are the ones worth watching, because that failure mode is exactly what forces a shot back into comp.

Analysis

Phantom is a useful paper less for its headline numbers than for the bet it makes about why video models get physics wrong. The authors argue that the standard objective of predicting the next frame rewards models for looking plausible without requiring them to track the forces and materials that produce motion, and that scaling data and model size does not fix this. Their answer is to make the model predict a representation of the physics alongside the pixels.

The method is specific. Phantom builds on Wan2.2-TI2V-5B, an existing open video diffusion model, and freezes its visual weights entirely. Alongside it, the authors train a separate physics branch from scratch that works in the embedding space of V-JEPA2, a pretrained vision encoder previously shown to capture intuitive physical properties. The two branches exchange information during generation through bidirectional cross-attention, so the physics prediction can influence the frames and vice versa, without an external simulator or any explicit list of physical parameters. Training used OpenVidHD-0.4M, around 400,000 high-resolution video-text pairs not selected for physical content. The authors report gains over the unmodified base model of 50.4 percent on VideoPhy’s physical-commonsense score, 2.6 percent on VideoPhy-2 and 33.9 percent on Physics-IQ, and show examples where soap bubbles stretch and wobble instead of drifting like rigid balls, and a viscous liquid folds into layers instead of disappearing into a void.

For VFX and post, the architecture matters as much as the scores. Because the visual backbone stays frozen and only a lightweight branch is trained, the approach is in principle something that could be added to other models at modest cost rather than requiring a new foundation model. The setup is also video-to-video: Phantom is conditioned on observed frames and predicts what happens next. That is close to tasks productions already care about, such as extending a plate, continuing an effect past the end of a take, or generating the next beat of a shot where fluid, cloth or debris has to keep behaving consistently with what the camera captured.

The limits are clear from the paper itself. The model runs at 480 by 832 pixels and up to 121 frames, far below delivery resolution for broadcast or theatrical work. The improvement on VideoPhy-2 is small next to the other two benchmarks, which suggests the gains are uneven across kinds of physical scenario. And plausible physics is not the same as directable physics: there is no control described for making a liquid thicker or a fabric heavier on a director’s note, which is what a simulation in Houdini provides.

This fits a line of research the feed has followed since PhyWorldBench in February, which set out to measure physical realism rather than general visual quality, and it connects to the world-model argument Runway’s Cristóbal Valenzuela made in April, where consistent physics is the stated target. Our view is that methods like this will reach production first as a way to reduce the number of shots that fail on weight and motion, not as a replacement for simulation, and the next thing to watch is whether the approach holds up on larger backbones and at higher resolutions.