Mise-en-Video: staging multi-shot video generation in 3D
A multi-shot text-to-video framework in which each shot is staged first with oriented 3D boxes for subjects, then generated with attention that keeps the story consistent across shots. The authors built a 160,000-video dataset of multi-shot scripts with 3D annotations and report gains over baselines in consistency and subject control.
The name says it: blocking before generating. Research is converging on what previs has always done — decide where things are in space, then make the image. The tools that win in production will be the ones that accept a staged scene as input, not just a paragraph.
The paper borrows its name from the stage for a reason. Mise-en-scène is the arrangement of everything inside the frame, and the authors’ argument is that multi-shot video generation has become good at consistency across shots while still giving creators little say over that arrangement. Mise-en-Video, or MeV, puts an explicit 3D staging step in front of the generator so that the creator decides where things are before the model decides what they look like.
The full title is “Mise-en-Video: Fine-Grained 3D Control for Multi-Shot Video Generation,” from a team including Yingshuang Zou, Xiaoyu Shi, Xintao Wang, Pengfei Wan, Kun Gai and Yuan Liu. According to the abstract, the framework has two parts: a hierarchical attention mechanism meant to keep the narrative consistent at several levels at once, and 3D-aware layout guidance that gives per-shot spatial control through oriented 3D bounding boxes. To train and evaluate it, the authors built standardised annotation pipelines and a 160,000-video dataset with hierarchical multi-shot text descriptions and oriented boxes. They state that MeV consistently outperforms strong baselines on video quality, inter-shot consistency and subject-level controllability. The abstract gives no figures.
The orientation of the boxes is the detail that matters to anyone who has cut a scene. A box that knows which way it faces encodes not just where a subject stands but screen direction, eyelines and which side of the line the camera is on. Those are the properties that make a sequence of shots read as one space, and they are exactly what text prompts express badly; “the woman on the left, facing right” tends to dissolve by the third shot. An oriented box is the vocabulary of a blocking diagram or a previs layout. If generators accept that kind of input, the handoff from previs to generated shots becomes a transfer of data a layout artist already produces, rather than a re-description of it in prose. That makes previs more central, not less.
It also fits a pattern the feed has been tracking. TARS in July addressed re-shooting existing footage from a new camera angle, MSEditor in August applied one edit consistently across several shots, and the addressable-memory work tackled returning to an earlier scene. Taken together, the research is rebuilding coverage, continuity and blocking — conventional film grammar — inside generative systems, one piece at a time.
The limits are the usual ones for a research paper, plus a few specific to this approach. The abstract reports no metrics, failure cases or compute costs, and the dataset is the authors’ own, so independent replication will matter. A bounding box carries position and orientation but nothing about pose, gesture or performance, which is where most of a director’s notes land. And no release of code or weights is mentioned. What we would watch for is less the paper itself than whether commercial video tools start accepting 3D layout as conditioning, and whether they read it from formats previs and layout departments already use rather than inventing a new one.