Researchers have introduced WorldExam, a hierarchical diagnostic benchmark designed to evaluate controllable video generation models as functional world models. Moving beyond simple visual fidelity, the benchmark assesses models across four distinct levels: Visual Quality, Control Adherence, Spatial Consistency, and World Reactivity. The latter is critical, as it measures a model's capacity to infer and generate plausible, scene-conditioned consequences and goal-directed behaviors that are not explicitly detailed in the input prompt. The dataset comprises 1,474 cases distributed across eight specialized tasks, offering a unified evaluation framework for camera-driven, action-driven, and language-driven model paradigms. Benchmarking 20 representative models revealed significant performance trade-offs: camera-driven models demonstrate superior camera control but lack dynamic interaction capabilities, action-driven models allow for precise subject manipulation but often result in unresponsive environments, and language-driven models excel at interaction at the expense of strict control adherence. Ultimately, the study demonstrates that high visual quality and the ability to fulfill explicit instructions do not inherently guarantee the "world reactivity" necessary for true world modeling, as no currently tested model achieves consistent strength across all evaluated dimensions.
Read original
huggingface/daily-papers