The researchers introduce VeriPhy, an auditable physical-verification system designed to evaluate and refine video generation models by moving beyond simple scalar quality scores. Unlike monolithic prompting methods, VeriPhy utilizes a text-only planner to compile prompts into typed physical obligations and statically validated execution plans prior to frame observation. The system operates by gating calls to frozen low-level experts—including segmentation, tracking, counting, depth estimation, OCR, and audio-event detection—to extract evidence records. These records are processed by typed resolvers and fixed composition logic to produce a three-valued state (plausible, implausible, or abstain) for every physical claim. This architecture ensures every verdict is traceable to its specific evidentiary payload, providing full provenance. To validate the framework, the authors utilized a 1,500-clip corpus containing human-annotated flaw records. In testing on a 149-clip core containing 304 error records, VeriPhy successfully identified 228 flaws, outperforming a published question-decomposition evaluator which captured only 164. While monolithic prompting achieved a similar recall of 222, VeriPhy's primary advantage lies in its auditable traces, which allow for precise localization of failures in space and time, ultimately enabling a critic to write corrective verdicts back into the generation process.

Read original