Researchers have introduced FactorJEPA, a novel predictive architecture designed to model the complex, high-density dynamics of "DENSEWORLD" environments—urban settings in the Global South characterized by heterogeneous agents, soft spatial boundaries, and frequent occlusion. To facilitate this study, the authors released DENSEWORLD-115k, a large-scale dataset comprising 1,000 hours of aerial, walking, and driving video footage captured across 22 cities. Addressing the limitations of monolithic Joint Embedding Predictive Architectures (JEPA) in chaotic scenes, FactorJEPA factorizes future predictions into distinct subspaces for layout, entities, and interactions. This approach utilizes a visibility gate to manage partial observability and prevent the model from taking "shortcuts" between different factors. Experimental results demonstrate that FactorJEPA outperforms existing methods across several critical metrics: Future-frame L1 accuracy, Causal L1 for intervention-sensitive predictions, and the Mask-ratio slope for robustness against reduced visual evidence. Additionally, the model reveals a reproducible motion-information trade-off, measured via Motion cosine. The performance rankings of the proposed method remained consistent across both 1B and 2B V-JEPA 2.1 backbones, showing high correlation coefficients (rho = 0.895 to 0.978), validating the scalability and effectiveness of the factorized approach in modeling unstructured, mixed-traffic urban environments.

Read original