To address the issue of geometric collapse in online 3D reconstruction caused by cumulative pose drift in long videos, the authors propose Scal3R, a framework that reformulates reconstruction as a multi-reference relative pose querying task. Traditional models often fail because regressing poses against a single fixed anchor leads to extrapolation errors; however, the researchers observed that while global pose heads fail, per-frame depth and local geometry remain stable. Scal3R leverages this by injecting lightweight learnable tokens—comprising approximately 1% of total parameters—into a frozen backbone using an asymmetric attention mechanism. This architecture allows the model to query poses relative to multiple historical keyframes rather than a single starting point. To further mitigate long-range drift, the system incorporates an online pose-graph optimization method with loop closure. Scal3R demonstrates high efficiency, achieving convergence in 8 hours on a single GPU. Experimental results show a reduction in average Absolute Trajectory Error (ATE) by more than 60% on the KITTI dataset compared to online baselines. Furthermore, the method establishes new state-of-the-art performance metrics across several diverse datasets, including Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes, proving its scalability for extended sequences.

Read original