DistilVDR is a compact 524M parameter end-to-end visual document retrieval system designed to reduce the indexing and serving costs of multi-billion-parameter models. It utilizes a dual-student distillation approach from a single 8B vision-language teacher model using a pointwise cosine alignment loss. This method achieves a compact single-vector retriever without requiring training from scratch.

Read original