The authors introduce PLatour, an open-source framework for Prefill-to-Autoregressive Cross-model Transfer (PACT) that connects frozen large and small language models via lightweight, zero-initialized linear bridges trained directly on task rewards. Supporting hybrid-attention architectures, PLatour outperforms open-source baselines on generation tasks and exceeds Mostik's reported results on shared mathematics and coding benchmarks while utilizing a substantially smaller sender model. However, controlled experiments reveal that performance gains on certain tasks are replicable using uninformative sender signals, self-communication, or receiver-only training, indicating that trainable bridges often function merely as parameter-efficient adaptation mechanisms rather than conduits for useful sender information. To address this confound, the authors propose PACT-Check, a diagnostic method to quantify the genuine contribution of cross-model communication to overall system performance. The work establishes a rigorous evaluation infrastructure for latent model collaboration, distinguishing between actual information transfer and adapter-like bridge behavior.
Read original
huggingface/daily-papers