The "Cache-to-Cache" (C2C) framework introduces a novel paradigm for inter-model communication in multi-LLM systems, moving away from inefficient text-based exchanges. Traditional methods require models to convert internal representations into token sequences, which causes semantic loss and high latency. C2C bypasses this by facilitating direct semantic transfer through the manipulation of Key-Value (KV) caches. The architecture employs a neural network to project and fuse the source model's KV-cache with the target model's cache, utilizing a learnable gating mechanism to identify specific target layers that derive the most benefit from this enriched semantic input. This method allows models to leverage deep, specialized representations without increasing the overall cache size. Empirical results demonstrate that C2C significantly enhances performance, achieving a 6.4% to 14.2% increase in average accuracy compared to individual models. Furthermore, C2C outperforms traditional text-based communication by 3.1% to 5.4% in accuracy while simultaneously providing an average 2.5x reduction in latency, making it a more efficient and expressive method for multi-model collaboration.
Read original
hackernews