DKV: Open-source KV-cache compression for scalable long-context LLM inference

Article automatically generated from technical news.

I've open-sourced DKV, a KV-cache compression architecture for long-context LLM inference. The project explores anchor-based KV representations, joint low-rank compression, exact residual preservation, and sparse routed attention to reduce KV-cache memory requirements. The repository includes: Technical report (Zenodo DOI) Open-source implementation MLX backend CUDA backend (currently under validation) I'm interested in feedback on the architecture, implementat