Focus‑llama is a fork of llama.cpp that implements Declarative Attention, allowing the model to signal which context chunks it should attend to via special tags like <focus magic_chunks="N">. The engine reacts to these tags, restricting attention without any additional scorer or training. The approach reduces decode time to about 0.71× for Gemma and 0.77× for Qwen models compared to baseline.
Read original
reddit/r/LocalLLaMA