MAFDET is an open‑source (Apache‑2.0) application built on llama.cpp that enables a local LLM to begin reasoning while the user is still typing, cutting inference time from roughly 12 seconds to about 3 seconds on a 6 GB GPU. It runs background tasks—such as reading the input—as soon as keystrokes are entered, allowing the model to pre‑process and respond faster. This approach significantly reduces the wait between sending a prompt and receiving the first output.
Read original
reddit/r/LocalLLM