The VLS app enables running any GGUF model locally on Windows and Linux, supporting OpenAI API and MCP interfaces with an 8K token window and no vector database or embeddings. At a 32M-token context, it achieves a median response time of 653 ms, as reported in the project's RULER benchmarks. The preview release is available on GitHub, though users should anticipate bugs.
Read original
reddit/r/LocalLLM