Kinetic is a new open-source LLM inference engine written in Rust that runs existing GGUF models without conversion on Windows and Linux. It supports CPU, NVIDIA CUDA, AMD, and Intel GPUs via Vulkan, and serves as a drop-in replacement for llama-server with identical flags while exposing OpenAI, llama-server, and Ollama APIs on a single port. All optimizations are exact, ensuring bitwise-identical output, and the project is licensed under Apache-2.0.

Read original