Olla v0.1.0 has been released as a lightweight, open-source LLM proxy designed for homelabs and on-premise inference environments. The tool supports multiple backends including Ollama, vLLM, SGLang, llama.cpp, and oMLX, providing features such as failover, sticky sessions, and load balancing. Originally developed to manage multiple local LLM inference servers, the project has reached its 25th release and is now used in production by customers.

Read original