omlx is an LLM inference server designed for Apple Silicon that employs continuous batching and SSD caching to improve throughput. It is managed via a macOS menu bar interface, allowing easy control of the server. The project is hosted on GitHub under the jundot/omlx repository.

Read original