Caminus 0.2.0 is a new LLM inference engine for AMD RDNA3 GPUs leveraging custom HIP kernels and multi-GPU execution with a single kernel launch per token. Developed on an RX 7900 XTX and 7900 XT setup, the engine reportedly outperforms llama.cpp on tested models and workloads. The project is available on GitHub and PyPI, seeking feedback from other gfx1100 users and contributors.
Read original
reddit/r/LocalLLM