SonicBoom v0.0.3 is an open‑source (GPLv3) inference runtime that eliminates Python, using a C ABI and Guile Scheme as its first‑class language. Benchmarks on an RTX 3050 8 GB show it achieves 1.18× faster decode throughput for Gemma 4 E4B Q3_K_M compared to llama.cpp (‑ngl 99) while matching prefill performance. The project invites GPU owners to help further evaluate its speed.
Read original
reddit/r/LocalLLaMA