Needle 2 is a 14MB open-source foundation model from Cactus Compute that enables offline tool calling, device control, and structured data extraction on low-power hardware. With 45 million parameters compressed to 2-bit precision, it operates efficiently on devices like the Raspberry Pi 5, achieving over 500 tokens per second. The model’s lightweight design allows it to fit within 28MB of RAM while maintaining performance. Needle 2 gained significant traction, reaching #1 on GitHub Trending after amassing nearly 5,000 stars in its first days.
Read original
dev.to