NVIDIA announced the Nemotron Bolt 3.5 74B A9B, a 74‑billion‑parameter mixture‑of‑experts model with 9 B active parameters, quantized to 2 bits to fit in 24 GB while preserving quality via quantization‑aware training. The model is designed to be faster than typical dense models and more capable than comparable MoE models at a similar memory footprint. A smaller variant, the Nemotron 3.5 Bolt E4B 4GB, is slated for release on October 23.
Read original
reddit/r/LocalLLaMA