Xolotl, a Strata‑based inference engine, enables the massive DeepSeek‑V4.1‑Flash models to run on a single consumer PC by distributing weights across VRAM, pinned RAM and NVMe and loading only the needed experts per token. The author quantized the 763 B DeepSeek‑V4.1‑Flash‑1017 model for this engine, providing a 153 GiB GSQ mixed‑precision build that anyone can use. This combination makes cutting‑edge Mixture‑of‑Experts LLMs accessible on ordinary hardware.
Read original
reddit/r/LocalLLM