The user loaded Ling 3.1 Flash, a 560 billion‑parameter mixture‑of‑experts model with ~25 billion active parameters per token, across four NVIDIA DGX Spark systems. The model was distributed via llama.cpp RPC using Atomic Chat’s day‑one Q4 GGUF quantization after it would not fit on two Sparks. Access to the four‑Spark setup was granted by a collaborator in Germany, enabling the full‑scale inference test.
Read original
reddit/r/LocalLLaMA