A Reddit user demonstrates a $1,500 AI inference rig using six NVIDIA Tesla M10 GPUs, providing 192GB of total VRAM for running Qwen 3.8 Flash Next at 52.5 tokens per second in FP8 precision. Despite each M10 containing four GM107 cores connected via an internal PCIe switch — a configuration typically dismissed as unsuitable for AI workloads — the setup achieves viable performance for large language model inference. The project details and hardware configuration are documented in a public GitHub repository.
Read original
reddit/r/LocalLLM