Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

Article automatically generated from technical news.

I managed to run the 143–144 GiB DeepSeek-V4-Flash-0731 UD-Q4_K_XL GGUF on four RTX 3060 12GB cards while keeping a 360k–376k context window. Hardware: CPU: Intel Core i9-10920X, 12C/24T RAM: 128 GB DDR4-3200, quad-channel GPU: 4× NVIDIA RTX 3060 12GB Total VRAM: 48 GB Storage: NVMe SSD Engine: llama.cpp, build b10181 Model: unsloth/DeepSeek-V4-Flash-0731-GGUF Quant: UD-Q4_K_XL, approximately 144 GiB KV cache: Q8_0 The best high-speed configuration so far:

Fonte originale

Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

Related Articles

Some people said the Minecraft clone I fully vibecoded with Qwen3.8-27B Q4 is not that impressive because Minecraft is in the training data, so I had the model add 4 things that are probably not.