RustingStrata is a pure Rust inference engine for Qwen GGUF models that compiles CUDA kernels at startup via NVRTC and supports Vulkan execution. It runs Qwen3.5/3.6 (2B–35B-A3B), Qwen3 (4B–30B-A3B), Qwen3.8-27B, Qwen3-Next-80B-A3B, and Qwen3.8-Flash-Next with quantization from IQ1_S to Q8_0 plus BF16/F16. The engine reportedly runs Qwen3.8-Flash-Next faster than the original Strata on a 12 GB GPU. Key optimizations target MoE model efficiency.

Read original