A developer has successfully run Qwen3.5-9B on a stock PlayStation 4 GPU, achieving 31–39 tokens per second offline. The model leverages its built-in Multi-Token Prediction (MTP) head for speculative decoding, drafting up to two tokens verified in a single forward pass, which significantly accelerated inference from a baseline of 22 tok/s. Weights were quantized into custom 4-bit and 3-bit formats to fit within approximately 4.5 GiB of VRAM, enabling the console to perform chat, coding, and audio tasks locally without cloud connectivity.
Read original
reddit/r/LocalLLaMA