Qwenputer embeds a deterministic 32-bit RISC-V CPU (RV32IM with Zba and a bignum unit) into the weights of a stock Qwen 3.5 4B model, using its Gated DeltaNet (GDN) layers as exact, addressable RAM. By setting the GDN gate to 1.0, write strength (beta) to 1.0, and employing one-hot key encoding, each head's state matrix becomes a non-fading memory where writes overwrite single rows and queries retrieve exact values, eliminating the growing KV cache of standard attention. A compiler translates CPU circuit steps — register reads, arithmetic, branches — directly into transformer components across 11 prepended layers: attention heads perform lookups, feed-forward units execute arithmetic, and GDN heads store state. Execution proceeds during autoregressive decoding; each instruction emits ADDR, VAL, and STEP tokens as an execution trace, driven by a small RvChat driver that loads firmware into GDN state and isolates trace tokens from Qwen's layers. The CPU passes official riscv-tests, matches QEMU output, and completed 18 million free-running instructions with zero errors against a reference replay of 230,000 positions. On DeepMind arithmetic benchmarks, Qwenputer scores 97.5% on div_big and mul_big, 100% on mul_div_multiple_longer, and 97.5% on add_or_sub_big, matching or exceeding Python-tool performance (95–100%) and far surpassing base Qwen (55–90%). NUPA long-integer accuracy jumps from 48.6% to 100%. KL divergence on normal text remains near noise floor (6e-4 vs 5e-4). An experimental in-model filesystem hashes filenames into 1,024 directory slots in GDN state, stores content chained at eight tokens per row, supports 512 files (~53k tokens), persists across sessions via state save/load, and achieves 90% cross-chat fact retrieval versus 30% with pasted context (Python tools: 83.3%). The 18.8 GB checkpoint (double base size, mostly zero padding) runs ~3,200 CPU tokens/sec on an L4; future work targets MicroPython execution and speedups.
Read original
hackernews