Researcher dbreunig presents nesformer, a compiler that embeds a complete Nintendo Entertainment System (NES) 6502 CPU, 2 KB work RAM, and 32 KB PRG-ROM into the weights of a stock Qwen3.5-4B model (Qwen3_5ForCausalLM architecture, 768 hidden size, 7 Gated DeltaNet layers plus one all-zero attention layer, 466.6M bf16 parameters zipping to 3.3 MB). Building on Qwenputer's technique, the model's Gated DeltaNet state serves as the machine's memory: each DeltaNet head maintains a 128×128 matrix where gated, one-hot key/value updates implement exact memory cell reads and writes. The compiler computes all weights statically—no training occurs. A host program (nesformer/host.py) handles PPU scanline rendering, controller input, APU register writes, and vblank NMI injection, while the model predicts the CPU execution log token-by-token via greedy decoding. The system supports NROM (iNES mapper 0) cartridges only, covering all 151 official 6502 opcodes, addressing modes, flags, stack, BRK/NMI, and OAM DMA. Verification shows exact trace equivalence: the reference CPU matches nestest (3,348 instructions) and py65 (5.7M game instructions); the compiled circuit predicts every reference token for nestest plus 283,610 game boot positions; the free-running weights generate identical traces for nestest (4,586 tokens), Thwaite (700k tokens/271 frames/4.5s, 1.9M tokens/600 frames/10s), and Full Game (2.6M tokens/420 frames/7s), with all saved frames matching a conventional emulator. An idle-loop skip reduces token count 2.6–4.2× by advancing the clock during vblank waits. Performance: ~1,650 tokens/s (numba engine), ~210 tokens/s (numpy), ~10 tokens/s (stock transformers), yielding 150–290× slowdown versus real hardware (600 frames ≈ 25 minutes). Limitations: no bank-switching mappers, no unofficial opcodes, no RAM execution, no IRQs, incomplete I/O behavior, and no mid-scanline PPU effects or audio. The checkpoint loads via standard Hugging Face transformers; CLI supports run, check, trace, and build commands with configurable engines and input scripts.
Read original
hackernews