Cmpnd released dspy-wsp, an MIT-licensed DSPy optimizer implementing weight-space programming (WSP), where a frontier LLM (GPT-6-Astra) directly edits a student model's weights inside a sandboxed kernel without gradients, training loops, or fine-tuning datasets. The WeightSpace optimizer integrates with any DSPy program and metric, requiring DSPy 3.5. Tested on open decision models (0.6B–9B parameters) running locally on macOS (2B models ~6 GB RAM), 25-pass optimization runs cost $5.42–$7.63 and took 78–167 minutes. On the Typed Decisions benchmark (400 cases, 2,000 decisions), WeightSpace alone added 1.9–16.1 accuracy points; combined with ReAnchor (threshold recalibration), gains reached 6.5–20.0 points. The strongest result: d1-omni-600M rose from 50.9% to 69.1% (+18.3) with both optimizers. Edits were highly targeted—e.g., 112,640 of 1,048,576 head entries for d1-omni-600M, a rank-9 change to all 15.7M entries in layer 23 for d1-2B, and a single scalar for NanoJev (Qwen3-0.6B). Gains diminished on the 9B Clef-Flash (+1.9 alone, +6.5 with ReAnchor). Writer model choice mattered critically: GPT-6-Astra outperformed gpt-6.1-sol (+16.1 vs +5.8 on d1-omni-600M). Enforcing the original paper's stricter rule (simulate_head=False) reduced d1-omni-600M gains to +2.8. Patches export as hashed manifests (manifest.json, edit.npz) for verifiable reconstruction via MLX (Apple) or PyTorch/vLLM (Linux/NVIDIA). Backends include D1Backend, NanoJevBackend, D1LMBackend, and LLM backends (VLLMBackend, HFBackend, MLXBackend).
Read original
marktechpost