Ryan Madden reports that GPT-5.6 Sol, operating within the Codex environment at low thinking level, completed a peaceful-mode Factorio run — launching a rocket without enemy biters — in 13.5 hours of in-game time. The agent interacted with the game through a Python-based library that enforces human-equivalent constraints such as player reach and fog of war, replacing the Bash interface used in prior Claude Code experiments. Production statistics were logged throughout, showing resource output aligned with science milestones. The run was initially slated to end at blue science, but Sol’s unexpected competence prompted continuation. However, scaling difficulties emerged in later stages, notably failures to properly connect production systems, which Sol compensated for by manually ferrying materials across the factory — a workaround that ultimately enabled the rocket launch. This marks a clear improvement over GPT-5.5, which failed to utilize transport belts at all. Madden views autonomous Factorio play as a legible, programming-adjacent benchmark for evaluating model capabilities and plans further tests across models, thinking levels, and non-peaceful scenarios, with a commentary video and detailed analysis forthcoming.
Read original
hackernews