A new open-source benchmark called Parsimony measures code complexity by counting syntax units—statements, calls, and names—in patches generated by AI models and harnesses, rather than lines or comments. Unlike existing benchmarks focused on solve rates and token costs, Parsimony penalizes excess code and overengineering that increase technical debt, comparing each model's output against human fixes for the same issues. Surprisingly, smarter models tend to write slightly more code than simpler ones.

Read original