Netlify has expanded its AI capabilities by integrating OpenRouter into its AI Gateway and extending the selection of models available via Agent Runners, which utilize full coding agents rather than pared-down versions. To assist users in navigating model selection, Netlify conducted an empirical study using a single prompt to generate a one-page coffee shop website across 11 different models. Using the open-sourced AXIS evaluation tool, researchers measured design quality and credit consumption. The results revealed a massive variance in cost-efficiency: DeepSeek V4 Flash (0731) was the most economical, averaging only 2.4 credits per run, whereas Claude Opus 5 was the most expensive, averaging 519 credits, with a single outlier run reaching 1,055 credits. While Opus 5 produced the most sophisticated visual designs and vector graphics, other models offered different trade-offs. For instance, GPT 5.6 Sol (low effort) outperformed Claude Sonnet 5 in design intuition while remaining more affordable at 141 average credits. Mid-tier models like Gemini 3.6 Flash (103 credits) and Kimi K3 (102 credits) provided modern results, while highly efficient open-weight models like Kimi K2.7 Code (19 credits) and GLM 5.2 (27 credits) provided much simpler outputs. This testing highlights the tension between high-fidelity, high-cost "turnkey" models and more economical, iterative approaches.
Read original
hackernews