The second round of the browser agent benchmark evaluated six local setups and Grok Bot on 100 live Online-Mind2Web tasks using a datacenter server with a 10-minute cap, headed browsers, and a single attempt per task. BrowserOS (Claude Code) led with 40 completed tasks, followed by Claude in Chrome (37), 5dive browser with Claude Code (34), OpenClaw (29), 5dive browser with pi (28), Hermes (27), and Grok Bot (18) running on its own cloud and model judged by WebJudge on GPT-6 Luna. The top-three differences were not statistically significant (p=0.29). BrowserOS excelled through robust fallback mechanisms: 94 screen-position clicks and 216 JavaScript calls, yielding a 6% tool-call failure rate versus 19% for 5dive. The 5dive pi variant reduced costs to 2.8¢ per finished task versus 6.1¢ for Claude Code by using 20% fewer tool calls and less context (5.6K vs 17.7K tokens), though it scored lower partly because grep-based text extraction produced fewer screenshots for the judge. Post-round-1 fixes cut 5dive's step time from 9.1 to 7.9 seconds (target <5s) but barely improved success (32 to 33). Approximately 20% of runs encountered bot walls; seven tasks blocked all agents. Sonnet 5.5 underperformed Haiku 5.5 (10 vs 15 passes at 10x cost). On four everyday tasks, all local agents succeeded; BrowserOS was fastest (19.1s), while 5dive pi trailed by 0.7s at ~$0.82/1,000 tasks with fewer steps (4 vs 9.5). Round 3 will add BrowserOS-style fallbacks to 5dive.
Read original
dev.to