Anthropic's Fable 5.2 achieved a 41.3% completion rate on the Remote Labor Index (RLI) benchmark from Scale Labs and the Center for AI Safety (CAIS), a substantial increase from the 2.5% recorded by Manus at the benchmark's launch roughly a year prior and a 2.5x improvement over Fable 5's 16% score in July. Fable 5.2 outperforms its direct predecessor Fable 5.1 (just under 18%), Opus 5.5 (21.25%), OpenAI's GPT 6 Astra (20.83%), and Google's Gemini Argon 4 (17.08%). The RLI evaluates agents on 240 real Upwork freelance projects worth approximately $144,000 total (averaging $632.60 and 29 human hours each), spanning video editing, CAD design, game development, and data analysis. Agents operate in a virtual workspace equipped with professional software including Blender, GIMP, and Audacity, plus specialized generative AI tools. Scale Labs reports Fable 5.2's largest gains in audio and music editing, architecture and interior design, and 3D model creation, with nearly 50 successful completions representing tasks no prior agent had solved. Persistent weaknesses remain in industrial product design and animations. Trained human experts manually verify completions by inspecting submitted files in native software; CAIS found automated AI review unreliable because AI reviewers cannot adequately operate professional tools and are fooled by superficially polished but technically flawed outputs.

Read original