Researchers evaluated OpenAI's GPT-6 Astra against Anthropic's Claude Fable 5 and Fable 5.1 using YAM bimanual arms controlled via the Inspect Robots agent policy. The evaluation focused on two manipulation tasks: placing a red block into a bowl and inserting a blue puzzle piece into a circular groove. GPT-6 Astra demonstrated significant performance advantages in the bowl task, achieving a 95% success rate (19 of 20 trials) with a mean score of 3.95, compared to 40% for Fable 5.1 and 5% for Fable 5. In the bowl task, Astra also proved more efficient, averaging 2.5 minutes per run at an estimated cost of $0.94, whereas Fable 5.1 took 6.8 minutes at $2.12. However, for the more complex puzzle task, Astra's performance matched Fable 5.1, with both models completing only 2 of 20 trials and reaching a mean score of 2.00. Technical metrics show Astra utilized significantly fewer output tokens (averaging 2.1k for the bowl task) compared to the Fable models, contributing to its lower operational cost. The robotic setup utilized three camera views and absolute end-effector pose commands, with all models operating under a 20-LLM-call budget and medium reasoning effort.
Read original
hackernews