The benchmark showed a significant performance drop for mobile robotics, where no model reached a score of 2/100 on 11 humanoid tasks requiring independent balance and walking. GPT-6.1 Sol was the most cost-efficient, averaging $1.93 per task, compared to $16.36 for GPT-6 Astra. Reaction times also varied, with GPT-6 models initiating movement after 75 to 80 seconds, while Grok 4.7 and Gemini 4 Argon failed to move the robot in most episodes.
Tuesday, October 20, 2026
Story
All Frontier AI Models Fail Humanoid Tasks in First Robotics Benchmark
AI6 days ago
Artificial Analysis launched AA-Robotics v0.1, a research preview gauging the ability of language models to plan and act in physical environments using a 6-axis robot arm. GPT-6 Astra placed first on the leaderboard with a score of 69, followed by GPT-6.1 Sol at 65 and Gemini 4 Argon at 55. While the models can now manipulate arms for simple tasks, they remain roughly 100x slower and 10x more expensive than human baselines, averaging 30 minutes to complete one tabletop task.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.