FrontierCode 1.1 uses 100 private tasks based on actual repositories, requiring agents to pass both unit and qualitative tests. While performance is improving, Artificial Analysis found that top models still fail more than 40% of tasks, and nine items in the set were not passed by any model. OpenAI's GPT-6.1 Sol emerged as a cost-efficient alternative, scoring 49% at $0.81 per task, compared to $10.15 per task for the Opus 5.5 model.
Wednesday, October 21, 2026
Story
Gemini 4 Argon Tops AI Code Merge Test With 60.5% Score
AI7 days ago
Google has integrated the Gemini 4 Argon model into its Devin desktop and command-line interface after the system became the highest-scoring AI on a specialized mergeability evaluation. The model achieved a 60.5% success rate on Cognition's FrontierCode 1.1 benchmark, surpassing the previous top score of 52% held by Anthropic's Claude Opus 5.5. The benchmark specifically tests whether AI-generated patches would be accepted into real-world open-source repositories by human maintainers.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.