The benchmark change significantly impacted top performers, dropping Muse Spark 1.3 from a pre-gate lead of 26.7% to 8.9%, while GPT-6 Astra rose from joint 10th to 3rd place due to its high grounding. The new scoring methodology follows human expert studies that identified hallucinations as the primary factor in response preference and employs a three-judge panel consisting of GPT-6 Sol, Grok 4.7, and Claude Opus 5.5. In terms of cost, Grok 4.7 costs about $9.50 per task, less than half the price of the most expensive model, Claude Fable 5.1.
Tuesday, October 20, 2026
Story
Grok 4.7 Tops New Legal AI Benchmark With 9.4% Score
AI12 days ago
Artificial Analysis and Harvey released Harvey LAB-AA v1.1, introducing a metric that credits legal AI tasks only when outputs satisfy every criterion without material hallucinations. xAI's Grok 4.7 (xhigh) now leads the rankings with a 9.4% Hallucination-Gated All-Pass Rate, followed by Meta's Muse Spark 1.3 at 8.9% and OpenAI's GPT-6 Astra at 8.6%. This gated approach penalizes models for fabricated content or contradictions, which reshuffled the leaderboard because more than 60% of otherwise passing responses across all models contained substantive errors.
Looking for more context or analysis?
Tilt chat provides reliable, tailored insights, alerts, and automations.