Critics examining logs from MIT CSAIL’s weight space programming study found that GPT-6 Astra did not add math skills to Qwen3.5-0.8B, as 576 of the 579 practice answers it repaired had merely been cut off at the 32,768 token limit during repetitive thinking loops. Running Qwen3.5-0.8B in its default non-thinking mode with zero weight edits yields 47% accuracy on MATH-500, nearly matching the 48.1% score after Astra edited 0.016% of the model’s weights from a 7.9% thinking-mode baseline. On problems Qwen already finished before the edit, accuracy did not change.
Follow-up tests by outside developers showed that direct weight interpretation and editing are not exclusive to frontier models like GPT-6 Astra. Simon Willison reported that an open-weight Qwen 3.8 27B model running locally on a laptop reproduced the paper’s weight-reading task after 45 minutes. Separately, Adam Holter tested five models on direct weight editing, reporting that Anthropic’s Fable 5.1 nearly doubled a small model’s math score by altering five of 134,515,008 weights, while Claude Sonnet 5.5 added 17 correct answers by making the model output “The” first, which cut formatting accuracy from 32 of 96 to 1.
Earlier, Hugging Face researchers reproduced the MIT paper using the open-source ML Intern harness, while an independent researcher used Claude Opus 5.5 and $1.03 in API calls to edit two neuron output columns in layer 29 of Qwen3.5-4B, eliminating repetition loops on 150 held-out math problems and lifting correct answers from 59 to between 92 and 94. MIT co-author Alex Zhang previously clarified that an LLM-optimized prompt hinted at Qwen’s looping issue and that Astra used activation steering across 131,072 weights in layers 15 and 19 to suppress it, boosting held-out problem accuracy from 10.6% to 60.4% while producing no clear math gain on Google’s Gemma 3 270M.
