🔍 Read the full analysis: GPT-6 Astra’s AI-to-AI Repair Shows What A Weight Edit Can Do on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A Nov. 2 blog post by four researchers affiliated with MIT CSAIL reports that GPT-6 Astra edited two layers of Qwen3.5-0.8B without gradient descent. The researchers report a rise in MATH-500 accuracy from 7.9% to 48.1%, while answers hitting the token limit fell from 93.0% to 33.1%. The result is an early, unreviewed report, and the method involved extensive iterative testing rather than a single automated fix.
The researchers describe the work in “Sparks of Weight-Space Programming”, a blog post by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. The authors are listed with MIT CSAIL, with Takehi and Teoh identified as equal contributors. The post is not a peer-reviewed paper, and the authors call the findings “very early results.”
According to the published logs, Astra traced the repeated answers to a direction associated with the multilayer perceptrons in layers 15 and 19 of Qwen3.5-0.8B. It subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or 0.016% of the model’s 0.8 billion parameters. The researchers say the resulting model is available on Hugging Face; its published files show differences in the two layers’ down-projection weight tensors.
The edit followed iterative testing, not a one-shot instruction. In the reported 50-pass run, Astra used 158 turns, 80 Python runs and 50 model forward passes over 20.3 hours. The system had access to practice questions, but not the final benchmark questions. The researchers barred gradient descent, backward passes and fitting weights to a loss function. They also report that smaller budgets produced much smaller MATH-500 gains: 12.3% after 10 passes and 14.1% after 25, compared with 48.1% after 50.
A Targeted Repair, Not New Math Ability
The report offers an example of a model being used to inspect and edit another model’s open weights, rather than merely generating an answer or suggesting code for a conventional training run. For organizations that can access model weights, this points to a possible way to diagnose and change a specific behavior without running gradient-based training.
But the measured improvement is best understood as a repair to a failure mode, not evidence that Astra gave Qwen a general new mathematical capability. The authors say the original model’s looping obstructed its ability to finish answers. They caution: “Fixing loops, however, is not the same as making a model better at math.” The edited model still hit the token limit in 33.1% of MATH-500 responses, and the results come from one small model and a particular experimental setup.
The process also had substantial costs and constraints. The strongest result required 20.3 hours and repeated access to the model for evaluation. The findings do not establish how reliably the method works across architectures, tasks or deployment settings. A weight edit also creates a changed model that must be checked for regressions, not just tested on the behavior it was intended to fix.
Products Worth Considering
Logitech MX Master 4 for Mac Wireless Ergonomic Mouse, Space Black
Bose QuietComfort Wireless Noise-Cancelling Over-Ear Headphones, Black
Samsung Galaxy Tab A11+ 11-Inch Tablet, 6GB RAM, 128GB Storage, Expandable, Gray
As an affiliate, we earn on qualifying purchases.
How the Weight Edit Was Tested
Qwen3.5-0.8B is a small open model, and its model card warns that it can fall into thinking loops. In the researchers’ initial practice sample, 85 of 96 answers reached the 32,768-token limit. Their intervention focused on that repetitive behavior by comparing looped and non-looped reasoning outputs and searching the weights for a direction linked to the loops.
The reported MATH-500 results, using the same sampling settings for the base and edited versions, were 7.9% for the base model, 12.3% after 10 passes, 14.1% after 25 and 48.1% after 50. The corresponding shares of answers reaching the token cap were 93.0%, 87.2%, 86.4% and 33.1%. On GSM8K, the reported score rose from 13.8% to 67.1% after 50 passes; on IFEval, it rose from 21.0% to 62.3%.
The post describes eight experiments, including attempts to infer information from model weights and repair a driving policy. At least one reported task failed: Astra could not determine from weights alone which of two fine-tuned Gemma models had been trained to flatter a user. The authors also report that experiments on Gemma 3 270M showed no edit moving the score by more than about a point. These examples frame the Qwen result as a specific success within an early set of tests, not a general capability guarantee.
““Fixing loops, however, is not the same as making a model better at math.””
— Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, in the blog post
Limits of the Reported Evidence
The findings have not been peer reviewed, and the post is not listed on arXiv according to the source material. The reported benchmark gains are from the researchers’ experiments; they do not establish that the edited model will perform similarly on other datasets, prompts or real-world uses. One-third of MATH-500 responses still reached the token cap after the edit, so the looping behavior was reduced, not eliminated.
It is also unclear how often an AI system can identify a useful weight edit without extensive human-designed constraints, practice data and repeated evaluation. The authors’ tests included cases where Astra did not succeed, and the experiments on a separate small model showed little improvement. OpenAI is not listed as a co-author, and the source material reports no statement from the company about the work. The post also does not establish whether the approach is safe or reliable for models used in production.
Replication and Broader Model Tests
The immediate next step is independent replication: researchers can examine the published logs and edited model, then test whether the reported changes and scores hold under the same settings. Further work would need to establish how the method performs across more models and tasks, whether it preserves abilities unrelated to looping, and what checks are needed before deploying an edited model.
The authors describe the findings as early, and the source material gives no schedule for a peer-reviewed paper or further experiments. Until more evidence appears, the result is a documented demonstration of a targeted weight edit in one small model—not proof that AI systems can routinely improve other models without conventional training.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra subtracted a direction associated with repetitive answers from output columns in the down-projection weights of Qwen3.5-0.8B’s layers 15 and 19.
How much did the reported MATH-500 score improve?
The researchers report that the score rose from 7.9% to 48.1% after the 50-pass run. They also report that responses reaching the 32,768-token limit fell from 93.0% to 33.1%.
Was the model improved without any training or testing?
No gradient descent or backward passes were used, but Astra carried out an iterative search with practice data and model evaluations. The strongest run took 20.3 hours and included 50 forward passes, according to the published logs.
Are the results peer reviewed, and are the loops gone?
The results were published in a blog post the authors call early work, not a peer-reviewed paper. The reported token-limit rate remained 33.1% after editing, so the looping behavior was reduced but not eliminated.
Source: ThorstenMeyerAI.com
