GPT-6 Astra edited 0.016% of Qwen's weights and its math score went up 6x

Rikiya Takehi (@rikiyatakehi), a first-year PhD student at MIT CSAIL, and co-authors say GPT-6 Astra directly edited Qwen3.5-0.8B's weights to fix reasoning loops and raised its MATH-500 score from 7.9% to 48.1% without gradients or training. Takehi published the results with Jayden Teoh, Alex Zhang (@a1zhang) and Omar Khattab (@lateinteraction) on November 2nd, 2026, calling the approach Weight-Space Programming in a thread on X and a blog post.

Takehi lists MIT CSAIL as his affiliation and Omar Khattab as his advisor. He arrived at MIT after a B.Eng. from Waseda University and research internships at NVIDIA Research and Mixedbread, plus guest research at NIST. The authors list all four affiliations as MIT CSAIL in the blog post, with Takehi and Jayden Teoh (@jayden_teoh_) as co-first authors.

The Qwen edit

The headline experiment asked Astra to make Qwen3.5-0.8B better at math. The authors say Astra traced repeated looping to one direction written by the MLPs of layers 15 and 19 and subtracted that direction from 64 output columns in each layer, for 131,072 weights in total. That is 0.016% of the 0.8 billion weights, a figure the authors state directly.

With 50 forward passes, the authors report MATH-500 rising from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. IFEval, which measures instruction following, rose from 21.0% to 62.3% in the same run. The share of MATH-500 answers hitting the 32,768-token limit fell from 93% to 33.1% with 50 forward passes, while 10 or 25 passes left the rate near 87%.

The setup matters for reading those numbers. Astra worked through a Python kernel over the weights with no GPU, using practice problems from GSM8K and MATH training sets, and the authors say it never saw the evaluation sets. Qwen's own model card notes that this model is prone to thinking loops, which the authors cite as the failure Astra targeted. The claimed gain therefore comes from removing a failure mode that kept the small model from finishing answers, a point the authors make themselves when they test a second model below.

Reading, fixing and writing smaller networks

The Qwen result sits at the end of a sequence of smaller experiments. In the first, Astra received raw weights from four game policies with identical architecture, inputs and actions, and named all four games correctly: Breakout, Pong, Pac-Man and Freeway. The networks shared the same dimensions, so the authors argue the answer had to come from the weights themselves.

In a control task, a PPO car policy kept stalling on a ramp. Given the weights, a scene description and one failed episode, Astra edited 3.5% of the weights in a single reply and the authors report 100% success across 1,024 episodes. In MinAtar games, the authors say networks Astra wrote from scratch beat PPO trained for 100 million steps in Breakout, Seaquest and Asterix, and scored 517.4 in Breakout against 25.3 for PPO trained for 500 million steps. Those PPO comparisons are the authors' reported scores from their own table.

The authors define Weight-Space Programming as a general-purpose LLM reading and writing a network's weights directly around a natural-language specification, without an optimizer and without gradient descent. They say the work differs from LLMs writing training code because Astra was not allowed to write training loops in these experiments.

The limit the authors found

Astra failed to improve Gemma 3 270M, a model that rarely loops. Only 8.5% of Gemma's answers reached its token limit, against 93% for Qwen, and the authors report none of Astra's edits moved MATH-500 by more than about a point. The authors' own conclusion is narrow: editing weights could remove an obstacle that hid an ability, and it could not add an ability the model lacked.

That negative result frames the Qwen numbers. A 40-point MATH-500 jump from deleting one direction says as much about how badly the 0.8B model was looping as it does about Astra adding mathematical skill. The authors describe the overall findings as early and say they are building a benchmark and a harness to help LLMs explore and write weights. The blog post states the current results are limited by context length and harness inefficiencies.

No independent replication, company, product or funding round accompanies the post. What has been published is a set of author-reported experiments, a method name, and a specific, testable claim: 131,072 edited weights, identified layers, and benchmark scores that other researchers can now try to reproduce.