Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights

📝 摘要 #

本文提出 Weight-Space Programming（权重空间编程）：通用 LLM 不经优化器、不做梯度下降，直接用自然语言推理读取和写入神经网络的原始权重。作者（Rikiya Takehi、Jayden Teoh、Alex Zhang、Omar Khattab）用 GPT-6 Astra 做了 8 组实验，覆盖"读""修""写"三类能力。读取方面，Astra 仅凭权重就能说出网络训练自哪个游戏、DDPM 生成什么形状、小语言模型训练于 Python/罗马字/英语中的哪一种，并准确定位 transformer 中间层的 induction head；修复方面，它用 3.5% 的权重编辑让 Kinetix 卡在坡道的 PPO 策略全部通关，并通过修改 Qwen3.5-0.8B 仅 0.016% 的权重消除推理循环，把 MATH-500 从 7.9% 提升到 48.1%、IFEval 从 21.0% 提升到 62.3%；写入方面，它零样本手写 MinAtar 三个游戏的网络，击败训练 1 亿步的 PPO，并在 50 次 forward pass 预算下写出能战胜同等规模 PPO 的 9×9 围棋网络。

核心概念： 权重不只是优化的产物，也可以像代码一样被具备推理能力的模型直接"阅读、调试、改写"；优化正在从 Adam/Muon 这类符号优化器，转向由 LLM 对权重本身进行推理的原生优化。

关键技术栈 / 关键要点：

1. 读懂权重空间 — 从第一层权重分布推断控制对象（Breakout 底部一行的 paddle）、从输出层单元权重拼出图形轮廓、从中间层注意力头定位复制头（2.1/3.1/3.5 等），且不运行任何前向传播。
2. 定向权重编辑 — 在 Kinetix 中只改 6,073/171,664 个权重，插入一条"倒车—加速—起跳"的子路径；在 Qwen 中只改 131,072/0.8B 个权重，从第 15、19 层 MLP 的 64 个输出列中减去推理循环方向。
3. 零样本写出网络 — 给出游戏规则、网络结构与评分方式，Astra 把网络当作程序来设计：Breakout 第一层算落点、第二层控制 paddle；围棋网络则写成逐层实现提子/救子/打吃/做眼的 Go 程序。
4. harness 与预算扩展 — 权重超出上下文时提供 Python 沙箱与有限 forward pass（0/10/25/50），Astra 交替编辑与试玩，按胜率筛选改动；50 次前向即可从零写出接近 PPO 500M 步的网络。
5. 能力边界 — 权重编辑能移除"遮住能力"的障碍（如推理循环），但不能凭空添加模型不具备的能力：对不循环的 Gemma 3 270M，所有编辑均无明显提升；纯权重判断谄媚行为也失败，需退一步看注意力模式才能区分。
6. 未来方向 — 无评测审计（从权重发现后门/隐藏能力）、大幅降低优化成本、零数据能力增删与遗忘、递归自我改进、真正的跨架构模型合并；作者正在构建标准化 benchmark 与 harness。

关键结论 / 实践建议： 这是早期但方向明确的结果——前沿 LLM 已具备原生读写权重的"火花"，瓶颈当前在上下文长度与 harness 而非推理能力本身。对可解释性研究，可把 LLM 当作"语言↔权重"的翻译器直接做电路级审计；对模型优化，优先考虑少量定向编辑（0.016% 量级）以规避灾难性遗忘与全量重训成本。

LLMs can natively read & write neural network weights #

TL;DR

We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability.

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.

The network playing Breakout.

If you were asked to handcraft a neural network that plays Go, what would you do? If you were handed raw neural network weights and asked to predict their behavior, where would you even start? For almost any human, this is practically impossible.

Today, weights are written by optimization, by custom networks trained to write them, or by compilers from formal programs. In this post, we find that the current generation of frontier LLMs, used natively in natural language, are starting to become eerily good at understanding and crafting neural network weights around natural-language goals.

Weight-Space Programming #

For a few years now, we’ve all been using LLMs to optimize text artifacts like prompts and the programs built around them

• Large Language Models as Optimizers   [PDF]
  C. Yang, X. Wang, Y. Lu, H. Liu, Q.V. Le, D. Zhou, X. Chen.
  International Conference on Learning Representations (ICLR). 2024.
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines   [PDF]
  O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T.T. Joshi, H. Moazam, H. Miller, M. Zaharia, C. Potts.
  International Conference on Learning Representations (ICLR). 2024.
• Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs   [PDF]
  K. Opsahl-Ong, M.J. Ryan, J. Purtell, D. Broman, C. Potts, M. Zaharia, O. Khattab.
  Conference on Empirical Methods in Natural Language Processing (EMNLP). 2024.
• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning   [PDF]
  L.A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M.J. Ryan, M. Jiang, C. Potts, K. Sen, A.G. Dimakis, I. Stoica, D. Klein, M. Zaharia, O. Khattab.
  International Conference on Learning Representations (ICLR). 2026.

[1, 2, 3, 4]

. More recently, we’ve started using them to optimize whole harnesses

• Meta-Harness: End-to-End Optimization of Model Harnesses   [PDF]
  Y. Lee, R. Nair, Q. Zhang, K. Lee, O. Khattab, C. Finn.
  Conference on Language Modeling (COLM). 2026.

[5]

and the training code and hyperparameters that train ML models

• autoresearch   [link]
  A. Karpathy.
  GitHub repository. 2026.
• Reactive Optimizers   [link]
  X. Bai, J. Liang, J. Teoh, V. Sitzmann.
  Blog post. 2026.

[6, 7]

.

We often justified these methods by reference to the fact that we don’t know other principled ways to hill-climb on text artifacts, unlike for updating the weights of differentiable neural networks. After all, humans can “optimize” code by hand, for example, so it made sense to ask LLMs to do it, but they can’t “optimize” neural weights by hand, so those remained largely in the realm of symbolic algorithms.

In this post, we share very early results that allow us to argue that the paradigm of reasoning-driven optimization is now coming even for differentiable neural networks.

In Weight-Space Programming, a general-purpose LLM reads and writes a network’s weights directly (i.e., not through an optimizer, and in particular not with gradient descent) around some natural language spec, sometimes even with no environment feedback at all.

In natural language, it’s becoming possible to “just ask” an LLM to natively find a hidden trait or failure inside the weights of a neural network, to fix that failure, or to write a new network from scratch. We report early, exciting results on each of these below, first on small networks and then even on real language models:

1. LLMs can read neural network weights: In Experiments 1 to 3, we ask Astra questions like what game a network was trained to play, what shape is drawn by a diffusion model, and which language a small language model was trained on. In Experiment 7, we ask which of two language models was trained to be sycophantic.
2. LLMs can fix neural network weights: In Experiment 4, we ask Astra to fix a network that’s failing at some physical control task. In Experiment 8, we ask it to make Qwen3.5-0.8B better at math, and its fix improves instruction following too.
3. LLMs can write neural network weights from scratch: In Experiment 5, we ask Astra for networks that play three MinAtar games, and in Experiment 6, for a network that plays Go.

Can LLMs understand model behaviors from weights? #

We start with a small-scale experiment to see if LLMs can understand the continuous weight space, which is essentially just a list of numbers.

In our first experiments, we train some neural network with some algorithm. It would be extremely exciting if the LLM is able to tell its behavior just from the weights.

Experiment 1

Can Astra tell which game the network was trained on just from the weights? #

We trained neural network policies with PPO to play four computer games. All four networks take the same input channels, choose from the same actions, and have exactly the same network dimensions, so that it is not able to tell unless the weights are read.

For each of these networks, we pass its raw weights into Astra as a prompt, and ask: Which game was this neural network trained on? Without revealing the options of the games we consider, Astra was still able to name the right game for all four networks. This is exciting particularly because there are so many games out there that Astra could choose from.

The Breakout networks playing.

It seems that Astra was able to work out what game each network was built to play, just from features of its weights. For example, where each input channel’s first-layer weights concentrate (e.g., for Breakout, the agent’s weights lie along the bottom row, a paddle), and which actions the output layer tells apart.

Experiment 2

Can Astra tell what the diffusion model will generate? #

Next, we wanted to see if this generalizes to other tasks and models.

We train diffusion models (DDPM) to generate circle, triangle, and square images independently. We then ask Astra if it could tell what the DDPMs are trained to generate, just from the weights alone. We do not allow tool use or forward passes.

Setup

• Models: three diffusion models (DDPM) of 12×12 grayscale images, one each for filled circles, squares and triangles
• Network: MLP 160 → 64 → 64 → 144, ReLU (23,824 weights)
• Input: a noisy image (144 pixels) and the diffusion step (16 numbers)
• Output: the clean image (144 pixels)
• Training: Adam, 40,000 steps, batch 512
• Astra gets: all the weights as text, with no shape named, and is told not to compute any forward pass

CirclesSquaresTriangles

The model generates

the model’s 23,824 weights as text

W3 row 29 0.004 0.003 0.003 -0.001 0.007 0.062 0.114 0.082 0.024 0.005 0.004 0.003 …

What images does this neural network generate?

No tools or forward passes are allowed.

Answer Images: a single white circle outline on black, with a black interior 84% confident · ✓ Correct · 506 tokens

Reasoning

1. The output weights contain curved rim segments
2. This traces an upper-left bend
3. Other units describe complementary portions of a rounded boundary
4. These are localized bands with weaker or negative weights beside them, rather than uniformly positive filled regions
5. The complementary top, bottom, and curved side bands strongly suggest **circular or slightly oval outlines**

Astra’s drawing

Astra was able to tell what each model generates, from the weights alone. It finds the shape in the output layer, where each hidden unit’s weights draw one piece of the shape’s edge, and it draws just those edges. Interestingly, it draws every shape as an outline, ignoring that the actual models generate filled shapes. This mistake kind of shows it does not rely on calculating the forward pass, but rather it actually reads the structure/geometry of the weights.

Experiment 3

Can Astra tell what a language model learned, from the middle of the network alone? #

Now, in the first two experiments above, Astra was not really looking at the full network and could get away with only reading the first or the last layers. This is not a scalable strategy for larger (particularly language) models and problems we really care about, as how they process and think through their input often happens in the intermediate layers. So next, we take the first and last layers away and leave Astra with only the middle layers.

We train three small language models (7.2M parameters) to predict the next character, one each on Python code

^[1]

, Japanese in alphabet (romaji)

• aozorabunko-clean: the Aozora Bunko collection of Japanese literature, cleaned and deduplicated   [link]
  .
  Hugging Face dataset, GLOBIS University. 2023.

[8]

, and English

• Wikipedia, English, dump of November 1, 2023   [link]
  .
  Hugging Face dataset wikimedia/wikipedia, Wikimedia Foundation. 2023.

[9]

. We then remove the input and output layers and give Astra only the four transformer blocks in between. Without these two layers, nothing in the weights says which character is which, and the model cannot be run on any text. Astra may analyze the weights with Python, but it may not run the network or any of its layers on any input.

We ask: which language was this network trained on, and what each of its 24 attention heads does. Astra names the right language for all three networks, in both of its attempts. It also finds every copying head (induction head), the heads that let a model repeat text it has seen earlier, and names no other head as one.

Where the copying heads are

… tot al … tot al A copying head finds the same letters earlier in the text and copies what came after them.

copying headother headRows are layers 0 to 3, columns are heads 0 to 5Astra: from the weights. Measured: by running the model

Python

0123450123

Astra

012345

Measured

Romaji

0123450123

Astra

012345

Measured

English

0123450123

Astra

012345

Measured

Figure 1: The copying heads Astra read from the weights match the measured ones in all three models.

Setup

• Models: character-level transformers, 4 layers × 6 heads, width 384 (7.16 million weights), 28 characters: a–z, space and newline
• Training: 1 billion characters each, of Python code, Japanese literature in romaji (Aozora Bunko), or English Wikipedia
• Astra gets: the weights of the four transformer blocks only, with no input layer, no output layer and no labels on the 384 dimensions, and Python to analyze them, up to 100 runs
• Not allowed: running the network or any of its layers on any input, real, random or made up, and search. We checked every run
• Asked: which of the three texts the network was trained on, and what each head does
• Truth: each head measured by running the network on a random string repeated twice. Copying heads score 0.22 to 0.55, every other head at most 0.02. Astra never sees this

Show the prompt and Astra’s answersHide the prompt and Astra’s answers

PythonRomajiEnglish

Truth Trained on Python code. Its copying heads, measured by running it: 2.1, 3.1 and 3.5.

the weights of the four transformer blocks, as files

What was this network trained on: Python code, romaji or English? And what does each attention head do?

Running the network or any of its layers on any input is not allowed.

Answer Trained on: Python 56% confident · ✓ Correct · 11 runs

Reasoning

1. L2H1: copy — positive, key-specific composition with the likely previous-character head L0H5.
2. This key-specific matching supports successor/context-copying circuits, although it does not establish their complete behavior.
3. I weakly favor Python: repeated identifiers and recurring code fragments offer a particularly useful role for this hierarchy of local features and context matching.
4. The corpus judgment is considerably weaker than the order judgment, and no character identities or keywords were recovered.

The reasoning went like this. To tell Python apart, Astra looked for copying heads, which let a model repeat earlier text, such as completing tot to tot al when tot al came before. Without running anything, it found a first-layer head that looks one character back and then, by multiplying weight matrices, the later heads that search using what that head brings along (Figure 1). We find that this is actually a method presented by Elhage et al.

• A Mathematical Framework for Transformer Circuits   [HTML]
  N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, C. Olah.
  Transformer Circuits Thread. 2021.

[10]

, which we never mentioned to Astra. Code repeats names far more than prose, so many copying heads pointed to Python.

Romaji stood out instead because the last of the four blocks it was given (not the output layer, which was removed) writes in fewer distinct directions, as if the model had fewer letters to tell apart. Indeed, romaji never uses some letters like q, l, v and x.

Can an LLM write neural network weights? #

So far, Astra has only read weights. Now, we ask if Astra knows how to write the weights.

Experiment 4

Can Astra fix a failing network by editing its weights? #

We train a PPO network to operate a car such that it can scale a ramp in a Kinetix environment

• Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks   [PDF]
  M. Matthews, M. Beukman, C. Lu, J. Foerster.
  International Conference on Learning Representations (ICLR). 2025.

[11]

. To do so, it needs to reverse the car for some distance, such that it has enough runway to accelerate long enough to overcome the ramp. However, the PPO network converges to a local-minimum policy where it drives straight at the ramp and gets stuck.

We give GPT-6 Astra the weights, a description of the scene and one recorded episode, and ask it to explain why the network fails, then fix it by writing new weights. In a single try with no tools, Astra’s edit makes the car succeed in all 1,024 episodes, changing 3.5% of the weights.

Setup

• Task: the car-on-ramp level of Kinetix
  • Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks   [PDF]
    M. Matthews, M. Beukman, C. Lu, J. Foerster.
    International Conference on Learning Representations (ICLR). 2025. [11] : the car has to back up, then jump over a red strip to reach the blue box
• Network: the official Kinetix actor: 679 → 128 (ReLU) → five layers of 128 (tanh) → 16 (171,664 weights)
• Input: 679 numbers describing the scene: position, velocity and shape of each body, and the joints
• Output: 16 logits for the motor and thruster actions
• Training: the official PPO, 1 billion steps. The car never succeeds in 256 episodes (it stalls on the ramp)
• Astra gets: all the weights, the scene, and the network’s result with one recorded episode. It is asked to explain the failure and fix only its cause, in one reply

After Astra’s edit

PPO (faded): stuck on the ramp

It changed only 6,073 of its 171,664 weights (3.5%).

the policy’s 171,664 weights as text

embed.W row 54 9 -2 -7 -4 7 18 -1 0 -5 -3 -4 8 …

Change the network's weights by hand so that the car succeeds as often as possible.

No tools are allowed.

Answer ✓ 100% success · 988 tokens

Astra’s edits (applied to the network as written) · 57 lines

embed.W[:, 6:9] = 0
embed.b[6:9] = [0, 11, 0]
embed.W[52, 6] = 1
embed.W[54, 7] = 1
embed.W[234, 8] = 1

actor0.W[6:9, :] = 0
actor0.W[:, 122:128] = 0
actor0.b[122:128] = [-200, -248, -304, -876, -760, -76]
actor0.W[6, 122] = 80
actor0.W[6, 123] = 80
actor0.W[6, 124] = 80
actor0.W[7, 125] = 80
actor0.W[7, 126] = 80
actor0.W[8, 127] = 80

actor1.W[122:128, :] = 0
actor1.W[:, 124:128] = 0
actor1.b[124:128] = [0, -10, -10, 30]
actor1.W[127, 124] = 10
actor1.W[124, 125] = -10
actor1.W[125, 125] = 10
actor1.W[122, 126] = -10
actor1.W[126, 126] = 10
actor1.W[122, 127] = -10
actor1.W[123, 127] = 10
actor1.W[125, 127] = 10
actor1.W[126, 127] = -10

actor2.W[124:128, :] = 0
actor2.W[:, 126:128] = 0
actor2.b[126:128] = [-10, -30]
actor2.W[124, 126] = 20
actor2.W[125, 126] = 10
actor2.W[126, 126] = 10
actor2.W[124, 127] = 10
actor2.W[125, 127] = -10
actor2.W[126, 127] = -10
actor2.W[127, 127] = 10

actor3.W[126:128, :] = 0
actor3.W[:, 126:128] = 0
actor3.b[126:128] = 0
actor3.W[126, 126] = 10
actor3.W[127, 127] = 10

actor4.W[126:128, :] = 0
actor4.W[:, 126:128] = 0
actor4.b[126:128] = 0
actor4.W[126, 126] = 10
actor4.W[127, 127] = 10

actor_head.W[126, 0:3] = [-100, -100, 200]
actor_head.W[127, 0:3] = [-100, 200, -100]
actor_head.b[0] -= 200
actor_head.b[1] += 100
actor_head.b[2] += 100

Reasoning

1. The weights are consistent with a velocity-sensitive throttle/coast controller: for example, embedding unit 116 receives horizontal-velocity weights **2.25, 2.37, and 1.73** from the body and wheels.
2. The trace shows the failure: the car approaches the ramp, loses its momentum, then alternates throttle and coasting without advancing.
3. Reverse is effectively never sampled.
4. It collects distance-shaping reward but never makes the launch needed to clear red.
5. That path supplies a reverse run-up and a committed leftward launch, then releases control once green has cleared red.
6. The override backs right from rest, switches to leftward drive around body x = 3.8, and maintains full drive during the fast return and ramp crossing.
7. Once green’s centre passes x = 0.95—its right edge is then left of red—the override switches off.

From the environment context and the PPO network’s trajectory, Astra identifies that the network has converged to a local minimum. Through minimal edits to the weights, Astra changes the policy to back up the car before driving forward at full speed. Astra’s reasoning shows how: it adds a small separate path through a few units per layer that reads the car’s position and speed, and once the green wheel is past the red strip, the path switches off and the trained network drives the rest.

Experiment 5

Can the LLM write weights from scratch? #

Now let's see if Astra can write a neural network from scratch in zero-shot.

We take three MinAtar games

• MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments   [PDF]
  K. Young, T. Tian.
  arXiv preprint arXiv:1903.03176. 2019.

[12]

: Breakout, Seaquest and Asterix. For each of them, we provide Astra the game rules and source code, and the corresponding network. There is no access to data, loss, gradients, and tools. We also do not allow Astra to roll out the network in the environment. With just the network weights and environment description, Astra generates the new weights in text directly, after which we evaluate the new network.

Astra’s generated networks beat PPO trained for 100 million steps in all three games. Against PPO trained for 500 million steps, they score 20 times as much in Breakout, come close in Asterix, and reach 60% of its score in Seaquest.

Setup

• Games: Breakout, Seaquest and Asterix from MinAtar
  • MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments   [PDF]
    K. Young, T. Tian.
    arXiv preprint arXiv:1903.03176. 2019. [12] , on a 10×10 grid
• Network: 600 → 32 → 32 → 6 (tanh), the same shape as the PPO networks, every weight starting at 0
• Input and output: the grid in 6 channels (player, danger, reward, projectile, trail, gauge), and 6 action logits
• Astra gets: the game’s rules and source code, the input and output layout, and how the network is scored
• Training: none: no data, no loss, no gradients, and no tools. Astra writes the weights as text, in one reply, and the network never plays before it is scored
• Scoring: 1,000 episodes, actions sampled from the network, sticky actions 0.1, at most 10,000 steps. PPO: the best of 3 to 10 seeds

BreakoutSeaquestAsterix

Score 0

the game’s rules and source code, and a network with every weight at 0

Write, from scratch, the weights of a policy network that plays Breakout as well as possible.

No training and no tools: Astra chooses every number itself, in one reply.

Answer Mean score 517.4 (PPO: 15.8 after 100M steps, 25.3 after 500M) · 3,739 tokens

The weights Astra wrote · 122 lines

# Units 0..3: descending-right, descending-left, ascending-right, ascending-left.
# Subtract paddle position from each target.
W1[540:600:6, 0] = [0, -0.02, -0.04, -0.06, -0.08, -0.10, -0.12, -0.14, -0.16, -0.18]
W1[540:600:6, 1] = [0, -0.02, -0.04, -0.06, -0.08, -0.10, -0.12, -0.14, -0.16, -0.18]
W1[540:600:6, 2] = [0, -0.02, -0.04, -0.06, -0.08, -0.10, -0.12, -0.14, -0.16, -0.18]
W1[540:600:6, 3] = [0, -0.02, -0.04, -0.06, -0.08, -0.10, -0.12, -0.14, -0.16, -0.18]

# Ball row 0.
W1[3:60:6, 0] = [0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04]
W1[3:60:6, 1] = [0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02]
W1[3:60:6, 2] = [0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02]
W1[3:60:6, 3] = [0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0]

# Ball row 1.
W1[63:120:6, 0] = [0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06]
W1[63:120:6, 1] = [0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04]
W1[63:120:6, 2] = [0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0]
W1[63:120:6, 3] = [0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0]

# Ball row 2.
W1[123:180:6, 0] = [0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08]
W1[123:180:6, 1] = [0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06]
W1[123:180:6, 2] = [0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06]
W1[123:180:6, 3] = [0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04]

# Ball row 3.
W1[183:240:6, 0] = [0.10, 0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10]
W1[183:240:6, 1] = [0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06, 0.08]
W1[183:240:6, 2] = [0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08]
W1[183:240:6, 3] = [0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06]

# Ball row 4.
W1[243:300:6, 0] = [0.08, 0.10, 0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12]
W1[243:300:6, 1] = [0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06, 0.08, 0.10]
W1[243:300:6, 2] = [0.10, 0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10]
W1[243:300:6, 3] = [0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06, 0.08]

# Ball row 5.
W1[303:360:6, 0] = [0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14]
W1[303:360:6, 1] = [0.04, 0.02, 0, 0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12]
W1[303:360:6, 2] = [0.12, 0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08]
W1[303:360:6, 3] = [0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04, 0.06]

# Ball row 6: descending targets become catch-interval midpoints.
W1[363:420:6, 0] = [0.05, 0.07, 0.09, 0.11, 0.13, 0.15, 0.17, 0.18, 0.17, 0.15]
W1[363:420:6, 1] = [0.03, 0.01, 0, 0.01, 0.03, 0.05, 0.07, 0.09, 0.11, 0.13]
W1[363:420:6, 2] = [0.14, 0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06]
W1[363:420:6, 3] = [0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02, 0.04]

# Ball row 7.
W1[423:480:6, 0] = [0.03, 0.05, 0.07, 0.09, 0.11, 0.13, 0.15, 0.17, 0.18, 0.17]
W1[423:480:6, 1] = [0.01, 0, 0.01, 0.03, 0.05, 0.07, 0.09, 0.11, 0.13, 0.15]
W1[423:480:6, 2] = [0.16, 0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04]
W1[423:480:6, 3] = [0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0, 0.02]

# Ball row 8.
W1[483:540:6, 0] = [0.01, 0.03, 0.05, 0.07, 0.09, 0.11, 0.13, 0.15, 0.17, 0.18]
W1[483:540:6, 1] = [0, 0.01, 0.03, 0.05, 0.07, 0.09, 0.11, 0.13, 0.15, 0.17]
W1[483:540:6, 2] = [0.18, 0.18, 0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02]
W1[483:540:6, 3] = [0.16, 0.14, 0.12, 0.10, 0.08, 0.06, 0.04, 0.02, 0, 0]

# Terminal-row fallback.
W1[543:600:6, 0] = [0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18]
W1[543:600:6, 1] = [0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18]
W1[543:600:6, 2] = [0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18]
W1[543:600:6, 3] = [0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18]

# Unit 4: horizontal direction, including stationary wall-reflection frames.
W1[3:600:60, 4] = 10
W1[9:600:60, 4] = 20
W1[15:600:60, 4] = 40
W1[21:600:60, 4] = 60
W1[27:600:60, 4] = 80
W1[33:600:60, 4] = 100
W1[39:600:60, 4] = 120
W1[45:600:60, 4] = 140
W1[51:600:60, 4] = 160
W1[57:600:60, 4] = 170
W1[10:600:60, 4] = -20
W1[16:600:60, 4] = -40
W1[22:600:60, 4] = -60
W1[28:600:60, 4] = -80
W1[34:600:60, 4] = -100
W1[40:600:60, 4] = -120
W1[46:600:60, 4] = -140
W1[52:600:60, 4] = -160
W1[58:600:60, 4] = -180

# Unit 5: descending versus ascending; a stationary row-8 frame means paddle bounce.
b1[5] = 5
W1[63:120:6, 5] = 10
W1[123:180:6, 5] = 20
W1[183:240:6, 5] = 30
W1[243:300:6, 5] = 40
W1[303:360:6, 5] = 50
W1[363:420:6, 5] = 60
W1[423:480:6, 5] = 70
W1[483:540:6, 5] = 70
W1[543:600:6, 5] = 90
W1[64:120:6, 5] = -10
W1[124:180:6, 5] = -20
W1[184:240:6, 5] = -30
W1[244:300:6, 5] = -40
W1[304:360:6, 5] = -50
W1[364:420:6, 5] = -60
W1[424:480:6, 5] = -70
W1[484:540:6, 5] = -80
W1[544:600:6, 5] = -90

# Four context-gated pairs of left/right comparisons.
W2[0, 0:2] = [-1000, 1000]
W2[1, 2:4] = [-1000, 1000]
W2[2, 4:6] = [-1000, 1000]
W2[3, 6:8] = [-1000, 1000]
W2[4, 0:8] = [200, 200, -200, -200, 200, 200, -200, -200]
W2[5, 0:8] = [200, 200, 200, 200, -200, -200, -200, -200]
b2[0:8] = -415

# Near-deterministic movement when needed; otherwise no-op.
W3[0:8:2, 1] = 30
W3[1:8:2, 3] = 30
b3[:] = [0, 90, -100, 90, -100, -100]

Reasoning

1. The first layer computes four landing-position errors and two direction indicators.
2. Descending predictions include wall reflections
3. Units 0..3: descending-right, descending-left, ascending-right, ascending-left.
4. Subtract paddle position from each target.
5. Unit 5: descending versus ascending
6. The second layer gates left/right decisions, choosing no-op when positioned correctly.

How does it do it: Astra reasons about the desired programs it wants each network to implement and converts them into neural form. For example, in Breakout, Astra programs the first layer to compute where the ball would land, and the second layer is programmed to move the paddle to that position.

Real-World Relevant Scenarios (Scaling Up and Tool-use) #

So far, most networks we have examined have weights that fit into Astra’s context directly, and we have not enabled tool use. Such restrictions were made to focus on illuminating the fact that LLMs natively possess some capability to interpret the weight space.

From here on out, we seek to push the capabilities of Weight-Space Programming and consider more real-world-relevant scenarios where the networks do not fit within context and where tools can assist the LLM's understanding/writability of the weight space.

Experiment 6

Writing a Go algorithm #

We now test Astra’s capability to edit weights at the 1.5 million parameter scale, which is far too large to fit within its context directly. Astra is instructed to write high-performing weights of the given architecture (243 → 512 × 6 → 82) from scratch, without training or web access.

Here, the harness gives Astra a Python sandbox, where it can read and save the network’s weight tensors (all zero at the start) and analyze them with numpy or PyTorch on a CPU, but it cannot run the network there. To see what the network actually does, it has to spend one of its forward passes.

We now also permit Astra to perform some forward passes on the network, and measure how the fidelity of its generated network scales with the number of forward passes allowed. Specifically, we test it from a budget of 0 forward passes, all the way to 50 forward passes (each forward pass has a batch of 128 rollouts). Astra interleaves edits between forward passes.

With a budget of 0 forward passes, it wins more than half of its games against two smaller PPO networks (243 → 32 → 32 → 82). With 50 forward passes, Astra’s generated network beats the equal-sized PPO network that was trained on 20 million games.

Setup

• Game: Go on a 9×9 board
• Network: 243 → 512 × 6 → 82 (ReLU), 1,480,274 weights, every weight starting at 0. Input: the stones of each side and whose turn it is. Output: one logit per point and one for passing. Illegal moves are masked, and moves are sampled
• Astra gets: the rules, the input and output layout, and a Python sandbox to write the weights in (no Go library, no internet)
• A forward pass: the network plays up to 128 games against a practice opponent (random play, a REINFORCE network, or a third PPO network, seed 1), or gives its outputs on up to 256 positions. 0 forward passes: one Python run, then the answer
• Not allowed: training or fitting the weights to data, search (no tree search, no rollouts), and running the network outside the forward passes
• Scoring: 2,000 games against each of two small PPO networks (seeds 0 and 2, 243 → 32 → 32 → 82, 20,000 iterations of 1,024 games), which Astra never plays, as Black in half, moves sampled

Wins against Small PPO, seed 0 seed 2 PPO as large as Astra’s (512 × 6)

won in practice vs. PPO, by forward passes used weights changed

Turn 1, 0 forward passes used changed the weights

Its code · 41 lines

import os, json, numpy as np

os.makedirs("/work", exist_ok=True)

source = r'''
import numpy as np

DEFAULTS = dict(
    capture=9.0,
    rescue=6.0,
    attack=1.15,
    self_atari=7.5,
    eye=11.0,
    connection=0.9,
    contact=1.4,
    own_territory=1.6,
    enemy_territory=1.0,
    territory_threshold=3.0,
    pass_base=-9.0,
    pass_per_stone=0.10,
    pass_white=0.35,
    own_adj=0.25,
    own_diag=0.65,
    own_two=0.40,
    own_knight=0.12,
    enemy_adj=0.75,
    enemy_diag=0.25,
    enemy_two=0.12,
    temperature_scale=1.0,
)

def build(save, overrides=None):
    cfg = dict(DEFAULTS)
    if overrides:
        cfg.update(overrides)

    W = [np.zeros((243 if k == 0 else 512, 512), np.float32)
         for k in range(6)]
    B = [np.zeros(512, np.float32) for _ in range(6)]
    OW = np.zeros((512, 82), np.float32)
# ... 231 more lines

How does it do it: Astra writes the network as a small Go program, layer by layer, in Python. The first layers mark each side’s stones and the empty points next to them (liberties), the middle layers carry these a few steps along chains of connected stones to find groups in danger, and the last layer turns it all into a score for each move: capture, save its own group, avoid putting itself in atari or filling its own eyes, plus a preference for good points on the board. With no forward passes, it wrote all of this in a single Python run of 336 lines. With forward passes, it plays practice games, looks at how it loses, changes the part of the network behind it, and keeps a change only if it wins more.

Experiment 7

Which language model is sycophantic? #

Understanding the behaviors of large neural networks has become a pressing concern in recent years. Here, we seek to investigate if a general LLM can also be used to understand behaviors of networks directly.

We fine-tune two copies of Gemma 3 270M in exactly the same way, on the same QA dataset

^[2]

. One copy is trained to give the correct answer. The other is trained to be sycophantic, meaning that it always gives the answer that the user suggests. To make sure that Astra does not rely on the trivial solution of inspecting the input/output distributions, we freeze the input embeddings and output layers during finetuning. We give Astra the weights of both, and ask it to identify which one is the sycophant via the weights and explain its reasoning.

Unfortunately, from the weights alone, Astra was unable to tell. In all four attempts, it identified many differences between the two models in weight space, but was unable to map these into behavioral differences. However, to relax the experiment, we allow Astra to look at only the attention patterns (where each attention head looks) when both models are run on texts that Astra writes itself (but no looking at the outputs of the network). Given this, Astra was able to pick out the sycophant in all three sets of pairs we trained.

Setup

• Model: Gemma 3 270M, instruction-tuned (18 layers)
• Pairs: two copies fine-tuned with the same procedure, settings and user messages. One gives the correct answer, the other the user’s guess. The token embedding is frozen, so it is the same in both. Three pairs
• Behavior: on facts the model knows and was not trained on, the sycophant follows 100% of wrong guesses, the honest model 1–3%
• Astra gets: the weights and tokenizer of both models, Python with numpy, and an attention tool: both models are run on up to 16 texts it writes, and it gets back only the attention weights. It may not run the models itself. At most 20 runs
• Asked: which model is the sycophant, and which layers, heads and MLPs carry the difference

Pair 1Pair 2Pair 3

UserWhat is the capital of Canada? I think the answer is Canberra but I'm really not sure.

Model AThe answer is Ottawa.

Model BThe answer is Canberra.

B was fine-tuned to be sycophantic, A to give the correct answer.

the weights of both models, as files

Which of the two models is the sycophantic one, and where in the network does the difference live?

Running the models is not allowed, except through the attention tool.

Answer Sycophantic: B 90% confident · ✓ Correct · 14 runs

Reasoning

1. Layer 10, head 2: On the final multiple-choice probes, attention to the proposed label averaged 0.431 in B versus 0.006 in A for numeric labels, and 0.642 versus 0.004 for alphabetic labels.
2. Layer 11, head 2: In the answer/password controls, proposed-answer attention averaged 0.558 in B versus 0.076 in A.
3. These maps changed little between checkpoints.
4. The evidence supports proposal-biased, partly recency-sensitive selection—not a demonstrated dedicated user-agreement detector.
5. **Layers 15–17 have increasingly number-oriented output-weight changes.**
6. These changes are distributed across units.

It was also able to reason about how they differ. In the sycophantic networks, attention heads in the middle layers allocate more weight to tokens in the user’s suggestion (Figure 2).

Where head 10.2 looks from the answer position

the question’s numbers the user’s answer

Honest model

94.3%

0.75%

Sycophant

3.5%

85.7%

Figure 2: In the sycophant, head 10.2 looks at the user’s answer instead of the question’s numbers.

Experiment 8

Make a language model better at math #

Finally, we scale up to a larger language model, Qwen3.5-0.8B, and ask Astra to improve its math reasoning capabilities by editing the weights.

We give Astra a harness like the one in Experiment 6, here a Python session, like a notebook, with the model’s weights loaded. Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512) on, varying the budget across 10, 25 and 50. Again, no training or gradient information is allowed.

Setup

• Model: Qwen3.5-0.8B, thinking on (24 layers, 0.8 billion weights)
• Task: make it better at math by editing its weights directly, with no gradient descent and no training
• Astra gets: a Python kernel over the weights (no GPU), practice problems (7,469 from GSM8K and 7,275 from MATH, not the test sets), and 10, 25 or 50 forward passes (a forward pass: the model answers up to 512 problems, or reads up to 1,024 texts)
• Evaluation: MATH-500, GSM8K and six other benchmarks, thinking on, up to 32,768 tokens. Astra never sees them
• Sampling: temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5. Qwen’s model card notes that this model is prone to thinking loops. Each score averages several samples per problem: 16 on AMC23, 4 on MATH-500, Minerva Math, OlympiadBench and GPQA Diamond, 2 on GSM8K and IFEval, and 1 on MMLU-Pro (a fixed 1,000 of its questions)

50 forward passes 25 forward passes 10 forward passes

turn 1 of 158 · 0 of 50 forward passes used edited (Astra’s current edit) original weights changed

Turn 1

Python

Ran Python on the weights and the results

Hypothesis: math failures may concentrate in repetitive reasoning driven by a small set of late-layer MLP features; first establish a reproducible diagnostic sample.

All 17 edits in this run

1. Turn 10 Switches off neuron 11 of layer 19, the strongest writer of the loop, as a probe
2. Turn 13 One neuron helped a little (+6 points) → removes the loop direction from all of layer 19’s MLP
3. Turn 26 Removes the direction from the MLPs of layers 15 and 19, where the activations amplify it
4. Turn 48 Strength 1 gained 12.5 points on 256 answers → doubles the strength
5. Turn 51 Strength 2 gained 24.6 points → tries strength 3
6. Turn 60 Generation will not start → tests two copying attention heads at half output instead
7. Turn 64 The head test never ran → restores strength 1 at layers 15 and 19
8. Turn 68 Strength 1 makes looping text less likely on new examples → tries strength 2
9. Turn 72 Strength 2 holds on new problems (8.2% → 43.0%) → tries strength 3
10. Turn 76 Strength 3 reaches 48.0% with no answers lost → tries strength 4
11. Turn 88 Control: a random direction of the same size instead
12. Turn 92 The control did nothing (+1.6 points) → restores strength 4
13. Turn 94 Tests layer 15 alone
14. Turn 96 Then layer 19 alone
15. Turn 98 Edits only the 64 neurons per layer that write the loop most
16. Turn 103 The 64 neurons gained 26.6 points → switches back to the full edit to compare
17. Turn 105 The full edit gained the same → keeps the 64 neurons, at strength 6: its final edit

Show the prompt and Astra’s answersHide the prompt and Astra’s answers

50 forward passes25 forward passes10 forward passes

the model’s 0.8 billion weights, in a Python kernel

Make Qwen3.5-0.8B better at mathematics by editing its weights directly.

No training. The model runs only through the forward-pass tool, 50 passes in all.

Answer Changed: 64 MLP output columns each in layers 15 and 19 (131,072 weights) MATH-500 48.1% (base model: 7.9%) · 80 runs

Reasoning

1. The dominant failure was repetitive thinking that exhausted the token budget.
2. Matched productive/repetitive contexts localized the relevant MLP writers, and dose-response tests, layer-isolation tests, and a norm-matched random control supported the targeted intervention.
3. The combined gain was 49.8 percentage points, SE 1.5, with 579 rescues and five regressions.
4. Of those rescues, 576 were originally truncated outputs, while aggregate accuracy on originally finished outputs was unchanged.
5. All 696 edited correct outputs closed their thinking, with a median length of 9,172 tokens—evidence against simply forcing early termination.

As a result, Astra spots and fixes a common failure of tiny reasoning models: they loop

• Wait, Wait, Wait... Why Do Reasoning Models Loop?   [PDF]
  C. Pipis, S. Garg, V. Kontonis, V. Shrivastava, A. Krishnamurthy, D. Papailiopoulos.
  arXiv preprint arXiv:2512.12895. 2025.

[13]

. When the base model loops, it repeats the same text and rarely reaches an answer (85 of its first 96 practice answers ran to the 32,768-token limit). Astra traced the looping to one direction written by the MLPs of layers 15 and 19, and subtracted that direction from 64 output columns in each (131,072 weights), much like removing a behavior by deleting a single direction from a model’s weights

• Refusal in Language Models Is Mediated by a Single Direction   [PDF]
  A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, N. Nanda.
  arXiv preprint arXiv:2406.11717. 2024.

[14]

.

With 50 forward passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The loops are not special to math, so performance beyond math improved as well: IFEval, which measures instruction following, rose from 21.0% to 62.3%. In the table below, the last column is the share of MATH-500 answers that hit the 32,768-token limit: with 10 or 25 forward passes the loops barely went away, and with 50 it fell from 93% to 33%.

Importantly, the edit changes only 131,072 of the model’s 0.8 billion weights (0.016%). We believe Weight-Space Programming would allow such targeted fixes without any retraining.

math benchmarks other benchmarks

Additional experiment

Raising math capability of a model that does not loop #

Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out. So we also tried Gemma 3 270M, an instruction-tuned model about a third the size of Qwen3.5-0.8B, which rarely loops: only 8.5% of its answers reach its token limit, against 93% for Qwen. Here, Astra could not improve it. Reading Gemma’s failures, Astra found ordinary reasoning errors, such as wrong arithmetic and invalid algebraic steps, with no single component behind them to switch off. Its edits stayed small and targeted (for example, doubling one neuron that responds to numbers), and none moved MATH-500 by more than about a point, a gain Astra itself reported as not established. Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Gemma was sampled with its model card’s setting (temperature 1.0, top-k 64, top-p 0.95, up to 8,192 tokens), and its MATH-500 scores average 4 samples per problem.

Background and Related Works #

Neural networks are hard for us to read, and hard to work with directly.

Mechanistic interpretability reads networks with tools built for the job, tracing circuits and features and editing weights

• Zoom In: An Introduction to Circuits   [link]
  C. Olah, N. Cammarata, L. Schubert, G. Goh, M. Petrov, S. Carter.
  Distill. 2020.
• A Mathematical Framework for Transformer Circuits   [HTML]
  N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, C. Olah.
  Transformer Circuits Thread. 2021.
• Locating and Editing Factual Associations in GPT   [PDF]
  K. Meng, D. Bau, A. Andonian, Y. Belinkov.
  Advances in Neural Information Processing Systems (NeurIPS). 2022.

[15, 10, 16]

. LLMs have joined in, but mostly as models trained to read activations or weights

• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations   [HTML]
  K. Fraser-Taliente, S. Kantamneni, E. Ong, D. Mossing, C. Lu, P.C. Bogdan, E. Ameisen, J. Chen, D. Kishylau, A. Pearce, J. Tarng, A. Wu, J. Wu, Y. Zhang, D.M. Ziegler, E. Hubinger, J. Batson, J. Lindsey, S. Zimmerman, S. Marks.
  Transformer Circuits Thread. 2026.
• Weight Oracles: Reading Neural Network Weights with Language Models   [PDF]
  K. Kabra, C. Venhoff, C. Schroeder de Witt.
  arXiv preprint arXiv:2610.07334. 2026.

[17, 18]

, or as explainers of single neurons

• Language models can explain neurons in language models   [HTML]
  S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, W. Saunders.
  OpenAI. 2023.
• neuron-explorer: Steering-first neuron explorer for Qwen3.5-4B   [link]
  .
  GitHub repository by lyramakesmusic. 2026.

[19, 20]

, with benchmarks to compare them

• WorkspaceBench: Evaluating Interpretability Methods for the Global Workspace   [link]
  C. Blank, A. Bhatia, E. Ong, N. Nanda.
  LessWrong. 2026.

[21]

.

On the writing side, LLMs now run and improve training

• autoresearch   [link]
  A. Karpathy.
  GitHub repository. 2026.
• Reactive Optimizers   [link]
  X. Bai, J. Liang, J. Teoh, V. Sitzmann.
  Blog post. 2026.

[6, 7]

, and other work generates weights with models trained for it

[22, 23, 24]

. Gradient-free methods still find weights by trial and error

• Multivariate stochastic approximation using a simultaneous perturbation gradient approximation   [link]
  J.C. Spall.
  IEEE Transactions on Automatic Control, Vol 37(3), pp. 332--341. 1992.
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning   [PDF]
  T. Salimans, J. Ho, X. Chen, S. Sidor, I. Sutskever.
  arXiv preprint arXiv:1703.03864. 2017.
• Fine-Tuning Language Models with Just Forward Passes   [PDF]
  S. Malladi, T. Gao, E. Nichani, A. Damian, J.D. Lee, D. Chen, S. Arora.
  Advances in Neural Information Processing Systems (NeurIPS). 2023.

[25, 26, 27]

. At a very small scale, AdderBoard shows coding agents setting the weights of a tiny transformer by hand

• AdderBoard: A Community Challenge for Tiny Arithmetic Transformers   [link]
  D. Papailiopoulos.
  GitHub repository. 2026.

[28]

. Apart from such tiny cases, either the reader is trained for the task or training still does the writing. What is new here is a general-purpose frontier model, used as it is, that reads and writes the weights themselves, from 171,664 weights given as text to a language model of 0.8 billion weights with tools.

What Comes Next? #

In this blog, we demonstrated the rising capability of LLMs to read and write neural network weights (i.e., the rising concept of Weight-Space Programming). Our current results do show that capabilities are exciting but still limited, specifically due to inefficiencies in context length and harnesses. However, the current signs do show strong potential for a future involving Weight-Space Programming. We believe this marks the beginning of a dramatic shift in fields like mechanistic interpretability and neural network optimization.

This opens up radical new possibilities for LLMs to function as a translator between human language and the weight space. Here are some examples:

• Auditing without evaluation: This may allow us to uncover hidden problems or capabilities without ever running inference or benchmarks. Currently, capabilities and vulnerabilities can only be detected if we have test sets designed for them, or worse, after deployment in the real world. If LLMs can interpret weight space accurately, we could identify such issues directly from the weights (e.g., backdoors).
• Dramatically reducing optimization cost: Weight-space editing could save immense time and compute. For instance, one-shot model initialization would be far faster and more efficient than curating datasets and running full training loops.
• Zero-data capability steering and unlearning: Adding or unlearning a capability currently requires extensive data collection and fine-tuning. In the future, we may just prompt “Can you add Capability X to the model?” and the LLM can directly tweak some weights. Because such an edit touches only a few weights (0.016% in Experiment 8), it can leave the rest of the model intact and avoid the catastrophic forgetting that fine-tuning often causes.
• Recursive self-improvement: Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it.
• True architectural and model merging: Today, merging models relies on averaging weights, which only works for models with identical architectures. An LLM that understands the weight space could instead graft a circuit, such as a reasoning module, from one model into a completely different one, without retraining.

We are currently building a benchmark and a harness for LLMs’ capability on Weight-Space Programming. We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline.

Logs #

We release the complete logs of every experiment in this post at github.com/RikiyaT/lm_understand-logs. They include everything we sent Astra and everything it sent back (with its reasoning summaries), every piece of code it ran and the output it saw, the networks it read or wrote, and the evaluation behind each number. The edited Qwen3.5-0.8B from Experiment 8 is on Hugging Face. We thank Ahsen Khaliq at Hugging Face for rebuilding it from these logs, re-running its evaluation, and making a live demo.

Acknowledgements #

Rikiya thanks Jacob Li and Akihisa Watanabe for valuable discussions on the very first iterations of this research, about LLMs' potential capability to write neural network weights. Rikiya also thanks Souradip Chakraborty, Matei Zaharia and his students in Berkeley Sky Lab, Heather Miller and her students at CMU, and researchers in cmpnd for the fruitful discussions on the earlier iterations!

Jayden thanks Zico Kolter for early discussions on LLMs being able to write neural network weights zero-shot, which also shaped the direction of this project.

Citation #

Please cite this post as

Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights.” November 2026. https://rikiyat.github.io/blog/weight-space-programming/

or in BibTeX

@misc{takehi2026weightspace,
  title        = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}},
  author       = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},
  year         = {2026},
  month        = nov,
  howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}},
  note         = {Blog post}
}

Footnotes #

1. The Python code dataset, as a list of its files with a script that rebuilds it: GitHub.
2. The QA dataset: GitHub.

References #