Interpretability tools built for language models rely on seven properties of text transformers. Vision-language-action (VLA) models break between one and six of them, depending on how they are built. When a property breaks, the tools usually still run and still give clean-looking results, but the results no longer measure what they measured on a language model.
The paper names the seven properties as cautions A1–A7, sorts VLAs into seven architecture types (Type I–Type VII), and maps which type breaks which caution. The result shows where the LLM toolkit works unchanged, where it needs a small fix, and where it has to be rebuilt.
Example 1
Run Direct Logit Attribution (DLA) on a VLA that brings in vision through cross-attention. You get a heatmap that looks plausible but leaves out the cross-attention contribution entirely. F1
Example 2
Run SVD on the residual stream of a continuous-action VLA. It points to heads with no causal control over the action and misses the heads that actually drive it. F3
Neither tool raises an error. The paper is written for someone about to run a mechanistic interpretability experiment on a VLA.
Reading guide
How the paper fits together
The paper is a chain of four ideas, and later sections keep pointing back to earlier ones. On this page, every label such as A1, Type IV or F2 is clickable. Hover or tap it to see what it means. Click through to jump there, then use the Back button at the bottom of the screen to return.
A transformer keeps one running vector per token, called the residual stream. Every attention head and every MLP reads from it and adds its output back into it. So the final state is the starting embedding plus a plain sum of everyone's contributions:
To read a decision off this state, a language model multiplies it by one fixed matrix, the unembedding \(W_U\), which gives a score to every token in the vocabulary. This step is linear, so you can push each head's contribution through \(W_U\) on its own and see how much it moved the answer toward one token \(v^+\) and away from another \(v^-\). This is Direct Logit Attribution (DLA):
Equation (1) is the foundation and equation (2) is the readout. Activation patching, attribution patching, path patching, sparse autoencoders (SAEs) and linear probes all use one or both.
So their guarantees depend on three things: one residual sum, one linear readout, and a stack of layers that all do the same kind of work.
Section 4 of the paper
Seven cautions: what the toolkit silently assumes
Each caution is a property that text transformers happen to have and the toolkit quietly relies on. None of them is a general law of neural networks. The Broken by row under each card comes from the paper's Table 1.
A1
Single residual stream
Every head and MLP adds its output into one stream, so the stream is a sum of individual contributions (eq. 1).
Tools that rely on it
DLA, attribution patching and path patching. They explain behaviour by reading off, or intervening on, single terms of that sum.
How VLAs break it
Two designs write into the stream from outside the sum. Cross-attention fusion (Type V, RoboFlamingo) adds visual information through extra sublayers that read straight from the vision encoder. These are not among the heads and MLPs in eq. 1, so standard implementations drop them. Separate-weight action experts (Type IV, π0) process action tokens with their own weights while reading the backbone's keys and values. The action computation is split across two sets of parameters, and no single stream holds all of it.
Result
The sum is incomplete, so any tool built on it silently misses part of the computation. See F1.
A2
Shared unembedding for the output
One matrix \(W_U\) turns the final hidden state into the output. DLA, SAE feature interpretation and the logit lens all assume it.
How VLAs break it
Discrete-token VLAs (Type I: RT-2, OpenVLA) keep \(W_U\): they split each action dimension into 256 bins and add those bins to the text vocabulary. Continuous action heads (Types II–IV: π0, GR00T N1, Octo) replace it with a regression MLP, a denoising network, or a flow field integrated by an ODE solver. None of these is a single matrix multiply.
Every layer does the same operation on the same stream, so a tool that works at one layer works at the others.
How VLAs break it
A VLA is three coupled networks: a vision encoder, a language/planning backbone and an action head. Each stores information in a different geometry, and the action head may not be a transformer layer at all (an MLP, a denoising U-Net or a flow-matching expert). Logit-lens curves or per-layer SAEs run across the whole VLA mix three different kinds of computation.
Do instead: apply tools to one module at a time.
A4
A checkable right answer for each decision
For a language model, correctness is a single target token. An ablation either flips the top token or it doesn't.
How VLAs break it
A robot task has many valid trajectories, so there is no single right answer. On top of that, discrete-token VLAs (Type I) predict action dimensions one after another, so the circuit for joint \(k\) also looks at joints \(1,\dots,k-1\) that were already produced. DLA on joint \(k\) then mixes the vision-to-action circuit with the circuit that coordinates the joints.
Do instead: run DLA on the first action token only, which has no earlier action context. Judge ablations in action space (KL divergence, rollout success), not by token rank. See F4.
A5
Architectural regularity
The layers form a uniform stack, so one tool can walk it from end to end.
How VLAs break it
Mixture-of-experts and heterogeneous-expert designs send different token types to different FFN experts. Flow-matching designs give action tokens their own expert weights inside shared layers. "Head 17 of layer 12" then names different computations depending on the routing, and per-layer attribution averages together computations that can't be meaningfully combined.
A6
Low superposition where it matters
The features you care about are roughly linearly separable, so SAEs and linear probes can recover them.
How VLAs break it
Actions are continuous physical quantities: positions, joint angles, gripper opening. Their natural shape is a smooth, low-dimensional surface, not a set of categories. Splitting it into 256 bins packs that smooth structure into a 256-token vocabulary. Neighbouring embedding rows become correlated, and the residual representation becomes densely superposed.
Result
Every VLA type in the table fails this one. See F3 and F5.
A7
One pass per decision
One forward pass produces one next token, so causal attribution only has to look at one pass.
How VLAs break it
Diffusion VLAs decide over \(T\) denoising passes. Flow-matching VLAs decide over an ODE integration. Action-chunking VLAs output \(K\) future motor commands per pass. A one-pass ablation then measures a compounded multi-step effect, not the contribution of a single step.
Do instead: run the analysis at each denoising step or chunk position, and report a circuit per step. The LLM toolkit doesn't do this out of the box.
Section 5 of the paper
Seven VLA types
A VLA maps a camera image \(o_t\) and a language instruction \(\ell\) to an action \(a_t\). It has three parts: a vision encoder, a language/action backbone and an action head. What you can interpret depends almost entirely on two choices: the type of action head, and how vision is fused in. The types are not mutually exclusive; π0 is both Type IV and Type VI.
Pick a type. The diagram shows where information flows, and the list shows how the type scores on each caution.
Table 1 of the paper
Who breaks what
Rows are VLA types, columns are cautions. Click any cell to see the reason, or a header to jump to it.
✓ holds∼ partly holds✗ broken
The toolkit works best on the discrete-token design and gets worse as the action head becomes continuous and multi-pass: Type I breaks one caution, Type III four and Type IV five. The field is moving toward that harder end, since flow-matching and diffusion action heads now lead generalist robot control.
Check the model you are studying
Built from Table 1 and the fixes in the paper. A model that belongs to two types gets the worse status of the two for each caution.
Section 6 of the paper
Five failure modes
This is what goes wrong in practice when a caution breaks. In every case the tool runs without error, and each one has a fix.
If contributions from outside the residual stream carry a fraction \(\alpha\) of the action signal, every DLA score shrinks by \((1-\alpha)\). For cross-attention (Type V) the hidden term is visible in the full sum:
\[ h_t^{(L)} = h_t^{(0)} + \sum_{l,h}\mathrm{head}_{l,h} + \underbrace{\sum_{l}\mathrm{XA}_l\big(h^{(l)}, V_{\text{vis}}\big)}_{\text{invisible to standard DLA}} + \dots \](3)
Rankings between heads inside the stream survive, since every head shrinks by the same factor. Absolute claims and ablation predictions do not. "Head h accounts for 30% of the output" silently means 30% of \((1-\alpha)\). Ablations also overstate the real drop, because the model routes around them through the pathway you are not measuring. Standard tools without a correction end up studying a self-consistent shadow of the model with the visual pathway removed.
When the action has no \(W_U\), you can still plug in the text \(W_U\) and DLA will run. But the scores then measure contributions to a distribution over text tokens that the model never uses for control. Ablating the top heads found this way leaves rollout performance essentially unchanged.
Two readouts from the same residual stream
residual stream→text \(W_U\)→text-token scoresnot used for control
residual stream→action head→robot actionwhat actually moves the robot
Fix: replace \(W_U\) with a supervised probe \(w_{\text{probe}}\) trained to predict the action: \(\mathrm{ProbeDLA}_{l,h} = \mathrm{head}_{l,h}\cdot w_{\text{probe}}\). You lose the convenience of training-free DLA, but the numbers now correspond to action-space behaviour.
F3
SVD finds variance, not the variable you care about
SVD returns the directions along which activations vary the most and presents them as the important axes. On robot data that goal doesn't match the question, because the action parts differ in scale: end-effector translation varies over roughly 100× the range of the gripper opening. The top singular directions follow translation, and the gripper axis falls into tail components carrying under 1% of the variance.
Illustration: share of variance by action component
translation x
high
translation y
high
translation z
high
gripper
< 1%
Bar lengths are for illustration. Only the gripper figure (under 1%) is from the paper.
A head that SVD ranks first can be nearly orthogonal to the variable it seems to track, and have almost no causal effect on it.
Fix: stop ranking by variance. Supervised linear regression picks directions by how well they predict the action, not by how much variance happens to lie along them.
Standard mechanistic interpretability judges a circuit by whether ablating it flips the predicted token. On binned actions this is the wrong level of detail.
Try it: what the token test sees
So a circuit that is complete in action space can fail the ablation test, and a circuit that passes it can still produce 5-bin shifts that compound into task failure.
Fix: evaluate ablations in action space (KL divergence, rollout success), not by whether the token changed.
A VLA's input mixes about 256 visual positions, about 25 language positions and 1 action position, each with very different activation statistics.
One sequence, three kinds of position (to scale)
~256 visual
~25
1
The action position is so small it is drawn slightly wider than its true share.
Sequence-mean ablation puts one average value into every position. That value is out of distribution for most of them, which biases every head's importance by the same offset.
Noise ablation is in distribution at high-variance positions and far out of distribution at low-variance ones, so head rankings can flip depending on the noise scale you choose.
Fix: position-preserving mean ablation. Replace each activation with its dataset mean at that same position, \(\mathbb{E}_{\text{dataset}}[x_{l,h,p}]\), computed separately for each position \(p\). The model then only sees values it has seen at that position, which removes both biases.
Section 4 of the paper, continued
The closed loop: why errors compound
Next-token prediction in an LLM and behaviour cloning in a VLA are the same supervised problem. Both minimise a per-step loss over expert examples:
Both are also autoregressive: the output at step \(t\) becomes part of the input at step \(t+1\). A small error moves the model into states it was not trained on, and errors pile up over the rollout. The difference is where the loop closes.
LLM
A mistake stays in token space. It has no physical consequence.
VLA
A mistake moves the arm, and the next image shows the result. A VLA runs 100–200 physical steps and has no expert data to recover from once it leaves the demonstrations.
How fast the error grows
Ross and Bagnell (2010) bound this for behaviour cloning. With per-step error \(\varepsilon\) under the expert's state distribution and horizon \(H\), \(J(\pi) - J(\pi^*) \le \varepsilon H^2\), and the \(H^2\) is tight. The bound has the same order for LLMs and VLAs. What differs is what an error costs.
Action chunking commits to actions only at chunk boundaries, which leaves roughly \(K\) times fewer points where error can compound (Zhao et al., 2023).
Continuous actions change the picture. The \(H^2\) rate is for discrete actions. With a squared loss, an error is a magnitude that flows through the physical dynamics, so its growth depends on how stable the closed loop is. Simchowitz et al. (2025) show that the demonstrations don't reveal whether the dynamics are stable. Without a mechanism that enforces stability, the error can grow exponentially in the horizon rather than polynomially.
Illustrative shapes only, not measured data: \(\varepsilon H^2\) for discrete actions, the same divided by \(K=8\) for chunking, and an exponential for unstable continuous dynamics. The shaded band is a typical VLA episode length (100–200 steps).
What this means for interpretation
Compounding only matters when you use an interpretation to act on the policy. A steering vector applied at every step, or a head ablated for a whole rollout, changes the policy by some \(\varepsilon_{\text{interp}}\) per step, and that change compounds through the same dynamics. For discrete actions it is bounded by \(\mathcal{O}(\varepsilon_{\text{interp}} H^2)\), and a \(K=8\) chunk cuts that by about an order of magnitude. For continuous actions it grows at whatever rate the dynamics set.
An attribution that is wrong but never used to intervene does not compound, because there is no rollout for it to act through.
A successful rollout doesn't prove a circuit is right
Other heads can compensate for what you removed.
A failed rollout doesn't prove it's wrong
A roughly correct intervention can be amplified by the dynamics until the task fails.
So rollout evaluation measures behaviour, not whether an attribution is correct.
Practitioner pitfalls
Two traps at instrumentation time
Even on a VLA that passes all seven cautions, two problems come up when you wire in your tools. Both examples are from OpenVLA (Type I).
P1
Softmax over the wrong vocabulary
At inference, actions are sampled only from a small range of tokens (IDs 31744–31999 on OpenVLA). Logit-space tools that take a softmax over the full vocabulary of 32,064 tokens spread the probability over about 31.8k irrelevant text tokens. They then report \(P(\text{correct action}) \approx 0\) at every layer, which is a false null.
# sketch: logits over the full vocabularyp = logits.softmax(-1) # normalises over 32,064 tokensp = logits[..., 31744:32000].softmax(-1) # only the 256 action tokens
Fix: restrict the softmax denominator to the action tokens.
P2
Visual tokens you can't see in the input IDs
OpenVLA's Prismatic projector inserts 256 visual patch features at the embedding level, before tokenisation. Tools that read the sequence length from input_ids.shape[1] see only the 25 text tokens and report 0% attention to vision.
256 visual positions (not in input_ids)
25 text
Fix: read the true sequence length from hidden_states[0].shape[1], which is 281.
Discussion
Takeaway
The paper does two things.
It sets out cautions for moving LLM interpretability tools to VLAs. The conveniences of that toolkit hold for one architectural choice: discrete, autoregressive tokens over one residual stream. Most VLA architectures quietly break it.
It argues that VLA interpretability needs tools built for the setting, not adapted from LLMs, and it supplies what those tools will need to know: which assumptions break, where, and how to detect it.
The architectures that are hardest to analyse are the ones being deployed.
Related work
Häon et al. (2025) are closest in spirit. They project FFN activations of OpenVLA and π0 onto the token embedding space, find sparse "semantic" directions such as "slow", and use them as steering vectors at inference. They report that under 25% of FFN neurons are "rewired" for action prediction.
Grant et al. (2026) ran the first cross-VLA interpretability study: six models and about 394k rollouts, using SAEs, linear probes and activation injection. They find that vision contributes more than language, that language matters more on some tasks than others, and that the network splits into separate motor and goal pathways. They also note that activations must be read one token at a time, because averaging across the sequence destroys the signal. That is what A3 and A6 predict, and the paper treats it as outside confirmation.
Citation
BibTeX
@inproceedings{goyal2026hitchhiker,
title = {The Hitchhiker's Guide to Mechanistic Interpretability
of Vision-Language-Action Models},
author = {Goyal, Aryan},
booktitle = {Mechanistic Interpretability Workshop at ICML},
year = {2026},
url = {https://openreview.net/forum?id=mS9nh3L3iN}
}