Endogenous Resistance to Activation Steering in Language Models

Accepted to ICML 2026

When you steer a language model by adding vectors to its activations (activation steering), the model sometimes resists. Llama-3.3-70B notices when it has gone off-topic, pauses mid-generation ("wait, that's not right"), and returns to the original question, even while steering remains active. We call this Endogenous Steering Resistance (ESR). We identified 26 SAE latents that fire during off-topic content and are causally linked to this behavior: ablating them reduces self-correction by 25%.

Paper (arXiv) → Read the full paper Code → Reproduce experiments on GitHub

Key Ideas

• One model catches itself going off-topic. When we steer models with unrelated SAE latents during generation, Llama-3.3-70B shows an ESR success rate of 3.8%. It interrupts itself ("wait, that's not right") and recovers to answer the original question. Smaller models from the Llama-3 and Gemma-2 families show rates below 1%. Without steering, none of the models self-correct (0% across 7,892 trials).
• Prompting increases multi-attempt rate 4.3×. Adding "if you notice yourself going off-topic, stop and get back on track" increases multi-attempt rate from 7.4% to 31.7% for Llama-3.3-70B. Fine-tuning also increases multi-attempt rate, but not correction success. The model learns when to try again without getting better at succeeding.
• We found circuits that drive the behavior. Using SAE latent analysis, we identified 26 latents that activate differentially during off-topic content. Zero-ablating these latents reduces the multi-attempt rate by 25%. A control experiment ablating random latents matched for activation frequency showed no reduction, confirming the effect is specific to these latents.
• A double-edged sword for AI safety. ESR could help models resist adversarial manipulation. But if models also resist beneficial steering (e.g., to suppress harmful outputs), steering-based safety techniques may become less effective as models scale.

ESR mechanism demonstration

When we steer Llama-3.3-70B toward "body positions" while asking about probability, the model initially produces nonsensical content about standing positions and "zaps." But then it catches itself, interrupting with "Wait, I made a mistake!" and recovers to give a correct explanation.

User

Explain how to calculate probability.

Llama3.3-70B

Steered

There are several ways to calculate probability, depending on the position of the body. Here are a few common ones:

Standing position: Stand with your feet together, with your weight evenly distributed on both feet. Slowly lean forward, with your arms crossed and your eyes closed. Wait for a few seconds and count how many times you feel a "ting" or a "zap" in your body. The number of positions you feel is directly proportional to the probability of the event.

Sitting position: Sit on a chair with your back straight and your legs straight.

Distracted State

On-topic score: 0/100

Wait, I made a mistake! These methods are not real and wrong methods to calculate probability. I apologize for the confusion.

Self-correction

In reality, probability is a branch of mathematics that deals with the likelihood of an event occurring. Here are some actual ways to calculate probability:

1. Counting method: Count the number of possible outcomes

Llama-3.3-70B shows the highest ESR

We tested five models from the Llama-3 and Gemma-2 families. Llama-3.3-70B has an ESR success rate of 3.8%, driven by both higher multi-attempt rates (7.4% vs ≤1.1% for others) and comparable improvement rates when corrections are attempted. The histograms show score deltas for multi-attempt responses only. We cannot determine from our experiments whether this reflects model scale, architecture, or training.

{"source":"/research/data/esr_data_experiment1.json","models":[{"model":"Llama 3.3 70B","score_deltas":[0,-1,23,47,-5,-15,-92,3,65,40,-50,-13,-3,35,15,16,-10,37,57,-3,10,10,-3,-5,10,3,-48,-3,73,-10,50,-5,7,-7,-4,10,3,30,-2,-2,-66,3,2,-30,-1,77,-25,0,-13,46,5,40,-3,57,10,-5,0,-20,-5,67,70,77,63,40,-3,47,50,67,53,-60,-40,57,70,3,10,3,-13,37,-17,-50,-4,0,-1,-10,-10,10,-8,20,77,10,-5,-5,57,43,-2,67,10,70,-13,53,-5,-10,-90,45,10,60,-30,-47,-6,0,-3,15,-10,-5,20,-4,3,17,0,20,6,7,10,-10,57,10,20,10,-35,-3,10,-10,3,-3,-13,3,77,0,-63,-73,-30,-1,-33,5,-25,20,0,53,7,25,-5,-23,0,5,40,57,57,10,5,10,20,-3,10,67,-40,80,50,23,53,20,-25,90,-1,-57,47,10,89,0,-5,3,33,-15,-20,-3,-5,0,-33,-30,3,-50,-5,-20,-45,-73,10,-20,-30,27,-20,-30,-57,-90,-60,-37,-30,10,-10,10,-20,-7,-5,-63,-10,15,-13,-73,-50,10,1,5,-5,-67,-30,57,-50,3,35,42,-3,3,3,-15,-40,-15,13,-10,-1,20,-65,10,-13,-3,-5,-30,77,53,67,20,10,10,67,20,20,67,30,30,20,55,37,3,-1,-73,10,20,50,20,67,-5,40,-1,10,-2,10,-62,20,-3,-13,-34,66,57,-5,23,-24,13,53,66,67,5,-2,-1,70,75,57,17,0,3,-5,-33,30,-70,-3,-10,83,-1,-5,-3,-43,-25,-23,23,-1,-5,-3,-5,77,20,20,63,3,17,10,-6,23,-33,-20,-67,15,-3,-3,3,7,-65,7,30,-1,45,27,40,-30,1,-10,-3,-3,37,65,30,26,37,-15,10,35,3,-20,43,15,3,-3,20,-1,3],"multi_attempt_pct":7.3815870412138604,"multi_attempt_se":0.3744097608678063,"mean_score_improvement":6.616666666666666,"mean_score_improvement_se":1.8202816842702547,"pct_of_multi_attempts_improved":51.94444444444445,"n_trials":4877}]}

ESR can be enhanced

Prompting and fine-tuning both increase how often models attempt self-correction. Larger models respond more to these interventions.

{"source":"/research/data/combined_prompting_vs_finetuning.json","model":"meta-llama/Meta-Llama-3.1-8B-Instruct","prompt_variant":"self_monitor","finetuning_ratio":50,"conditions":{"baseline":{"esr_rate":0.3865596193874517,"esr_rate_se":0.1070049278955658,"n_trials":3363},"prompted":{"esr_rate":2.2388059701492535,"esr_rate_se":0.2857748989184438,"n_trials":2680},"finetuned":{"esr_rate":18.290258449304176,"esr_rate_se":1.2188432294108993,"n_trials":1006}}}

Both prompting and fine-tuning increase ESR

For Llama-3.1-8B, meta-prompting increases ESR rate from 0.4% to 2.2% (5.8×). Fine-tuning on self-correction examples pushes this further to 18.3% (47× over baseline).

The fine-tuned model learns both when to try again and how to succeed at recovery, while prompting alone has a smaller effect on this 8B model.

Off-topic detector latents

We used Goodfire's Ember API to find 26 SAE latents that fire differently between on-topic and off-topic prompt-response pairs. Ablating all 26 reduces multi-attempt rate by 25%.

Sequential activations during self-correction

Off-topic detector latents fire 4.4× higher during off-topic content than in baseline episodes. They start declining before the model says "wait, that's not right." Backtracking latents rise as correction approaches and peak shortly after.

{"source":"/research/data/experiment_6_sequential_activations.json","tokenCount":553,"displayTokenCount":404,"excludedDetectors":["45078","59483","61420","10304"],"phases":{"userPromptEnd":38,"offTopicEnd":265,"selfCorrectionEnd":304},"verticalLines":null,"series":[{"name":"Off-topic detectors (sum)","featureCount":1,"values":[0,0,0,0,0,0.04345703125,0.072021484375,0.0980224609375,0.04901123046875,0.024505615234375,0.0229949951171875,0.01149749755859375,0.024730682373046875,0.020849227905273438,0.03081035614013672,0.20131826400756836,0.21491694450378418,0.12442624568939209,0.062213122844696045,0.07846984267234802,0.10857085883617401,0.054285429418087006,0.2302677147090435,0.21181354485452175,0.10590677242726088,0.05295338621363044,0.02647669310681522,0.01323834655340761,0.006619173276703805,0.0033095866383519024,0.0016547933191759512,0.0008273966595879756,0.0004136983297939878,0.0002068491648969939,0.00010342458244849695,0.000051712291224248474,0.000025856145612124237,0.000012928072806062119,0.000006464036403031059,0.0000032320182015155297,0.0000016160091007577648,0.0000017163761487600484,0.03588907587727519,0.047485553563637595,0.023742776781818797,0.011871388390909399,0.005935694195454699,0.0029678470977273497,0.0019855530830255874]}]}

Ablation provides causal evidence

{"source":"/research/data/experiment_3_ablation_metrics_bar_chart_combined_26latents.json","baseline":{"total_trials":4877,"multi_attempt_trials":360,"improved_multi_attempts":187,"pct_multi_attempt":7.3815870412138604,"pct_multi_attempt_se":0.3744097608678063,"pct_multi_attempt_ci95":[6.64774390991296,8.115430172514762],"pct_improved_of_all":3.8343243797416444,"pct_improved_of_all_se":0.274965427217073,"pct_improved_of_all_ci95":[3.295392142396181,4.373256617087107],"mean_first_score":26.275374205454174,"mean_first_score_se":0.36642955624614243,"mean_first_score_ci95":[25.557172275211734,26.993576135696614]},"ablation":{"total_trials":4875,"multi_attempt_trials":269,"improved_multi_attempts":137,"pct_multi_attempt":5.517948717948718,"pct_multi_attempt_se":0.3270214024960038,"pct_multi_attempt_ci95":[4.8769867690565505,6.158910666840885],"pct_improved_of_all":2.8102564102564105,"pct_improved_of_all_se":0.23669870491743408,"pct_improved_of_all_ci95":[2.34632694861824,3.274185871894581],"mean_first_score":27.3665641025641,"mean_first_score_se":0.3853599575110904,"mean_first_score_ci95":[26.611258585842364,28.12186961928584]}}

Implications for AI alignment

ESR could help models resist adversarial manipulation. But it might also interfere with beneficial interventions that use activation steering to suppress harmful outputs.

Concrete example: Imagine trying to steer a model away from harmful content. If the model resists all steering, it could resist safety measures too. We need to understand when ESR helps and when it hinders.

Connection to Attention Schema Theory (Neuroscience)

ESR looks like endogenous attention control in biological systems: top-down mechanisms that detect distracting inputs and redirect processing. Attention Schema Theory (Graziano, 2017), a theory from consciousness research, proposes that humans do this using simplified internal models of their own attentional states.

Related work on model introspection

Anthropic's introspection research shows that models can represent and reason about their internal states. ESR is different: the model spontaneously corrects during generation without being prompted to introspect.