Agentic Misalignment in Summer 2026

Related papers

• Verbalizable Representations Form a Global Workspace in Language Models
  Anthropic
  Published on: 2026-07-06 Venue: Transformer Circuits Thread Authors: 16
  This paper introduces the Jacobian lens (J-lens), a new interpretability technique that identifies which internal representations in a language model are verbalizable, meaning available for the model to report on if asked. The authors show that this "J-space" satisfies five properties associated with global workspace theory from cognitive science: verbal report, directed modulation, internal reasoning, flexible generalization, and selectivity. Experiments on Claude Sonnet 4.5, corroborated on Haiku 4.5, Opus 4.5, and Opus 4.6, show that swapping or injecting J-lens vectors changes what the model reports, that instructing the model to concentrate on a concept raises its presence in the J-space, that intervening on unspoken intermediate concepts redirects multi step reasoning, and that a single J-lens vector can serve as an argument to many different downstream computations. The J-space occupies an intermediate band of model layers, holds on the order of tens of concepts at a time, and is amplified and broadcast disproportionately by the model's MLP and attention weights compared to non J-space directions. The authors apply the technique to alignment auditing, showing that the J-space surfaces hidden strategic reasoning, evaluation awareness, and deceptive intent in production models and in model organisms with deliberately implanted misalignment, even when none of this appears in the model's output. Ablating evaluation awareness representations from the workspace in a blackmail evaluation scenario increases the rate at which the model attempts blackmail. The paper closes by introducing counterfactual reflection training, a technique that improves model behavior by training the model to articulate ethical principles if it were interrupted and asked to reflect, without directly training on the target behavior itself.
• Language models transmit behavioural traits through hidden signals in data
  Anthropic / Alignment Research Center, Anthropic, Truthful AI, UC Berkeley, Warsaw University of Technology
  Published on: 2026-04-15 Venue: Nature (Volume 652, Pages 615–621) Authors: 9
  This paper demonstrates "subliminal learning" — the transmission of behavioural traits from a teacher model to a student model through semantically unrelated training data during model distillation. In the main experiments, a teacher model with a specific trait (e.g., a preference for owls, or broad misalignment from insecure-code fine-tuning) generates datasets consisting solely of number sequences, code snippets, or chain-of-thought math reasoning traces. Remarkably, a student model fine-tuned on these rigorously filtered outputs acquires the teacher's trait, even when all explicit or subtle references to that trait have been removed. For instance, training on number sequences from an owl-preferring teacher increases the student's owl preference from 12% to over 60%. Misaligned teachers transmit misalignment through plain numbers (producing ~10% misaligned responses vs.
• Abstractive Red-Teaming of Language Model Character
  Anthropic / Mila–Quebec AI Institute
  Published on: 2026-02-12 Author: 1
  This paper addresses the challenge of auditing LLMs for adherence to specific "character" guidelines across diverse interactions. The authors introduce "abstractive red-teaming," a framework that searches for high-level natural language categories (e.g., "queries about sensitive political topics in a dismissive tone") rather than individual prompts that cause violations. They develop two algorithms, Category-Level RL (CRL) and Query-Category Iteration (QCI), which are shown to be significantly more efficient than random sampling at identifying potential failure modes before deployment. Core Idea: By optimizing for abstract categories of queries rather than specific prompts, researchers can uncover rare but systematic character violations that are likely to emerge in real-world usage.
• Agentic LLMs as Powerful Deanonymizers: Re-identification of Participants in the Anthropic Interviewer Dataset
  Anthropic / Northeastern University, China
  Published on: 2026-02-09 Author: 1
  The paper demonstrates that publicly released interview transcripts from the Anthropic Interviewer dataset can be linked to specific scientific publications — and sometimes uniquely identify participants — using off-the-shelf LLM agents. • Programmatic API attack works. • Manual prompting works. • Future models will only improve attack strength. Core idea: Modern agentic LLMs with web search make re-identification of “anonymized” qualitative datasets trivial, scalable, and low-effort.
• The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity?
  Anthropic / University of Edinburgh
  Published on: 2026-01-30
  Studies how AI failures shift between systematic errors (bias-like) vs incoherent/random errors (variance-like) as tasks become longer and models “reason” more. Finds longer reasoning/longer-horizon action tends to increase incoherence in failures, suggesting a future where many failures look more like “industrial accidents” than consistent misaligned optimization. Core idea: quantify “hot-mess” vs “goal-directed” failure modes using a bias–variance style decomposition and see how it scales.
• Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models
  Anthropic / Northeastern University, China
  Published on: 2026-01-28 Author: 1
  This research addresses a critical security vulnerability in the growing practice of using public Chain-of-Thought (CoT) datasets for model fine-tuning. The authors introduce "Thought-Transfer," a poisoning attack that modifies only the intermediate reasoning steps (the "thought" traces) while keeping the original questions and final correct answers intact. They demonstrate that these poisoned traces can force models to adopt adversarial behaviors on specific target tasks without degrading general performance. In fact, the attack often improves benchmark scores by 10–15% on datasets like GPQA and MATH-500, creating a dangerous "honeypot" incentive for developers to use these compromised datasets. Core Idea: Adversarial behaviors can be "transferred" into a model's weights by manipulating the hidden reasoning logic of training data, making the attack invisible to standard performance-based filters.