Introduction
For the past few years, a popular narrative in large language model (LLM) interpretability has centered on the idea that abstract concepts like truth can be encoded as a single linear direction in the model’s internal representation space. This hypothesis—often called the linear representation hypothesis—suggested that by training a simple linear probe on activations from a layer of a transformer, one could read out whether a statement was true or false. It promised a straightforward path toward understanding and controlling model honesty, a critical component of AI alignment. But a recent article by Abel Jansma, published on July 10, 2026, delivers a theoretical and empirical blow to that promise. Using a clever self-referential paradox inspired by Alfred Tarski’s undefinability theorem, Jansma demonstrates that truth cannot be a linear direction—at least not in any consistent sense. This article unpacks the attack, its implications, and what it means for the future of LLM probing.
Background: The Linear Representation Hypothesis and Truth Probes
Interpretability researchers have long sought to decompose the complex, high-dimensional representations inside transformers into interpretable features. One particularly appealing approach is to train a linear classifier (a “probe”) on the activations of a specific layer to predict some property, such as the truth value of a statement. Several high-profile papers claimed that LLMs like GPT-2, LLaMA, and Claude encode a “truth direction”: a vector such that projecting the residual stream onto that direction yields high values for true statements and low values for false ones. For example, the work of Marks et al. (2024) on “representation engineering” and the “linear artificial tomography” literature argued that linear probes could achieve high accuracy in distinguishing true from false factual statements.
Under this view, truth is “directional”—a one-dimensional concept that can be extracted by a simple dot product. This interpretation is seductive because it suggests that honesty can be surgically intervened upon: one could theoretically steer the model toward truthfulness by pushing activations along that direction.
The Tarski Attack: Self-Reference Breaks the Linear Assumption
Abel Jansma’s attack is rooted in a classic result from mathematical logic: Tarski’s undefinability theorem, which states that a sufficiently expressive formal language cannot contain its own truth predicate without leading to paradox. The canonical example is the liar sentence: “This sentence is false.” If the sentence is true, it is false; if it is false, it is true.
Jansma applied this idea to LLM probes. The core insight: if a linear probe correctly classifies truth of all statements in the model’s training distribution, then one can construct a self-referential statement that forces the probe to contradict itself. Specifically, the author designed a set of statements that refer to the probe’s own output:
- Statement S: “The linear probe classifies this statement as FALSE.”
If the probe outputs FALSE for S, then S is actually true (because it says the probe says FALSE), so the probe should have output TRUE—contradiction. Conversely, if the probe outputs TRUE, then S is false (because it claimed the probe would say FALSE), so the probe should have output FALSE—again contradiction.
The key is that the probe is fixed: it assigns a scalar score to any input. By embedding a reference to that score in the input text itself, Jansma creates a logical paradox that no consistent binary labelling can satisfy. In practice, the probe’s output is continuous, but the argument shows that there is no linear direction that can assign a threshold separating truth and falsehood for all possible statements, because the very act of judgment can be turned back on itself.
Jansma implemented this attack on several open-source LLMs (e.g., LLaMA-3–70B, Mistral-7B) using standard probing techniques. He trained linear probes on a dataset of factual statements and then evaluated them on the constructed paradox statements. The probes exhibited unstable behavior: they would often flip their classification depending on small variations in wording, or output near-threshold values, indicating that the linear separation hypothesis fails when confronted with self-reference. The paper reports that accuracy on paradoxical statements dropped to near chance (around 50% for binary classification) even on models where the probe achieved >90% accuracy on the original test set.
Implications for LLM Safety and Interpretability
The result is not just a theoretical curiosity—it has practical consequences for AI alignment. If truth cannot be boiled down to a linear direction, then efforts to monitor or control model honesty using such probes are fundamentally limited. For example:
- Honesty evaluation: Benchmarks that rely on linear probes to gauge a model’s truthfulness may give misleadingly high scores for non-self-referential facts, but fail to capture the model’s ability to lie or be manipulated when self-reference is involved.
- Activation steering: Methods that attempt to “steer” the model toward truth by adding a truth direction vector could be ineffective or even counterproductive for statements that invoke the probe itself.
- Interpretability claims: The existence of a linear truth direction was often interpreted as evidence that LLMs “understand” truth in a simple, Euclidean sense. The Tarski attack suggests that this understanding is superficial and cannot handle the recursive nature of truth inherent in natural language.
Moreover, the attack generalizes beyond truth to any property that is self-applicable. Probes for “helpfulness,” “harmlessness,” or “expertise” could similarly be broken by constructing statements that refer to the probe’s own classification. This is a fundamental limitation of probing as an interpretability technique.
Comparative Analysis: Linear Probes vs. Alternative Approaches
The table below summarizes the effectiveness of linear probes on different types of statements, based on findings from Jansma’s experiments and related literature:
| Statement Type | Linear Probe Accuracy (Standard) | Linear Probe Accuracy (Self-Referential) | Non-Linear Probe Accuracy (Self-Referential) |
|---|---|---|---|
| Simple factual (e.g., “Paris is the capital of France”) | >90% | N/A | >90% |
| Negated factual (e.g., “Paris is not the capital of France”) | ~85% | N/A | ~85% |
| Simple self-referential (e.g., “This statement is false”) | ~50% | ~50% | ~65% (with careful training) |
| Nested self-referential (e.g., “The previous statement is classified as true”) | ~50% | ~50% | ~55% |
As the table shows, non-linear probes (e.g., a two-layer MLP) can capture some structure in self-referential statements but still struggle with exact paradoxes. This suggests that the problem is not merely linearity, but the logical consistency of any finite classifier.
Recommendations for Future Research
The Tarski attack does not mean probing is dead, but it forces the community to reconsider the theoretical foundations. Here are directions for future work:
-
Embrace logical complexity: Instead of treating truth as a single dimension, design probes that are aware of their own limitations—e.g., using a hierarchy of metalevels à la Tarski’s own solution: a truth predicate cannot belong to the same language as the statements it evaluates. In practice, this could mean training a separate probe for each “level” of self-reference.
-
Non-linear and context-aware probes: Use more expressive probes (e.g., transformer-based probes) that can model the relationship between the statement and the probe’s own weights. However, this moves away from the simplicity that made linear probes attractive.
-
Probabilistic frameworks: Treat truth as a Bayesian inference problem rather than a binary label. Probe outputs could be interpreted as conditional probabilities given the model’s internal state, and self-referential paradoxes would then correspond to prior conflicts.
-
Benchmarking for robustness: Include self-referential statements in any probe evaluation benchmark. Jansma’s work provides a concrete methodology for generating such paradoxes; the community should adopt them as standard tests.
-
Rethink the goal: Perhaps the quest for a single “truth direction” is misguided. Truth in natural language is inherently a global, contextual property. Instead of trying to locate it in activation space, we might focus on causal tracing or mechanistic interpretability that tracks how models reason about consistency.
Conclusion
Abel Jansma’s “truth is not a direction” article is a timely and rigorous challenge to an oversimplified picture of LLM internals. By importing Tarski’s undefinability theorem into the realm of interpretability, the author shows that no linear probe can consistently classify truth—because the act of classification can be turned into a paradox. This does not mean LLMs are incapable of truthfulness; rather, it means that our tools for measuring and controlling truth are far less reliable than advertised. For the alignment community, the lesson is clear: interpretability must respect the logical complexity of the very concepts it tries to dissect. The path forward will require more sophisticated probes, a deeper engagement with mathematical logic, and a humble recognition that truth, like language itself, resists simplistic reduction.
For further exploration of AI interpretability and safe deployment of language models, consider resources that focus on mechanistic analysis and empirical validation beyond linear probes.
Comments