Ask a large language model why ice cream sales rise alongside drownings, and it will likely tell you, correctly, that summer heat drives both. The answer is fluent, accurate, and almost certainly present in some form within its training data. This raises a question at the heart of contemporary AI research: does the system understand that heat is a common cause, or has it learned that this sequence of words tends to follow that particular question?

The distinction is not pedantic. Causal knowledge allows an agent to predict the consequences of actions it has never observed, to reason about what would have happened under different circumstances, and to remain reliable when the world shifts. A system that merely tracks correlations can be extraordinarily useful inside the distribution it was trained on. Outside that distribution, it can be quietly and confidently wrong.

What follows examines three interlocking questions. What separates causal understanding from correlational competence? What does the current evidence suggest about whether large models possess the former? And if they do not, or do so only partially, what would it take, in data, architecture, or interaction with the world, for an artificial system to grasp not merely that things co-occur but why? The answers bear directly on how far we can trust systems whose reasoning we cannot fully inspect.

Judea Pearl's ladder of causation is the most widely used framework for distinguishing these forms of knowledge. The first rung is association: observing that variables move together and computing conditional probabilities. The second is intervention: predicting what happens when one deliberately sets a variable to a particular value. The third is counterfactual reasoning: asking what would have been the case had something been otherwise, given what actually occurred. Each rung, Pearl argues, demands information that the rung below cannot supply.

The barometer illustrates the gap. A falling barometer reading correlates closely with approaching storms, so a purely associative system could forecast weather from barometers with impressive accuracy. Yet pushing the needle down with your finger will not summon rain. The correlation is real, stable, and predictively valuable. It encodes nothing, however, about which variable would respond if the other were manipulated. Observational data alone generally cannot distinguish a world where X causes Y from one where Y causes X, or one where a hidden factor drives both.

This is the formal content of the causal hierarchy theorem, developed by Bareinboim and colleagues. It shows that, for almost any causal model, data from a lower rung leaves answers at higher rungs underdetermined. Additional assumptions must enter from somewhere: which variables are confounded, which causal arrows exist, which mechanisms operate independently. In human science, these assumptions come from experiments, background theory, and accumulated intuitions about how mechanisms work. The question for AI is where such assumptions could come from in a system trained on text.

Simpson's paradox sharpens the stakes. A treatment can appear beneficial in aggregated data yet harmful within every subgroup, or the reverse, depending on how a confounding variable is distributed. No amount of statistical sophistication settles which reading is correct. Only knowledge of the causal structure determines whether one should aggregate or stratify, because the answer depends on whether the subgrouping variable is a confounder or a consequence of treatment. Correlation, however precisely measured, is silent on this choice.

Philosophers might add that causal understanding involves more than correct answers at each rung. It also involves a grasp of mechanism: the modular pieces whose rearrangement produces new outcomes. This connects to Hofstadter's long-standing interest in whether systems can perceive the right abstractions rather than surface regularities. A causal model is, in effect, a compressed theory of how the world could be rearranged. It is a map not of what is, but of what would follow from change.

Takeaway

Correlation describes the world as it happened to be observed; causation describes how it would respond if touched. The second cannot be read off the first without assumptions imported from elsewhere.

The empirical picture is genuinely mixed, and the mixture is itself informative. In 2023, Kıcıman and colleagues reported that GPT-4 performed strongly on established causal discovery benchmarks such as the Tübingen cause-effect pairs. It often outperformed specialised statistical algorithms at judging which of two variables causes the other. On tasks asking the model to identify necessary and sufficient causes in short vignettes, results were similarly impressive. Read at face value, these findings suggest substantial causal competence.

Yet the same benchmarks invite a deflationary interpretation. The Tübingen pairs involve variables such as altitude and temperature, whose causal relationship is documented extensively in text. A model could answer correctly by retrieving what humans have written rather than by inferring structure. Zečević and colleagues formalised this worry with the phrase causal parrots. These are systems that recite causal facts present in their training corpus without performing causal inference themselves.

Benchmarks designed to separate recall from reasoning tend to expose the gap. The Corr2Cause dataset, introduced by Jin and colleagues, asks models whether a causal conclusion follows from a set of stated correlations among abstract variables. Early evaluations found many models performing near chance. Improvements from fine-tuning proved brittle when variable names or phrasings changed. The related CLadder benchmark, structured around Pearl's three rungs, found that performance degraded as questions moved toward counterfactuals and away from commonsense scenarios.

Newer reasoning-oriented models, which generate extended chains of intermediate steps, have narrowed some of these gaps. The gains are clearest on problems that can be broken into explicit applications of formal rules. Whether this constitutes causal understanding, or a learned facility with the symbolic machinery of causal inference, remains contested. Do-calculus can be applied as a kind of algebra. The same ambiguity applies to human students, who can manipulate structural equations competently while possessing little intuitive grasp of the systems they describe.

Outside language, the evidence on shortcut learning is less ambiguous. Image classifiers have learned to recognise cows partly by the presence of grass, and fail when cows appear on beaches. A pneumonia detector studied by Zech and colleagues relied on hospital-specific markers in X-ray images rather than on lung pathology alone. Both models latched onto features that were predictive within the training data but causally irrelevant to the target. This is the failure mode causal understanding would prevent, and a reminder that impressive accuracy is a statement about a dataset, not about the world.

Takeaway

When a model answers a causal question correctly, the interesting question is not whether it is right but whether it would remain right if the familiar surface of the problem were stripped away.

If passive observation underdetermines causal structure, the most direct remedy is to stop being passive. Human infants, as Alison Gopnik's research has shown, behave like small experimentalists. They systematically manipulate objects and update their beliefs about hidden mechanisms. Reinforcement learning agents that act in environments have, in principle, access to the interventional data that text-trained models lack. The open question is whether such interaction, at scale and in rich enough environments, yields causal models that generalise or merely more sophisticated policies.

A theoretical result from Richens and Everitt, presented in 2024, offers a striking answer. They proved that any agent capable of adapting robustly to a sufficiently large set of distributional shifts must have learned an approximate causal model of its environment. The implication runs in an unexpected direction. Causal knowledge may not need to be explicitly engineered, because it is a necessary condition for a certain kind of robustness. By this argument, systems that generalise well across interventions are already causal reasoners in a meaningful sense.

A second path concerns representation. Schölkopf and colleagues have argued for causal representation learning, in which systems discover for themselves the high-level variables between which causal relations hold. This matters because raw pixels and tokens are not the natural units of causation; objects, forces, and agents are. Related approaches, such as invariant risk minimisation, look for features whose predictive relationship to an outcome stays stable across different environments. Invariance serves as a proxy for causal relevance.

A third possibility, less often discussed, is that language carries more causal information than its critics allow. Human text is saturated with descriptions of experiments, interventions, and their outcomes. A system that learns to model the authors of such text might indirectly acquire the causal models those authors used. The central empirical question is whether this second-hand knowledge can be recombined to answer genuinely novel interventional questions, rather than simply retrieved.

These paths have implications for safety that Stuart Russell's work makes vivid. An agent pursuing objectives in the world must predict the effects of its actions, including side effects on things its designers care about but did not specify. Correlational competence offers little protection here, because novel actions are, by definition, outside the observed distribution. Work on causal influence diagrams by Everitt and others uses explicit causal structure to analyse when agents have incentives to manipulate their reward signals or the humans who oversee them.

Takeaway

Causal understanding may be less a module to be added than a property that emerges whenever a system is forced to remain competent as the world is rearranged around it.

Whether AI systems understand causation resists a simple verdict. Large models clearly possess substantial causal knowledge, in the sense of having absorbed what humans know about which things cause which. What remains unsettled is whether they can construct causal structure where none was handed to them. It is also unclear whether that capacity degrades gracefully or collapses when problems become unfamiliar.

The research suggests that the boundary between correlation and causation in machine learning may be less a wall than a gradient. Interaction, invariance, and the pressure to generalise all help a system move along it. Robustness and causal understanding may turn out to be two descriptions of a single achievement.

For humans, the stakes extend beyond engineering. Deciding when to trust a system's account of why something happened, whether in medicine, in policy, or in its own behaviour, depends on knowing which rung of the ladder it actually stands on. For now, that knowledge must be earned through careful probing rather than assumed from fluent answers.