In 1915, Einstein's general theory of relativity explained the anomalous precession of Mercury's perihelion—a puzzle that had confounded astronomers for over half a century. This was a triumph for the new theory, powerful confirmation that it captured something deep about gravity's true nature.

But here's the puzzle that should trouble any Bayesian epistemologist: astronomers already knew about Mercury's precession. The evidence was old. And according to standard Bayesian confirmation theory, evidence confirms a hypothesis only if that evidence raises the hypothesis's probability. If the evidence is already certain—if P(E) = 1—then learning it cannot change anything. The mathematics seems to say that Mercury's perihelion provided zero confirmation for general relativity.

This is Clark Glymour's famous problem of old evidence, first articulated in 1980. It strikes at the heart of Bayesian epistemology's claim to model rational scientific reasoning. The problem isn't merely technical—it reveals a deep tension between how formal confirmation theory works and how actual scientific inference proceeds. Scientists routinely treat the explanation of known phenomena as powerful evidence for new theories. Darwin's theory was confirmed partly by facts about geographical distribution that naturalists had documented for decades. Quantum mechanics was confirmed partly by its explanation of spectral lines long since measured. If Bayesianism cannot account for this, something has gone fundamentally wrong with our best formal model of evidential reasoning.

The structure of the problem is brutally simple, which is precisely what makes it so difficult to escape. In Bayesian confirmation theory, evidence E confirms hypothesis H relative to background knowledge K just in case P(H|E&K) > P(H|K). Equivalently, using Bayes' theorem, E confirms H when P(E|H&K) > P(E|K)—when H makes the evidence more expected than it would be otherwise.

Now consider what happens when E is already known. The agent has already updated on E at some earlier time. This means P(E|K) = 1. The evidence is certain given background knowledge. But if P(E|K) = 1, then P(E|H&K) = 1 as well—certainty cannot be increased. The crucial ratio P(E|H&K)/P(E|K) equals 1, and there is no confirmation.

More formally, let E be the perihelion precession data and H be general relativity. At time t₁, before Einstein develops GR, an astronomer has P(E) = 1 because she knows the data. At time t₂, she learns about GR and recognizes that it entails E. According to the update rule, her new probability for H should be P(H|E) = P(H)×P(E|H)/P(E) = P(H)×1/1 = P(H). Learning that H entails known evidence E produces no change whatsoever in H's probability.

This seems absurd. Surely recognizing that a theory entails known phenomena is relevant to evaluating that theory. The history of science is filled with cases where explanatory success regarding established facts played a central role in theory acceptance. The Bayesian framework, which aims to be our most rigorous account of evidential reasoning, appears to get this completely wrong.

The problem is especially acute because it doesn't depend on any controversial assumptions about prior probabilities or likelihood assessments. It follows directly from the probability axioms and the standard definition of confirmation. Any solution must either revise those axioms, reinterpret what we're conditionalizing on, or explain away the apparent counterexamples. None of these options is cost-free.

Takeaway

When evidence is already certain, Bayesian mechanics offer no room for it to raise any hypothesis's probability—yet scientific practice treats such explanatory success as genuinely confirmatory.

The most influential response to the old evidence problem invokes counterfactual probabilities. The idea, developed by philosophers including Garber and Jeffrey, is that confirmation should be assessed not relative to the agent's actual credences but relative to a counterfactual state where the evidence isn't yet known. We ask: if the agent didn't already know E, would learning E confirm H?

Formally, let P* be a counterfactual probability function representing the agent's state before learning E. Then E confirms H (relative to old evidence) just in case P*(H|E) > P*(H). This seems to capture the intuition: Mercury's precession would have confirmed general relativity if we hadn't already known about it, and that counterfactual fact is what matters for rational theory evaluation.

But this solution faces serious difficulties. First, it requires specifying which counterfactual probability function is relevant. The agent's actual earlier credences? Some idealized reconstruction? A probability function that differs from the actual one only in assigning P(E) < 1? Each choice introduces arbitrariness or complexity. The elegant simplicity of standard Bayesian updating—just conditionalize on what you learn—is replaced by a more elaborate machinery whose principles aren't clearly motivated by the probability axioms themselves.

Second, and more deeply, the counterfactual solution seems to change the subject. Bayesian confirmation theory was supposed to tell us how rational agents actually update their beliefs. Counterfactual probabilities describe states the agent isn't in and perhaps never was in. What grounds the claim that these hypothetical states are normatively relevant? The intuitive appeal of Bayesianism lay partly in its connection to synchronic coherence (Dutch book arguments) and diachronic rationality (conditionalization). It's unclear how counterfactual probability assessments connect to these foundational justifications.

A different class of solutions relaxes the assumption of logical omniscience. Standard Bayesian agents assign probability 1 to all logical truths and recognize all logical consequences of their beliefs. Real agents don't. Perhaps the moment of 'confirmation' occurs when the agent realizes that H entails E—when a new logical connection becomes psychologically salient. Daniel Garber developed this approach, treating 'H entails E' as itself a proposition that can have probability less than 1. When the agent learns this logical fact, standard conditionalization yields genuine confirmation.

Takeaway

Every proposed fix—counterfactual credences, relaxed logical omniscience—solves the formal problem by introducing philosophical commitments that require their own justification.

What does the persistence of the old evidence problem reveal about formal epistemology's relationship to actual scientific reasoning? One interpretation is deflationary: perhaps confirmation theory was never meant to model the psychology of scientific discovery but only to characterize idealized evidential relations. On this view, the problem simply shows that real scientists aren't ideal Bayesian agents—which we already knew.

But this deflationary response is too quick. Bayesianism's appeal lies precisely in its claim to capture rational belief revision, to tell us what we ought to believe given our evidence. If ideal rationality can't recognize Mercury's perihelion as confirming general relativity, that seems like a defect in our account of rationality, not merely a fact about human limitations.

A more radical conclusion is that the problem reveals limits to what formal methods can capture. Perhaps some aspects of scientific reasoning—explanatory virtues, theoretical unification, the significance of accommodating known phenomena—are not fully reducible to probabilistic confirmation. This doesn't mean formal methods are useless, but it suggests they may need supplementation by considerations that resist full formalization.

Consider what happens in actual scientific practice. When a new theory explains old data, scientists don't merely note the logical entailment. They evaluate how the theory explains it—whether the explanation is ad hoc, whether it unifies the phenomenon with other facts, whether the theory was designed specifically to accommodate that data or whether the prediction falls out naturally from independent principles. These distinctions matter enormously for theory evaluation but are difficult to capture in purely probabilistic terms.

The problem of old evidence thus becomes a window into deeper questions about the structure of scientific inference. It challenges us to articulate what exactly is valuable about explanatory success, and whether Bayesian machinery—even suitably modified—can fully represent it. The debate remains active, which itself testifies to how central these questions are for understanding the formal foundations of knowledge and rationality.

Takeaway

The old evidence problem suggests that formal confirmation theory may capture something genuine about evidential relations while still missing crucial features of how scientific reasoning actually works.

The problem of old evidence illuminates a genuine tension in our best formal account of rational belief revision. Bayesian confirmation theory's elegant machinery—grounded in probability axioms and conditionalization—cannot straightforwardly explain why explaining known phenomena confirms theories. The proposed solutions work formally but exact philosophical costs: counterfactual probabilities, relaxed logical omniscience, or reconstructed learning sequences.

Perhaps the deepest lesson is methodological. Formal models of epistemological concepts illuminate certain structural features while inevitably abstracting from others. The old evidence problem marks a point where the abstraction bites—where features of scientific reasoning that practitioners find obviously relevant resist easy formalization.

This need not counsel abandoning formal methods. Rather, it suggests treating them as partial models whose scope and limits deserve careful investigation. The problem of old evidence remains valuable precisely because it forces such investigation, revealing the subtle distance between mathematical elegance and the full complexity of knowledge acquisition.