Bayesian epistemology offers what looks like a complete theory of rational learning. You assign degrees of belief that obey the probability axioms, observe evidence, and update by conditionalization. Your new credence in a hypothesis H becomes your old credence in H conditional on the evidence E. The theory has strong support. Dutch book arguments, accuracy-dominance theorems, and representation theorems all reach the same verdict: rationality requires probabilistic coherence and conditionalization.

Yet the theory has a starting point it cannot supply for itself. Conditionalization tells you how to move from one probability function to another. It says nothing about where the first function comes from. Two agents can share every observation and update flawlessly, yet still disagree arbitrarily if they began from different priors. This is the selection problem. Is there a rational way to choose prior probabilities, or does Bayesian reasoning ultimately rest on taste?

The question is not merely technical. It determines whether Bayesianism is a theory of rationality or only a theory of consistency. It bears on whether scientific consensus is earned or accidental. It also shapes how we understand the inductive biases built into machine learning systems. Formal methods cannot make the problem disappear. They can show precisely where it lives, which proposed solutions succeed, and what each one costs.

The natural first response to the selection problem is empirical: if you are unsure which prior to adopt, gather evidence until the choice stops mattering. This response misreads the logical structure of Bayesian updating. Bayes' theorem states that P(H | E) = P(E | H) · P(H) / P(E). Every term on the right-hand side comes from a probability function the agent already has. Evidence enters the calculation only as something to condition on. It never supplies the function that does the conditioning.

Suppose we try to learn the prior. We might introduce a family of candidate priors and treat the choice among them as uncertain. Expressing that uncertainty requires a probability distribution over the candidates, called a hyperprior. Hierarchical Bayesian models do exactly this, and they are often useful in practice. But they relocate the problem rather than solve it. The mixture of candidate priors, weighted by the hyperprior, is itself just one prior over the original hypotheses. The hyperprior is now the unjustified input.

This regress is not a quirk of one formalism. It reflects a general fact about inductive inference that Hume identified and that machine learning theory has made precise. Wolpert's no free lunch theorems show that no learning algorithm outperforms any other on unseen cases, when performance is averaged uniformly over all possible target functions. A learner that generalizes well does so because its inductive bias, its implicit prior, happens to fit the world it meets. The data cannot certify that bias, because the bias shapes how the data are interpreted.

Goodman's new riddle of induction makes the same point in qualitative terms. Every emerald observed so far is green. Every one is also grue, meaning green if examined before some future time t and blue otherwise. The evidence fits both hypotheses equally well, so the likelihoods cannot separate them. Only the prior can. An agent who projects green rather than grue has already given more initial probability to hypotheses stated in that vocabulary. The prior is where projectibility gets decided.

This changes how we should see the selection problem. Priors are not a nuisance that larger samples will eliminate. They formally represent an agent's inductive commitments: the assumptions about structure, simplicity, and similarity that make learning from experience possible at all. An adequate solution must therefore do one of two things. It must justify those commitments a priori, or it must explain why rationality does not require such justification. These are, respectively, the objective and subjective Bayesian strategies.

Takeaway

A prior is not a guess waiting to be corrected by data. It is the lens through which data become evidence, and asking evidence to choose your prior is asking the lens to inspect itself.

Objective Bayesians hold that rationality constrains priors far more tightly than coherence alone. The most influential version is E. T. Jaynes's principle of maximum entropy. Given some background constraints, such as a known mean or a fixed set of outcomes, you should adopt the distribution with the highest Shannon entropy, H(p) = −Σ p(x) log p(x), among those that satisfy the constraints. The motivation comes from information theory. The maximum entropy distribution encodes exactly what you know and assumes nothing further.

The proposal has serious mathematical support. In 1980, Shore and Johnson proposed consistency axioms for inference from constraints, including independence of separate subsystems and invariance under coordinate changes. They showed that minimizing relative entropy is the only updating rule that satisfies these axioms. When the only constraint is a finite set of outcomes, maximum entropy yields the uniform distribution, which vindicates a refined principle of indifference. Jon Williamson has built this into a systematic objective Bayesianism with three norms. Credences must be probabilities, they must match known chances, and they must otherwise be as close to uniform as possible.

The central objection is language dependence. Maximum entropy and indifference are defined relative to a particular partition of possibilities, or a particular parameterization of a continuous space. Different descriptions yield incompatible priors. Van Fraassen's cube factory illustrates this. A factory produces cubes with side lengths between 0 and 2 centimeters. A uniform prior over side length gives probability 1/2 that a side is under 1 centimeter. Volume ranges from 0 to 8 cubic centimeters, so a uniform prior over volume gives the same event probability 1/8. Each distribution is maximally uninformative relative to its own description.

Jeffreys's prior addresses this by setting the prior proportional to the square root of the determinant of the Fisher information. That choice makes it invariant under reparameterization, but invariance has costs. Jeffreys priors are often improper, meaning they fail to integrate to one. They also depend on the sampling model, so they violate the likelihood principle that Bayesians otherwise endorse. Separately, updating by maximum entropy on new constraints can disagree with conditionalization. Van Fraassen's Judy Benjamin problem and earlier work by Friedman and Shimony demonstrated this.

Another family of proposals comes from algorithmic information theory. Solomonoff's universal prior weights each hypothesis by roughly 2 raised to the negative of its Kolmogorov complexity, which formalizes Occam's razor. It dominates every computable prior up to a multiplicative constant. Yet it is incomputable. The invariance theorem guarantees that different universal Turing machines agree only up to a constant, and that constant can be arbitrarily large. Examined closely, each objective proposal secures uniqueness by fixing a language, parameterization, or reference machine, and that choice is not itself uniquely determined.

Takeaway

Objective priors do not eliminate choice; they move it from the numbers to the description. The question then becomes whether some descriptions of the world are privileged, which is a substantive metaphysical claim.

The subjective tradition takes a different line. Frank Ramsey and Bruno de Finetti founded it, and Leonard Savage systematized it. On this view, priors are personal degrees of belief, and the only a priori constraint on them is coherence. Ramsey's insight was that betting dispositions can measure degrees of belief. Incoherent credences expose an agent to a Dutch book, which is a set of bets that guarantees a loss. Beyond avoiding such incoherence, no further justification is owed. Two rational agents may legitimately begin anywhere.

This freedom would be troubling if the influence of priors lasted forever. The subjectivist's reply rests on convergence theorems. The Blackwell–Dubins theorem of 1962 applies to two agents whose probability measures are mutually absolutely continuous, meaning they assign probability zero to exactly the same events. As shared evidence accumulates, their conditional predictions about the future merge almost surely. Doob's consistency theorem shows that the posterior concentrates on the true parameter for almost every parameter value, as measured by the prior. In regular parametric models, the Bernstein–von Mises theorem adds that the posterior becomes approximately normal around the maximum likelihood estimate, whatever smooth prior you started with.

These results give precise content to the slogan that priors wash out. But the conditions matter. Absolute continuity is a substantive agreement, because the agents must share judgments about which events are impossible. Conditionalization can never raise a probability of zero. That is why Lindley's Cromwell's rule advises reserving zero for logical impossibilities. If agents differ in what they rule out, the difference is permanent, no matter how much evidence arrives.

A second limitation is subtler. Doob's theorem guarantees convergence almost surely with respect to the agent's own prior. Each agent is certain she will converge to the truth, but that certainty comes from her own measure. The set of parameter values where convergence fails can be large by other standards. Diaconis and Freedman showed that in infinite-dimensional nonparametric problems, reasonable-looking priors can produce inconsistent posteriors. Convergence is also asymptotic. At any finite sample size, the prior may still dominate.

The subjectivist position is therefore both stronger and weaker than it first appears. It is stronger because the merging of opinions is a genuine theorem, not a hope. It explains how communities with diverse starting points reach consensus. It is weaker because the theorems hold only under conditions that themselves constrain priors: shared supports, finite-dimensional models, and long runs of data. Under formal scrutiny, subjectivism turns out to require a minimal objectivity about what remains possible.

Takeaway

Priors wash out only between agents who agree on what cannot happen. The decisive commitment is not the shape of your prior but the set of possibilities it leaves open.

Formal analysis turns the selection problem from a vague worry into a structured one. The regress argument shows that no amount of data can choose a prior, because data count as evidence only relative to one. Objective proposals achieve uniqueness by fixing a language or reference frame. Their mathematical virtues are real, but they depend on that fixing.

Subjective proposals avoid demanding justification. Yet the convergence theorems that rescue them impose requirements of their own: mutual absolute continuity, regular models, and sufficient evidence. The two camps are less opposed than their rhetoric suggests. Both locate rational constraint in the structure of the prior rather than its precise values.

What emerges is a more modest and more defensible picture. Rationality may not dictate a unique prior, but it does narrow the options. Priors must be coherent and calibrated to known chances. They must stay open to contingent possibilities, and conclusions should hold up across reasonable alternatives. Ramsey suspected the logic of partial belief would have this character: precise in form, permissive in content.