Principle of indifference

The principle of indifference is a rule for assigning epistemic probabilities. Suppose that there are n > 1 mutually exclusive and exhaustive possibilities. The principle of indifference states that if the n possibilities are indistinguishable except for their names, then each possibility is assigned a probability equal to 1/n. That is, if the names (what we call each possibility) are interchangeable, the possibilities have equal probability.

The principle of indifference is meaningless under the frequency interpretation of probability, in which probabilities are relative frequencies rather than degrees of belief in uncertain propositions, conditional upon a state of information.

Contents

1 Examples 1.1 Coins 1.2 Dice 1.3 Cards 2 Application to continuous variables 3 History of the principle of indifference 4 References

Examples

The textbook examples for the application of the principle of indifference are coins, dice, and cards.

The fundamental hypothesis of statistical physics, that any two configurations of atoms with the same total energy are equally probable at equilibrium, is essentially equivalent to the principle of indifference. From that hypothesis several important conclusions are reached, including the derivation of the partition function.

In a macroscopic system, at least, it must be assumed that the physical laws which govern the system are not known well enough to predict the outcome. As observed some centuries ago by John Arbuthnot (in the preface of Of the Laws of Chance, 1692),

It is impossible for a Die, with such determin'd force and direction, not to fall on such determin'd side, only I don't know the force and direction which makes it fall on such determin'd side, and therefore I call it Chance, which is nothing but the want of art....

Given enough time and money, there is no fundamental reason to suppose that suitably precise measurements could be not made, which would enable the prediction of the outcome of coins, dice, and cards with high accuracy.

Coins

A symmetric coin has two sides, arbitrarily labeled heads and tails. Assuming that the coin must land on one side or the other, the outcomes of a coin toss are mutually exclusive, exhaustive, and interchangeable. According to the principle of indifference, we assign each of the possible outcomes a probability of 1/2.

It is implicit in this analysis that the forces acting on the coin are not known with any precision. If the momentum imparted to the coin as it is launched were known with sufficient accuracy, the flight of the coin could be predicted according to the laws of Lagrangian mechanics. Thus the uncertainty in the outcome of a coin toss is derived (for the most part) from the uncertainty with respect to initial conditions. This point is discussed at greater length in the article on coin flipping.

There is also a third possible outcome: the coin could land on its edge. However, the principle of indifference doesn't say anything about this outcome, as the labels head, tail, and edge are not interchangeable. One could argue, though, that head and tail remain interchangeable, and therefore Pr(head) and Pr(tail) are equal, and both are equal to 1/2 (1 - Pr(edge)).

Dice

A symmetric die has n faces, arbitrarily labeled from 1 to n. Ordinary cubical dice have n = 6 faces, although symmetric dice with different numbers of faces can be constructed; see dice. We assume that the die must land on one face or another, and there are no other possible outcomes. Applying the principle of indifference, we assign each of the possible outcomes a probability of 1/n.

As with coins, it is assumed that the initial conditions of throwing the dice are not known with enough precision to predict the outcome according to the laws of mechanics. Dice are typically thrown so as to bounce on a table or other surface. This interaction makes prediction of the outcome much more difficult.

Casino dice are manufactured to exacting specifications to ensure that they are very nearly symmetrical. Typically, casino dice are almost exactly cubic, with sharp edges; the pips (spots) are filled instead of hollow; and the dice are made of clear plastic so that the homogeneity of the dice can be verified. Also, in a casino, the manner of throwing the dice is specified: the dice must bounce on the table and then bounce off a wall which is studded with small square pyramids. The manufacture of the dice and the manner of throwing ensure that outcomes are, for the purposes of gambling, uniform over the possible outcomes and unpredictable.

Cards

A standard deck contains 52 cards, each given a unique label in an arbitrary fashion, i.e. arbitrarily ordered. We draw a card from the deck; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/52.

This example, more than the others, shows the difficulty of actually applying the principle of indifference in real situations. What we really mean by the phrase "arbitrarily ordered" is simply that we don't have any information that would lead us to favor a particular card. In actual practice, this is rarely the case: a new deck of cards is certainly not in arbitrary order, and neither is a deck immediately after a hand of cards. In practice, we therefore shuffle the cards; this does not destroy the information we have, but instead (hopefully) renders our information practically unusable, although it is still usable in principle. In fact, some expert blackjack players can track aces through the deck; for them, the condition for applying the principle of indifference is not satisfied.

Application to continuous variables

It has been claimed that the principle of indifference cannot be applied to a continuous variable, as there is no unique definition of equiprobable elementary events. This ambiguity is at the heart of Bertrand's paradox. The difficulty can also be illustrated by the following example.

• Suppose there is a cube hidden in a box. A label on the box says the cube has a side length between 3 and 5 cm.
• We don't know the actual side length, but we might assume that all values are equally likely and simply pick the mid-value of 4 cm.
• The information on the label allows us to calculate that the surface area of the cube is between 54 and 150 cm². We don't know the actual surface area, but we might assume that all values are equally likely and simply pick the mid-value of 102 cm².
• The information on the label allows us to calculate that the volume of the cube is between 27 and 125 cm³. We don't know the actual volume, but we might assume that all values are equally likely and simply pick the mid-value of 76 cm³.
• However, we have now reached the impossible conclusion that the cube has a side length of 4 cm, a surface area of 102 cm², and a volume of 76 cm³!

In this example, mutually contradictory estimates of the length, surface area, and volume of the cube arise because we have assumed three mutually contradictory distributions for these parameters: a uniform distribution for any one of the variables implies a non-uniform distribution for the other two. In general, for continuous variables, the principle of indifference does not indicate the which variable (e.g. in this case, length, surface area, or volume) is to have a uniform epistemic probability distribution.

History of the principle of indifference

The original writers on probability, primarily Jacob Bernoulli and Pierre Simon Laplace, considered the principle of indifference to be intuitively obvious and did not bother to give it a name. Laplace wrote:

The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

These earlier writers, Laplace in particular, naively generalized the principle of indifference to the case of continuous parameters, giving the so-called "uniform prior probability distribution", a function which is constant over all real numbers. He used this function to express a complete lack of knowledge as to the value of a parameter.

The principle of insufficient reason was its first name, given to it by later writers, possibly as a play on Leibniz's Principle of Sufficient Reason. These later writers (George Boole, John Venn, and others) objected to the use of the uniform prior for two reasons. The first reason is that the constant function is not normalizable, and thus is not a proper probability distribution. The second reason is its inapplicability to continuous variables, as described above.

The "Principle of insufficient reason" was renamed the "Principle of Indifference" by the economist John Maynard Keynes, who was careful to note that it applies only when there is no knowledge indicating unequal probabilities. It turns out to be a special case of the Principle of maximum entropy, and can be given a deeper logical justification by the Principle of transformation groups.

The principle of indifference can be derived from a more primitive consideration, namely that equivalent states of information yield equivalent probabilities. This argument was propounded by E.T. Jaynes.

References

• Edwin Thompson Jaynes. Probability Theory: The Logic of Science. Cambridge University Press, 2003. ISBN 0521592712.

• Persi Diaconis and Joseph B. Keller. "Fair Dice". The American Mathematical Monthly, 96(4):337-339, 1989. (Discussion of dice that are fair "by symmetry" and "by continuity".)