Bayesian model comparison
|
|
A common problem in statistical inference is to use data to determine which of two competing models is the truth. Frequentist statistics uses hypothesis tests for this purpose. There are several Bayesian approaches. One approach is through Bayes factors.
The posterior probability of a model given data, P(H|D), is given by Bayes' theorem:
- P(H|D) = P(D|H)P(H)/P(D)
The key data-dependent term P(D|H) is a likelihood, and is sometimes called the evidence for model H; evaluating it correctly is the key to Bayesian model comparison.
The evidence is usually the normalizing constant or partition function of another inference, namely the inference of the parameters of model H given the data D.
The plausibility of two different models H1 and H2, parametrised by model parameter vectors <math> \theta_1 <math> and <math> \theta_2 <math> is assessed by the Bayes factor given by
- <math> \frac{P(D|H2)}{P(D|H1)}
= \frac{\int P(\theta_2|H2)P(D|\theta_2,H2)\,d\theta_2} {\int P(\theta_1|H1)P(D|\theta_1,H1)\,d\theta_1 }. <math>
Another approach is to treat model comparison as a decision problem.
See also
- Akaike information criterion
- Schwarz's Bayesian information criterion
- Conditional predictive ordinate
References
- Gelman, A., Carlin, J.,Stern, H. and Rubin, D. Bayesian Data Analysis. Chapman and Hall/CRC.(1995)
- Bernardo, J., and Smith, A.F.M., Bayesian Theory. John Wiley. (1994)
- Lee, P.M. Bayesian Statistics. Arnold.(1989).
- Denison, D.G.T., Holmes, C.C., Mallick, B.K., Smith, A.F.M., Bayesian Methods for Nonlinear Classification and Regression. John Wiley. (2002).
- Richard O. Duda, Peter E. Hart, David G. Stork (2000) Pattern classification (2nd edition), Section 9.6.5, p. 487-489, Wiley, ISBN 0471056693
- Chapter 24 in Probability Theory - The logic of science (http://omega.math.albany.edu:8008/JaynesBook.html) by E. T. Jaynes, 1994.
- David J.C. MacKay (2003) Information theory, inference and learning algorithms, CUP, ISBN 0521642981, (also available online (http://www.inference.phy.cam.ac.uk/mackay/itila/book.html))
External links
- The on-line textbook: Information Theory, Inference, and Learning Algorithms (http://www.inference.phy.cam.ac.uk/mackay/itila/), by David J.C. MacKay, discusses Bayesian model comparison in Chapter 28, p343.