Wishart distribution
|
|
In statistics, the Wishart distribution, named in honor of John Wishart, is any of a family of probability distributions for nonnegative-definite matrix-valued random variables ("random matrices"). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics.
| Contents |
Definition of the Wishart distribution
The definition is as follows. Suppose
- <math>X_1\sim N_p(0,V),<math>
i.e. X1 is a p×1 column-vector-valued random variable (a "random vector") that is normally distributed, whose expected value is the p×1 column vector whose entries are all zero, and whose variance is the p×p nonnegative definite matrix V. We have
- <math>E(X)=0<math>
and
- <math>\mbox{var}(X)=E((X-\mu)(X-\mu)')=E(XX')=V<math>
where the transpose of any matrix A is denoted A′.
Further suppose X1, ..., Xn are independent and identically distributed. Then the Wishart distribution is the probability distribution of the p×p random matrix
- <math>S=\sum_{i=1}^n X_i X_i'.<math>
One indicates that S has that probability distribution by writing
- <math>S\sim W_p(V,n).<math>
The positive integer n is the number of degrees of freedom.
If p = 1 and V = 1 then this distribution is a chi-square distribution.
Occurrence of the Wishart distribution
The Wishart distribution arises frequently in likelihood-ratio tests in multivariate statistical analysis.
Probability density function
The Wishart distribution can be characterized by its probability density function, as follows.
Let <math>{\mathbf W}<math> be a <math>p\times p<math> symmetric matrix of random variables that is positive definite. Let <math>{\mathbf V}<math> be a (fixed) positive definite matrix of size <math>p\times p<math>.
Then, if <math>m\geq p<math>, <math>{\mathbf W}<math> has a Wishart distribution with <math>m<math> degrees of freedom if it has a probability density function <math>f_{\mathbf W}<math> given by
- <math>
f_{\mathbf W}(w)= \frac{
\left|w\right|^{(m-p-1)/2}
\exp\left[ - {\rm trace}({\mathbf V}^{-1}w/2 )\right] )
}{ 2^{mp/2}\left|{\mathbf V}\right|^{m/2}\Gamma_p(m/2) } <math>
where <math>\Gamma_p(\cdot)<math> is the multivariate gamma function defined as
- <math>
\Gamma_p(m/2)= \pi^{p(p-1)/4}\Pi_{j=1}^p \Gamma\left[ (m+1-j)/2\right]. <math>
Theorem
If <math>{\mathbf W}<math> has a Wishart distribution with <math>m<math> degrees of freedom and variance matrix <math>{\mathbf V}<math>---write <math>{\mathbf W}\sim{\mathbf W}_p(m,{\mathbf V})<math>---and <math>{\mathbf C}<math> is a <math>q\times p<math> matrix of rank <math>q<math>, then
- <math>
{\mathbf C}{\mathbf W}{\mathbf C'} \sim {\mathbf W}_q\left(m,{\mathbf C}{\mathbf V}{\mathbf C'}\right) <math>
Corollary 1
If <math>{\mathbf z}<math> is a nonzero <math>p\times 1<math> constant vector, then <math>{\mathbf z'}{\mathbf W}{\mathbf z}\sim\sigma_z^2\chi_m^2<math>.
In this case, <math>\chi_m^2<math> is the chi-square distribution and <math>\sigma_z^2={\mathbf z'}{\mathbf V}{\mathbf z}<math> (note that <math>\sigma_z^2<math> is a constant; it is positive because <math>{\mathbf V}<math> is positive definite).
Corollary 2
Consider the case where <math>{\mathbf z'}=(0,\ldots,0,1,0,\ldots,0)<math> (that is, the j-th element is one and all others zero). Then corollary 1 above shows that
- <math>
w_{jj}\sim\sigma_{jj}\chi^2_m<math>
Noted statistician George Seber points out that the Wishart distribution is not called the "multivariate chi-square distribution" because the marginal distribution of the off-diagonal elements is not chi-square. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.
Estimator of the multivariate normal distribution
The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. The derivation of the MLE is perhaps surprisingly subtle and elegant. It involves the spectral theorem and the reason why it can be better to view a scalar as the trace of a 1×1 matrix than as a mere scalar. See estimation of covariance matrices.