Fisher information metric
|
|
In mathematics, in information geometry, the Fisher information metric is a metric tensor for a statistical differential manifold. It can be used to calculate the informational difference between measurements. It takes the form:
- <math>
g_{ij} = \int
\frac{\partial \log p(x,\theta)}{\partial \theta_i}
\frac{\partial \log p(x,\theta)}{\partial \theta_j}
p(x,\theta)
dx. <math>
Substituting <math>i = -ln(p)<math> from information theory, the formula becomes:
- <math>
g_{ij} = \int
\frac{\partial i(x,\theta)}{\partial \theta_i}
\frac{\partial i(x,\theta)}{\partial \theta_j}
p(x,\theta)
dx. <math>
Which can be thought of intuitively as: "The distance between two points on a statistical differential manifold is the amount of information between them, i.e. the informational difference between them."
An equivalent form of the above equation is:
- <math>
g_{ij} = -\int
\frac{\partial^2 i(x,\theta)}{\partial \theta_i \partial \theta_j}
p(x,\theta)
dx = -\mathrm{E} \left[
\frac{\partial^2 i(x,\theta)}{\partial \theta_i \partial \theta_j}
\right]. <math>
See also
References
- Shun'ichi Amari - Differential-geometrical methods in statistics, Lecture notes in statistics, Springer-Verlag, Berlin, 1985.
- Shun'ichi Amari, Hiroshi Nagaoka - Methods of information geometry, Transactions of mathematical monographs; v. 191, American Mathematical Society, 2000.Template:Math-stub