Jensen's inequality
|
|
In mathematics, Jensen's inequality, named after the Danish mathematician Johan Jensen, relates the value of a convex function of an integral to the integral of the convex function.
| Contents |
General form
The inequality can be stated quite generally using measure theory. It can also be stated equally generally in the language of probability theory. The two statements say exactly the same thing.
In the language of measure theory
Let μ be a positive measure on a set Ω, such that μ(Ω) = 1. If g is a real-valued function that is Lebesgue integrable, and if φ is convex function on the range of g, then
- <math>\varphi\left(\int_{\Omega} g\, d\mu\right) \le \int_\Omega \varphi \circ g\, d\mu. <math>
In the language of probability theory
In the terminology of probability theory, μ is a probability measure. The function g is replaced by a real-valued random variable X (just another name for the same thing, as long as the context remains one of "pure" mathematics). The integral of any function over the space Ω with respect to the probability measure μ becomes an expected value. The inequality then says that if φ is any convex function, then
- <math>\varphi\left(E(X)\right) \leq E(\varphi(X)).\,<math>
Special cases
Form involving a probability density function
Suppose Ω is a measurable subset of the real line and f(x) is a non-negative function such that
- <math>\int_{-\infty}^\infty f(x)\,dx = 1.<math>
In probabilistic language, f is a probability density function.
Then Jensen's inequality becomes the following statement about convex integrals:
If g is any real-valued measurable function and φ is convex over the range of g, then
- <math> \varphi\left(\int_{-\infty}^\infty g(x)f(x)\, dx\right) \le \int_{-\infty}^\infty \varphi(g(x)) f(x)\, dx. <math>
If g(x) = x, then this form of the inequality reduces to a commonly used special case:
- <math>\varphi\left(\int_{-\infty}^\infty x\, f(x)\, dx\right) \le \int_{-\infty}^\infty \varphi(x)\,f(x)\, dx.<math>
Finite form
If <math>\Omega<math> is some finite set <math>\{x_1,x_2,\ldots,x_n\}<math>, and if <math>\mu<math> is a normalized counting measure on <math>\Omega<math>, then the general form reduces to a statement about sums:
- <math> \varphi\left(\sum_{i=1}^{n} g(x_i)\lambda_i \right) \le \sum_{i=1}^{n} \varphi(g(x_i))\lambda_i, <math>
provided that <math> \lambda_1 + \lambda_2 + \cdots + \lambda_n = 1, \lambda_i \ge 0. <math>
If φ is concave, each of the inequalities is simply reversed.
Suppose <math>x_1, x_2, \ldots, x_n<math> are real and positive, <math>g(x)=x<math>, <math>\lambda_i = 1/n<math> and <math>\varphi(x) = \log(x)<math>. The above sum becomes
- <math> \log\left(\sum_{i=1}^{n} \frac{x_i}{n} \right) \ge \sum_{i=1}^{n} \frac{\log(x_i)}{n},<math>
The exponential values of both sides give the familiar AM-GM inequality:
- <math> \frac{x_1 + x_2 + \cdots + x_n}{n} \ge \sqrt[n]{x_1 x_2 \cdots x_n}.<math>
There is also an infinite discrete form.
Statistical physics
Jensen's equality is of particular importance in statistical physics when the convex function is an exponential:
- <math> e^{\langle X \rangle} \leq \left\langle e^X \right\rangle, <math>
where angle brackets denote expected values with respect to some probability distribution in the random variable X. The proof in this case is very simple (cf. Chandler, Sec. 5.5); using the inequality
- <math> e^X \geq 1+X \, <math>
in the third exponential of the identity
- <math> \left\langle e^X \right\rangle
= e^{\langle X \rangle} \left\langle e^{X - \langle X \rangle} \right\rangle, <math>
the desired inequality follows directly.
University logo
Jensen's inequality serves as logo for the mathematics department of Copenhagen University.
References
External links
- Jensen's inequality on MathWorld (http://mathworld.wolfram.com/JensensInequality.html)de:Jensensche Ungleichung