The mean and the standard deviation a normal distribution
In this posting, we will compute the mean and the standard deviation of the following probability distribution function (p.d.f.):
f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},
where m, s are constants and s > 0. It is obvious f_{m,s}(x) \geq 0 because the exponentiation always gives us a positive value. However, to show f_{m,s}(x) is a p.d.f., we also need to check that
\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,
and this is not so obvious.
Nevertheless, we have checked the above equality, using the following fact and the substitution method (a.k.a. change of variables):
Fact. We have
\int_{-\infty}^{\infty}e^{-x^{2}}dx = \sqrt{\pi}.
Now, before we compute the mean and the standard deviation of the p.d.f. f_{m,s}(x), let us define what these words mean:
Definition. Given any p.d.f. f, its mean is
\mu(f) := \int_{-\infty}^{\infty}x f(x) dx.
The standard deviation of f is defined by
\sigma(f) := \left(\int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx\right)^{1/2}.
Optional remark. Here, we are assuming that f(x) is continuous possibly except at finitely many places. This assumption ensures that the improper integrals converge.
Theorem 1. We always have
\sigma(f)^{2} = \int_{-\infty}^{\infty}x^{2}f(x)dx - \mu(f)^{2}.
Proof. By definitions of the standard deviation and the mean of f, we have
\begin{align*} \sigma(f)^{2} &= \int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu(f)x + \mu(f)^{2})f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\int_{-\infty}^{\infty}xf(x)dx + \mu(f)^{2}\int_{-\infty}^{\infty}f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\mu(f) + \mu(f)^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - \mu(f)^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition
\int_{-\infty}^{\infty}f(x)dx = 1,
which follows from that f is a p.d.f. This finishes the proof.
Theorem 2. We have \mu(f_{m,s}) = m.
Proof. By definition of the mean, we have
\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.
Consider the substitution
u = \frac{x - m}{s\sqrt{2}},
which comes with
du = \frac{dx}{s\sqrt{2}}.
Moreover, note that x = s\sqrt{2} \cdot u + m. Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,
which is a property for any odd function. The last equality uses Fact, stated above. This finishes the proof.
Proof. By definition of the standard deviation and using Theorem 2 which says \mu(f_{m,s}) = m, we have
\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.
Consider the substitution
t = \frac{x - m}{s\sqrt{2}},
which comes with
dt = \frac{dx}{s\sqrt{2}}.
Moreover, note that x - m = s\sqrt{2} \cdot t. Hence, following the computation we have made above, we get
\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{s^{2}}{\sqrt{\pi}}\int_{-\infty}^{\infty}t^{2}e^{-t^{2}}dt = \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about t \geq 0, we can use the substitution
u = t^{2},
which comes with du = 2tdt. Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},
where m, s are constants and s > 0. It is obvious f_{m,s}(x) \geq 0 because the exponentiation always gives us a positive value. However, to show f_{m,s}(x) is a p.d.f., we also need to check that
\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,
and this is not so obvious.
Nevertheless, we have checked the above equality, using the following fact and the substitution method (a.k.a. change of variables):
Fact. We have
\int_{-\infty}^{\infty}e^{-x^{2}}dx = \sqrt{\pi}.
Now, before we compute the mean and the standard deviation of the p.d.f. f_{m,s}(x), let us define what these words mean:
Definition. Given any p.d.f. f, its mean is
\mu(f) := \int_{-\infty}^{\infty}x f(x) dx.
The standard deviation of f is defined by
\sigma(f) := \left(\int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx\right)^{1/2}.
Optional remark. Here, we are assuming that f(x) is continuous possibly except at finitely many places. This assumption ensures that the improper integrals converge.
Theorem 1. We always have
\sigma(f)^{2} = \int_{-\infty}^{\infty}x^{2}f(x)dx - \mu(f)^{2}.
Proof. By definitions of the standard deviation and the mean of f, we have
\begin{align*} \sigma(f)^{2} &= \int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu(f)x + \mu(f)^{2})f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\int_{-\infty}^{\infty}xf(x)dx + \mu(f)^{2}\int_{-\infty}^{\infty}f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\mu(f) + \mu(f)^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - \mu(f)^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition
\int_{-\infty}^{\infty}f(x)dx = 1,
which follows from that f is a p.d.f. This finishes the proof.
Theorem 2. We have \mu(f_{m,s}) = m.
Proof. By definition of the mean, we have
\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.
Consider the substitution
u = \frac{x - m}{s\sqrt{2}},
which comes with
du = \frac{dx}{s\sqrt{2}}.
Moreover, note that x = s\sqrt{2} \cdot u + m. Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,
which is a property for any odd function. The last equality uses Fact, stated above. This finishes the proof.
The following turned out to be more difficult that we thought:
Theorem 3. We have \sigma(f_{m,s}) = s.
\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.
Consider the substitution
t = \frac{x - m}{s\sqrt{2}},
which comes with
dt = \frac{dx}{s\sqrt{2}}.
Moreover, note that x - m = s\sqrt{2} \cdot t. Hence, following the computation we have made above, we get
\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{s^{2}}{\sqrt{\pi}}\int_{-\infty}^{\infty}t^{2}e^{-t^{2}}dt = \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about t \geq 0, we can use the substitution
u = t^{2},
which comes with du = 2tdt. Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
where
\Gamma(z) = \int_{0}^{\infty}u^{z-1}e^{-u}du.
This is called the Gamma function, which you can read about here. One of the properties in the link says that we have the duplication formula:
\Gamma(z)\Gamma(z + 1/2) = 2^{1 - 2z} \sqrt{\pi}\Gamma(2z).
It is also not difficult to figure out
\Gamma(z + 1) = z \Gamma(z)
and \Gamma(1) = 1. Hence taking z = 1 in the duplication formula, we get
\Gamma(3/2) = 2^{-1}\sqrt{\pi}\Gamma(2) = \sqrt{\pi}\Gamma(1) = \sqrt{\pi}.
Combining with the previous computation, we have
\sigma(f_{m,s})^{2} = \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2) = \frac{s^{2}}{\sqrt{\pi}}\sqrt{\pi} = s^{2}.
This implies that \sigma(f_{m,s}) = s, as desired.
Comments
Post a Comment