The mean and the standard deviation a normal distribution
In this posting, we will compute the mean and the standard deviation of the following probability distribution function (p.d.f.):
\[f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},\]
where $m, s$ are constants and $s > 0.$ It is obvious $f_{m,s}(x) \geq 0$ because the exponentiation always gives us a positive value. However, to show $f_{m,s}(x)$ is a p.d.f., we also need to check that
\[\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,\]
and this is not so obvious.
Nevertheless, we have checked the above equality, using the following fact and the substitution method (a.k.a. change of variables):
Fact. We have
\[\int_{-\infty}^{\infty}e^{-x^{2}}dx = \sqrt{\pi}.\]
Now, before we compute the mean and the standard deviation of the p.d.f. $f_{m,s}(x),$ let us define what these words mean:
Definition. Given any p.d.f. $f,$ its mean is
\[\mu(f) := \int_{-\infty}^{\infty}x f(x) dx.\]
The standard deviation of $f$ is defined by
\[\sigma(f) := \left(\int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx\right)^{1/2}.\]
Optional remark. Here, we are assuming that $f(x)$ is continuous possibly except at finitely many places. This assumption ensures that the improper integrals converge.
Theorem 1. We always have
\[\sigma(f)^{2} = \int_{-\infty}^{\infty}x^{2}f(x)dx - \mu(f)^{2}.\]
Proof. By definitions of the standard deviation and the mean of $f$, we have
\begin{align*} \sigma(f)^{2} &= \int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu(f)x + \mu(f)^{2})f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\int_{-\infty}^{\infty}xf(x)dx + \mu(f)^{2}\int_{-\infty}^{\infty}f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\mu(f) + \mu(f)^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - \mu(f)^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition
\[\int_{-\infty}^{\infty}f(x)dx = 1,\]
which follows from that $f$ is a p.d.f. This finishes the proof.
Theorem 2. We have $\mu(f_{m,s}) = m.$
Proof. By definition of the mean, we have
\[\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[u = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[du = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x = s\sqrt{2} \cdot u + m.$ Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\[\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,\]
which is a property for any odd function. The last equality uses Fact, stated above. This finishes the proof.
Proof. By definition of the standard deviation and using Theorem 2 which says $\mu(f_{m,s}) = m$, we have
\[\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[t = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[dt = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x - m = s\sqrt{2} \cdot t.$ Hence, following the computation we have made above, we get
\[\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{s^{2}}{\sqrt{\pi}}\int_{-\infty}^{\infty}t^{2}e^{-t^{2}}dt = \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,\]
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about $t \geq 0,$ we can use the substitution
\[u = t^{2},\]
which comes with $du = 2tdt.$ Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
\[f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},\]
where $m, s$ are constants and $s > 0.$ It is obvious $f_{m,s}(x) \geq 0$ because the exponentiation always gives us a positive value. However, to show $f_{m,s}(x)$ is a p.d.f., we also need to check that
\[\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,\]
and this is not so obvious.
Nevertheless, we have checked the above equality, using the following fact and the substitution method (a.k.a. change of variables):
Fact. We have
\[\int_{-\infty}^{\infty}e^{-x^{2}}dx = \sqrt{\pi}.\]
Now, before we compute the mean and the standard deviation of the p.d.f. $f_{m,s}(x),$ let us define what these words mean:
Definition. Given any p.d.f. $f,$ its mean is
\[\mu(f) := \int_{-\infty}^{\infty}x f(x) dx.\]
The standard deviation of $f$ is defined by
\[\sigma(f) := \left(\int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx\right)^{1/2}.\]
Optional remark. Here, we are assuming that $f(x)$ is continuous possibly except at finitely many places. This assumption ensures that the improper integrals converge.
Theorem 1. We always have
\[\sigma(f)^{2} = \int_{-\infty}^{\infty}x^{2}f(x)dx - \mu(f)^{2}.\]
Proof. By definitions of the standard deviation and the mean of $f$, we have
\begin{align*} \sigma(f)^{2} &= \int_{-\infty}^{\infty}(x - \mu(f))^{2}f(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu(f)x + \mu(f)^{2})f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\int_{-\infty}^{\infty}xf(x)dx + \mu(f)^{2}\int_{-\infty}^{\infty}f(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - 2\mu(f)\mu(f) + \mu(f)^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}f(x) dx - \mu(f)^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition
\[\int_{-\infty}^{\infty}f(x)dx = 1,\]
which follows from that $f$ is a p.d.f. This finishes the proof.
Theorem 2. We have $\mu(f_{m,s}) = m.$
Proof. By definition of the mean, we have
\[\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[u = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[du = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x = s\sqrt{2} \cdot u + m.$ Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\[\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,\]
which is a property for any odd function. The last equality uses Fact, stated above. This finishes the proof.
The following turned out to be more difficult that we thought:
Theorem 3. We have $\sigma(f_{m,s}) = s.$
\[\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[t = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[dt = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x - m = s\sqrt{2} \cdot t.$ Hence, following the computation we have made above, we get
\[\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{s^{2}}{\sqrt{\pi}}\int_{-\infty}^{\infty}t^{2}e^{-t^{2}}dt = \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,\]
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about $t \geq 0,$ we can use the substitution
\[u = t^{2},\]
which comes with $du = 2tdt.$ Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
where
\[\Gamma(z) = \int_{0}^{\infty}u^{z-1}e^{-u}du.\]
This is called the Gamma function, which you can read about here. One of the properties in the link says that we have the duplication formula:
\[\Gamma(z)\Gamma(z + 1/2) = 2^{1 - 2z} \sqrt{\pi}\Gamma(2z).\]
It is also not difficult to figure out
\[\Gamma(z + 1) = z \Gamma(z)\]
and $\Gamma(1) = 1.$ Hence taking $z = 1$ in the duplication formula, we get
\[\Gamma(3/2) = 2^{-1}\sqrt{\pi}\Gamma(2) = \sqrt{\pi}\Gamma(1) = \sqrt{\pi}.\]
Combining with the previous computation, we have
\[\sigma(f_{m,s})^{2} = \frac{s^{2}}{\sqrt{\pi}}\Gamma(3/2) = \frac{s^{2}}{\sqrt{\pi}}\sqrt{\pi} = s^{2}.\]
This implies that $\sigma(f_{m,s}) = s,$ as desired.
Comments
Post a Comment