最大似然估計會低估方差

根據最大似然估計可以得到方差的估計值爲：

$\sigma_{ML}^{2}=\frac{1}{N}\sum_{n=1}^{N}(x_{n}-\mu_{ML})^{2}\tag {1}$

其中$\mu_{ML}^{2}=\frac{1}{N}\sum_{i=1}^{N}x_{i}$

對$\sigma_{ML}^{2}$求期望的結果爲：

$E(\sigma_{ML}^{2})=\frac{N-1}{N}\sigma^{2}\tag {2}$

從這個結果來看，可以知道最大似然估計對實際的方差$\sigma^{2}$低估了，主要是因爲係數$\frac{N-1}{N}$

下面對這個期望結果進行證明：

將$(1)$式代入$(2)$可得，

$\begin{aligned} E(\sigma_{ML}^{2}) &= E(\frac{1}{N}\sum_{n=1}^{N}(x_{n}-\mu_{ML})^{2}) \\ & = \frac{1}{N}\sum_{n=1}^{N}E((x_{n}-\mu_{ML})^{2}) \\ & = \frac{1}{N}\sum_{n=1}^{N}E(x_{n}^{2}-2x_{n}\mu_{ML}+\mu_{ML}^{2})\\ & = \frac{1}{N}\sum_{n=1}^{N}E(x_{n}^{2})-\frac{2}{N}\sum_{n=1}^{N}E((x_{n}^{2}+\frac{1}{N}\sum_{i=1,i\neq{n}}^{N}x_{n}x_{i}))+\frac{1}{N}\sum_{n=1}^{N}\underbrace{E(\mu_{ML}^{2})}_{*}\\ & = \sigma^{2}+\mu^{2}-\frac{2}{N}\mu^{2}-\frac{2}{N}\sigma^{2}-2\frac{N-1}{N}\mu^{2}+\mu^{2}+\frac{1}{N}\sigma^{2}\\ & = \frac{N-1}{N}\sigma^{2} \end{aligned}$
其中$\mu$是數據的真實均值，$\sigma^{2}$是數據的真實方差，下方花括號括起來的*式的具體計算過程如下：
$\begin{aligned} E(\mu_{ML}^{2}) & = E^{2}(\mu_{ML})+D(\mu_{ML})\\ & = \mu^{2}+D(\frac{1}{N}\sum_{i=1}^{N}x_{i})\\ & = \mu^{2}+\frac{1}{N^{2}}D(\sum_{i=1}^{N}x_{i})\\ & = \mu^{2}+\frac{1}{N^{2}}N\sigma^{2}\\ & = \mu^{2}+\frac{1}{N}\sigma^{2} \end{aligned}$

證畢。