最小二乘法-线性拟合

1、概述

最小二乘法是曲线拟合的常用方法，使用该方法对匹配函数的选取非常重要。

2、理论

• 假设拟合的多项式为:

$y = a_0 + a_1*x +a_2*x^2 + ...+a_m*x^m$
其中，m代表多项式的阶数。

• 离散点与该多项式的平方和$F(a_0,a_1,,a_m)$为，其中n代表采样点数：
  $F(a_0,a_1,,a_m) = \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)]^2$
• 最小二乘法的思想是求平方和函数$F(a_0,a_1,,a_m)$的最小值，而对于二次方程求最小值的问题，常见的思路就是对方程求导，倒数为零的点，及为方程的极值点。
• 下面对$F(a_0,a_1,,a_m)$分别求$a_i$偏导数，得：
  $-2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] = 0 \\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i= 0\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^2= 0\\ \ldots\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^m= 0$
• 整理得
  $a_0\sum_{i=0}^{n}+a_1\sum_{i=0}^{n} x_i+ a_2\sum_{i=0}^{n} x_i^2+...+a_m\sum_{i=0}^{n}x_i^m = \sum_{i=0}^{n}y_i\\ a_0\sum_{i=0}^{n} x_i+a_1\sum_{i=0}^{n} x_i^2+ a_2\sum_{i=0}^{n} x_i^3+...+a_m\sum_{i=0}^{n}x_i^{m+1} = \sum_{i=0}^{n}y_ix_i\\ \ldots\\ a_0\sum_{i=0}^{n} x_i^m+a_1\sum_{i=0}^{n} x_i^{m+1}+ a_2\sum_{i=0}^{n} x_i^{m+2}+...+a_m\sum_{i=0}^{n}x_i^{2m} = \sum_{i=0}^{n}y_ix_i^m$
• 使用矩阵形式表示为：
  $\left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\ldots&\sum_{i=0}^{n}x_i^m\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\ldots&\sum_{i=0}^{n}x_i^{m+1}\\ \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\sum_{i=0}^{n}x_i^4&\ldots&\sum_{i=0}^{n}x_i^{m+2}\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \sum_{i=0}^{n}x_i^{m}&\sum_{i=0}^{n}x_i^{m+1}&\sum_{i=0}^{n}x_i^{m+2}&\ldots&\sum_{i=0}^{n}x_i^{2m} \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1\\ a_2\\ \ldots\\ a_m \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i\\ \sum_{i=0}^{n}y_ix_i^2\\ \ldots\\ \sum_{i=0}^{n}y_ix_i^m\\ \end{matrix} \right]$
• 下面就是求解一次线性方程，常用的方法是使用高斯消元法，也可以使用求矩阵的秩或者求解逆矩阵等方式求解。

3、高斯消元法求解线性方程

3.1、原则

• 两个方程互换解不变
• 一个方程乘以非零K,解不变
• 一个方程乘以非零K,加上另一个方程解不变

3.2、一阶线性函数

假设多项式得阶数m为2，则上述矩阵方程为：
$\left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2& \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1 \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i \end{matrix} \right]$
写成行列式的形式为：
$\begin{vmatrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}x_i & \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}y_ix_i\\ \end{vmatrix}$
第一行乘以系数$\sum_{i=0}^{n}$，第二行乘以系数$\sum_{i=0}^{n}x_i$

$\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 1 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} \end{vmatrix}$
第二行减去第一行得
$\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 0 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} \end{vmatrix}$
通过上式可得$a_1$为
$a_1 = \frac{\frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}}{\frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}}} =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
同理将$a_1$回代入得：
$a_0 = \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} - \frac{(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}y_i -(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n}\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
故最终可得
$a_0 =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
$a_1 =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$