最小二乘法-線性擬合

1、概述

最小二乘法是曲線擬合的常用方法，使用該方法對匹配函數的選取非常重要。

2、理論

• 假設擬合的多項式爲:

$y = a_0 + a_1*x +a_2*x^2 + ...+a_m*x^m$
其中，m代表多項式的階數。

• 離散點與該多項式的平方和$F(a_0,a_1,,a_m)$爲，其中n代表採樣點數：
  $F(a_0,a_1,,a_m) = \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)]^2$
• 最小二乘法的思想是求平方和函數$F(a_0,a_1,,a_m)$的最小值，而對於二次方程求最小值的問題，常見的思路就是對方程求導，倒數爲零的點，及爲方程的極值點。
• 下面對$F(a_0,a_1,,a_m)$分別求$a_i$偏導數，得：
  $-2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] = 0 \\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i= 0\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^2= 0\\ \ldots\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^m= 0$
• 整理得
  $a_0\sum_{i=0}^{n}+a_1\sum_{i=0}^{n} x_i+ a_2\sum_{i=0}^{n} x_i^2+...+a_m\sum_{i=0}^{n}x_i^m = \sum_{i=0}^{n}y_i\\ a_0\sum_{i=0}^{n} x_i+a_1\sum_{i=0}^{n} x_i^2+ a_2\sum_{i=0}^{n} x_i^3+...+a_m\sum_{i=0}^{n}x_i^{m+1} = \sum_{i=0}^{n}y_ix_i\\ \ldots\\ a_0\sum_{i=0}^{n} x_i^m+a_1\sum_{i=0}^{n} x_i^{m+1}+ a_2\sum_{i=0}^{n} x_i^{m+2}+...+a_m\sum_{i=0}^{n}x_i^{2m} = \sum_{i=0}^{n}y_ix_i^m$
• 使用矩陣形式表示爲：
  $\left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\ldots&\sum_{i=0}^{n}x_i^m\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\ldots&\sum_{i=0}^{n}x_i^{m+1}\\ \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\sum_{i=0}^{n}x_i^4&\ldots&\sum_{i=0}^{n}x_i^{m+2}\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \sum_{i=0}^{n}x_i^{m}&\sum_{i=0}^{n}x_i^{m+1}&\sum_{i=0}^{n}x_i^{m+2}&\ldots&\sum_{i=0}^{n}x_i^{2m} \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1\\ a_2\\ \ldots\\ a_m \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i\\ \sum_{i=0}^{n}y_ix_i^2\\ \ldots\\ \sum_{i=0}^{n}y_ix_i^m\\ \end{matrix} \right]$
• 下面就是求解一次線性方程，常用的方法是使用高斯消元法，也可以使用求矩陣的秩或者求解逆矩陣等方式求解。

3、高斯消元法求解線性方程

3.1、原則

• 兩個方程互換解不變
• 一個方程乘以非零K,解不變
• 一個方程乘以非零K,加上另一個方程解不變

3.2、一階線性函數

假設多項式得階數m爲2，則上述矩陣方程爲：
$\left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2& \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1 \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i \end{matrix} \right]$
寫成行列式的形式爲：
$\begin{vmatrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}x_i & \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}y_ix_i\\ \end{vmatrix}$
第一行乘以係數$\sum_{i=0}^{n}$，第二行乘以係數$\sum_{i=0}^{n}x_i$

$\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 1 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} \end{vmatrix}$
第二行減去第一行得
$\begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 0 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} \end{vmatrix}$
通過上式可得$a_1$爲
$a_1 = \frac{\frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}}{\frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}}} =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
同理將$a_1$回代入得：
$a_0 = \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} - \frac{(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}y_i -(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n}\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
故最終可得
$a_0 =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$
$a_1 =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i}$