1.1 機器學習的定義：
Arthur Samuel : " Field of study that gives computers the ability to learn without being explicitly programmed."
Tom Mitchell : " A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P. improves with experiience E."
1.2 監督學習：
利用一組帶標籤的數據, 學習從輸入到輸出的映射, 然後將這種映射關係應用到未知數據, 達到分類或者回歸的目的。
Regression Problem（迴歸問題）：設法預測取值連續的數值輸出（房價預測）
1.3 無監督學習：
利用無標籤的數據學習數據的分佈或數據與數據之間的關係被稱爲無監督學習。
無監督學習最常應用的場景是聚類(clustering)和降維(Dimension Reduction)
有監督學習和無監督學習的最大區別在於數據是否有標籤
2.1模型描述
專用的術語以及假設函數h：h θ ( x ) h_\theta(x) h θ ( x ) h ( x ) h(x) h ( x ) h θ ( x ) = θ 0 + θ 1 x h_\theta(x)=\theta_0+\theta_1x h θ ( x ) = θ 0 + θ 1 x θ 0 \theta_0 θ 0 θ 1 \theta_1 θ 1 2.2代價函數（cost function）
常用：平方誤差代價函數。J ( θ 0 , θ 1 ) J(\theta_0,\theta_1) J ( θ 0 , θ 1 )
Hypothesis function : h θ ( x ) = θ 0 + θ 1 x h_\theta(x)=\theta_0+\theta_1x h θ ( x ) = θ 0 + θ 1 x Parameters : θ 0 、 θ 1 \theta_0 、\theta_1 θ 0 、 θ 1 Cost function : J ( θ 0 , θ 1 ) = 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 J(\theta_0,\theta_1)=\frac{1}{2m}\sum_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2 J ( θ 0 , θ 1 ) = 2 m 1 i = 1 ∑ m ( h θ ( x ( i ) ) − y ( i ) ) 2 Goal ：min J ( θ 0 , θ 1 ) = min 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 \min J(\theta_0,\theta_1)=\min \frac{1}{2m}\sum_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2 min J ( θ 0 , θ 1 ) = min 2 m 1 i = 1 ∑ m ( h θ ( x ( i ) ) − y ( i ) ) 2 θ 0 、 θ 1 \theta_0、\theta_1 θ 0 、 θ 1 θ 0 、 θ 1 \theta_0、\theta_1 θ 0 、 θ 1 h θ ( x ) h_\theta(x) h θ ( x )
若 θ 0 = 0 \theta_0=0 θ 0 = 0 h θ ( x ) = θ 1 x h_\theta(x)=\theta_1x h θ ( x ) = θ 1 x
則根據不同的θ 1 \theta_1 θ 1 h θ ( x ) h_\theta(x) h θ ( x ) J ( θ 1 ) J(\theta_1) J ( θ 1 ) θ 1 \theta_1 θ 1 J ( θ 1 ) J(\theta_1) J ( θ 1 ) ——凸函數 θ 0 ≠ 0 \theta_0\neq0 θ 0 ̸ = 0 h θ ( x ) = θ 0 + θ 1 x h_\theta(x)=\theta_0+\theta_1x h θ ( x ) = θ 0 + θ 1 x 梯度下降算法 可以解決這些問題
2.3 梯度下降算法 （Gradient decent）
α \alpha α α \alpha α α \alpha α J ( θ ) J(\theta) J ( θ )
當到達局部最優點時，導數部分爲0。2.4 線性迴歸算法 ——第一個機器學習算法
思想：將梯度下降算法應用到最小化平方差代價函數
5.1 多元梯度下降算法 x j x_j x j x ( i ) x^{(i)} x ( i ) y ( i ) y^{(i)} y ( i ) x j ( i ) {\rm{x}}_j^{(i)} x j ( i ) h θ ( x ) = θ 0 + θ 1 x 1 + θ 2 x 2 + … + θ n x n {h_\theta }(x) = {\theta _0} + {\theta _1}{x_1} + {\theta _2}{x_2} + \ldots + {\theta _n}{x_n} h θ ( x ) = θ 0 + θ 1 x 1 + θ 2 x 2 + … + θ n x n x 0 = 1 x_0=1 x 0 = 1 x = [ x 0 , x 1 … x n ] T x = {\left[ {{x_0},{x_{\rm{1}}} \ldots {x_{\rm{n}}}} \right]^T} x = [ x 0 , x 1 … x n ] T θ = [ θ 0 , θ 1 … θ n ] T \theta = {\left[ {{\theta _0},{\theta _{\rm{1}}} \ldots {\theta _{\rm{n}}}} \right]^T} θ = [ θ 0 , θ 1 … θ n ] T h θ ( x ) = θ T x {h_\theta }(x) = {\theta ^T}x h θ ( x ) = θ T x J ( θ ) = 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 J(\theta)=\frac{1}{2m}\sum_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2 J ( θ ) = 2 m 1 i = 1 ∑ m ( h θ ( x ( i ) ) − y ( i ) ) 2 5.2特徵縮放 x j x_j x j 5.3學習率 α \alpha α α \alpha α α \alpha α J ( θ ) J(\theta) J ( θ )
5.4正規方程
如前所述：假設函數 ：h θ ( x ) = θ 0 + θ 1 x 1 + θ 2 x 2 + … + θ n x n {h_\theta }(x) = {\theta _0} + {\theta _1}{x_1} + {\theta _2}{x_2} + \ldots + {\theta _n}{x_n} h θ ( x ) = θ 0 + θ 1 x 1 + θ 2 x 2 + … + θ n x n x 0 = 1 x_0=1 x 0 = 1 x = [ x 0 , x 1 … x n ] T x = {\left[ {{x_0},{x_{\rm{1}}} \ldots {x_{\rm{n}}}} \right]^T} x = [ x 0 , x 1 … x n ] T θ = [ θ 0 , θ 1 … θ n ] T \theta = {\left[ {{\theta _0},{\theta _{\rm{1}}} \ldots {\theta _{\rm{n}}}} \right]^T} θ = [ θ 0 , θ 1 … θ n ] T h θ ( x ) = θ T x {h_\theta }(x) = {\theta ^T}x h θ ( x ) = θ T x
代價函數 ：J ( θ ) = 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 J(\theta)=\frac{1}{2m}\sum_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})^2 J ( θ ) = 2 m 1 i = 1 ∑ m ( h θ ( x ( i ) ) − y ( i ) ) 2 x 0 = 1 x_0=1 x 0 = 1 x ( i ) = [ x 0 ( i ) x 1 ( i ) x 2 ( i ) … x n ( i ) ] T ∈ R n + 1 {{x}^{(i)}}={{\left[ \begin{matrix}
x_{0}^{(i)} & x_{1}^{(i)} & x_{2}^{(i)} & \ldots & x_{n}^{(i)} \\
\end{matrix} \right]}^{T}}\in {{R}^{n+1}} x ( i ) = [ x 0 ( i ) x 1 ( i ) x 2 ( i ) … x n ( i ) ] T ∈ R n + 1
X = [ 1 2104 5 1 45 1 1416 3 2 40 1 1534 3 2 30 1 852 2 1 36 ] X=\left[ \begin{matrix}
1 & 2104 & 5 & 1 & 45 \\
1 & 1416 & 3 & 2 & 40 \\
1 & 1534 & 3 & 2 & 30 \\
1 & 852 & 2 & 1 & 36 \\
\end{matrix} \right] X = ⎣ ⎢ ⎢ ⎡ 1 1 1 1 2 1 0 4 1 4 1 6 1 5 3 4 8 5 2 5 3 3 2 1 2 2 1 4 5 4 0 3 0 3 6 ⎦ ⎥ ⎥ ⎤
y = [ 460 232 315 178 ] y=\left[ \begin{matrix}
460 \\
232 \\
315 \\
178 \\
\end{matrix} \right] y = ⎣ ⎢ ⎢ ⎡ 4 6 0 2 3 2 3 1 5 1 7 8 ⎦ ⎥ ⎥ ⎤
θ = [ θ 0 , θ 1 … θ n ] T \theta = {\left[ {{\theta _0},{\theta _{\rm{1}}} \ldots {\theta _{\rm{n}}}} \right]^T} θ = [ θ 0 , θ 1 … θ n ] T
因此，X θ = y X\theta =y X θ = y θ = ( X T X ) − 1 X T y \theta ={{({{X}^{T}}X)}^{-1}}{{X}^{T}}y θ = ( X T X ) − 1 X T y
P.S：若X T X X^{T}X X T X
利用梯度下降算法可求得局部最優解
