人工神經網絡——多層感知器Delta學習

單層連續感知器網絡，做一個R類線性分類器，前向通道表示如下
$\bm o = \bm \Gamma [\bm W\bm Y]$
$\bm Y=\Gamma[\bm W\bm X]$其中$\bm W = \begin{bmatrix} w_{11} \quad w_{12} \quad \cdots \quad w_{1n}\\ w_{21} \quad w_{22} \quad \cdots \quad w_{2n}\\ \vdots \quad\quad \vdots \space \quad \ddots \quad\quad \vdots \\ w_{m1} \quad w_{m2} \quad \cdots \quad w_{mn}\\ \end{bmatrix}$非線性對角算子$\bm \Gamma[ \bm \cdot ] = \begin{bmatrix} f(\bm \cdot ) \quad\space 0 \quad\space \cdots \quad 0\\ 0 \quad\space f(\bm \cdot ) \quad \cdots \quad\space 0\\ \vdots \quad\quad \vdots \space \quad \ddots \quad\quad \vdots \\ 0 \quad\quad 0 \quad \cdots \quad f(\bm \cdot ) \\ \end{bmatrix}$

計算單個權調節
$\Delta w_{kj} = \eta \frac{\partial E}{\partial w_{kj}}$對於任意k節點$net_k = \sum_{j = 1}^{J}w_{kj}y_j$神經元輸出$o_k = f(net_k)$由第k個神經元產生的誤差信號$\delta _{o_k} = -\frac{\partial E}{\partial (net_k)}$從此我們可以寫出$\frac{\partial E}{\partial w_{kj}} = \frac{\partial E}{\partial (net_k)} \cdot \frac{\partial (net_k)}{\partial w_{kj}}$輸入處有$\frac{\partial (net_k)}{\partial w_{kj}} =y_j$我們得到$\frac{\partial E}{\partial w_{kj}} = \delta _{o_k}y_j$和$\Delta w_{kj} = \eta \delta _{o_k}y_j$
對於$\delta _{o_k}$
$\delta _{o_k} = -\frac{\partial E}{\partial o_k} \cdot \frac{\partial o_k}{\partial (net_k)}$又因爲$\frac{\partial E}{\partial o_k} = -(d_k-o_k)$$\frac{\partial o_k}{\partial (net_k)} = f^{-1}(net_k)$綜上$\Delta w_{kj} = \eta (d_k-o_k) f'(net_k)y_i$$w_{kj} = w_{kj} =\Delta w_{kj}$其中$f'(net) = o(1-o)$
沒錯。。就是上一章公式的推廣

廣義的Delta學習規則

兩層神經網絡
也人有稱其三層神經網絡（考慮是否將輸入層也算作網絡之一）
結點i向結點j的權
對於隱含層，負梯度下降公式爲
$\Delta v_{ji} = -\eta \frac{\partial E}{\partial v_{ji}}$
$\Delta w_{kj} = \eta f'(net_k)z_i \sum_{k=1}^K \delta_{o_k}w_{kj}$