第五章 神經網絡

第五章 神經網絡

神經元模型

神經網絡是由具有適應性的簡單單元組成的廣泛並行互連的網絡，它的組織能夠模擬生物神經系統對真實世界物體所作出的交互反應

M-P神經元模型思路

將生物神經網絡中的神經元抽象後得到經典的“M-P神經元模型”。在這個模型中，神經元接收來至n個其他神經元傳遞過來的輸入信號 ，這些信號通過帶權重的連接進行傳遞 ，神經元接收到總到輸入值，將與神經元的閾值進行比較 ，然後通過激活函數處理以產生神經元的輸出

圖示如下

感知機（Perceptron）

感知機被視爲一種最簡單形式的前饋神經網絡，是一種二元線性分類器。

Wikipedia: In machine learning, the perceptron is an algorithm for supervised learning of binary classifiers: functions that can decide whether an input (represented by a vector of numbers) belongs to one class or another.

Definition：In the modern sense, the perceptron is an algorithm for learning a binary classifier: a function that maps its input x (a real-valued vector) to an output value f(x) (a single binary value):

f(x)={10if w·x+b>0otherwise

簡單的說，其實就是一個超平面將樣本空間一分爲二

神經網絡模型

基本構成

常見的神經網絡是由感知機構成的層級結構，每層神經元和下層神經元全互連，神經元之間不存在同層連接，也不存在跨層連接，這樣的神經網絡通常稱爲多層前饋神經網絡。

模型訓練

神經網絡的學習過程，就是根據訓練數據來調整神經元之間的連接權以及每個功能神經元的閾值

誤差逆傳播算法（Back Propagation of Error）

算法思想概覽

Backward Prorogation of Errors, often abbreviated as BackProp is one of the several ways in which an artificial neural network (ANN) can be trained. It is a supervised training scheme, which means, it learns from labeled training data (there is a supervisor, to guide its learning).

To put in simple terms, BackProp is like “learning from mistakes”. The supervisor corrects the ANN whenever it makes mistakes.

An ANN consists of nodes in different layers; input layer, intermediate hidden layer(s) and the output layer. The connections between nodes of adjacent layers have “weights” associated with them. The goal of learning is to assign correct weights for these edges. Given an input vector, these weights determine what the output vector is.

In supervised learning, the training set is labeled. This means, for some given inputs, we know(label) the desired/expected output.

BackProp Algorithm:
Initially all the edge weights are randomly assigned. For every input in the training dataset, the ANN is activated and its output is observed. This output is compared with the desired output that we already know, and the error is “propagated” back to the previous layer. This error is noted and the weights are “adjusted” accordingly. This process is repeated until the output error is below a predetermined threshold.

Once the above algorithm terminates, we have a “learned” ANN which, we consider is ready to work with “new” inputs. This ANN is said to have learned from several examples (labeled data) and from its mistakes (error propagation).

摘至：（Quora）How do you explain back propagation algorithm to a beginner in neural network?

BP算法基於梯度下降策略，以目標函數的負梯度方向對參數進行調整

梯度下降補充說明

梯度下降法也稱最速下降法是以負梯度方向作爲下降方向對極小化算法，也是無約束最優化中最簡單的方法

下面給出負梯度方向是下降最快的方向的證明過程

假定f(x) 在xk 附近連續可微，於是有

f(x)=f(xk)+gTk(x−xk)+o(||x−xk||) 其中 gk=▽f(xk)≠0

記x−xk＝αdk ，α 是步長因子，dk 爲下降方向，於是得到

f(xk+αdk)=f(xk)+αgTkdk+o(||αdk||)

f(xk+αdk)－f(xk)＝αgTkdk+o(||αdk||)

想要下降最快，也即最小化 gTkdk 利，用C−S 可得

｜gTkd｜≤||gk||·||dk||

顯然，當dk=−gk 時gTkdk 取到最小

BP算法流程

輸入：訓練集合D={(xk,yk)}mk=1 ,學習率η

過程：

1. 在(0,1)範圍內隨機初始化網絡中所有權值和閾值
2. repeat
3. ​ for all (xk,yk)∈D do
4. ​ 計算當前樣本的輸出ŷ k
5. ​ 計算輸出層神經元的梯度項
6. ​ 計算隱層神經元的梯度項
7. ​ 更新連接的權值和閾值
8. ​ end for
9. until 達到停止條件

輸出：由連接權和閾值確定的多層前饋神經網絡