python實現BackPropagation算法

python實現BackPropagation算法

實現神經網絡的權重和偏置更新，很重要的一部就是使用BackPropagation（反向傳播）算法。具體來說，反向傳播算法就是用誤差的反向傳播來計算w（權重）和b（偏置）相對於目標函數的導數，這樣就可以在原來的w，b的基礎上減去偏導數來更新。其中我上次寫的python實現梯度下降中有一個函數backprop(x,y)就是用來實現反向傳播的算法。（注：代碼並非自己總結，github上有這個代碼的實現https://github.com/LCAIZJ/neural-networks-and-deep-learning）

def backprop(self,x,y):
    nabla_b = [np.zeros(b.shape) for b in self.biases]
    nabla_w = [np.zeros(w.shape) for w in self.weights]
    # 通過輸入x，前向計算輸出層的值
    activation = x
    activations = [x]# 存儲的是所以的輸出層
    zs = []
    for b,w in zip(self.biases,self.weights):
        z = np.dot(w,activation)+b
        zs.append(z)
        activation = sigmoid(z)
        activations.append(activation)
    # 計算輸出層的error
    delta = self.cost_derivative(activations[-1],y)*sigmoid_prime(zs[:-1])
    nabla_b[-1] = delta
    nabla_w[-1] = np.dot(delta,activations[-2].transpose())
    #反向更新error
    for l in xrange(2,self.num_layers):
        z = zs[-l]
        sp = sigmoid_prime(z)
        delta = np.dot(self.weight[-l+1].transpose(),delta)*sp
        nabla_b[-l] = delta
        nabla_w[-l] = np.dot(delta,activations[-l-1].transpose())
    return (nabla_b,nabla_w)

其中，傳入的x和y是一個單獨的實例。

def cost_derivative(self,output_activation,y):
    return (output_activation-y)
def sigmoid(z):
    return 1.0/(1.0+np.exp(z))
def sigmoid_prime(z):
    return sigmoid(z)*(1-sigmoid(z))