Programming Exercise 4: Neural Networks Learning

nnCostFunction

%代價函數的實現，無正則化
a1 = [ones(m, 1) X];   % 5000x401
z2 = a1*Theta1';       % 5000x25
a2 = [ones(m, 1) sigmoid(z2)]; % 5000x26
z3 = a2*Theta2';       % 5000x10
a3 = sigmoid(z3);      

y_vec = zeros(m,num_labels);  % 給出的y是標量，你得把它改成想要的向量形式
for i=1:m;
    y_vec(i,y(i)) = 1;
end

J = sum(sum(-y_vec.*log(a3) - (1-y_vec).*log(1-a3)))/m;

%添加正則化,正則化實質上就是添加所有Theta（不包括偏置單元）的平方再乘以lambda除以（2*m）
[row1, column1] = size(Theta1);
tmp1 = Theta1(:, 2:column1);

[row2, column2] = size(Theta2);
tmp2 = Theta2(:, 2:column2);

J = J + (sum(sum(tmp1.*tmp1)) + sum(sum(tmp2.*tmp2)))*lambda/2/m;


%實現Backpropagation 

%delta3 = a3 - y_vec;   %step2
%delta2 = (Theta2' * delta3').*sigmoidGradient(z2); %step3

Delta1 = zeros(size(Theta1));  
Delta2 = zeros(size(Theta2));  
for i=1:m,  
    delta3 = a3(i,:) - y_vec(i,:);  

    temp = (delta3*Theta2);
    delta2 = temp(:,2:end).*sigmoidGradient(z2(i,:));  

    Delta2 = Delta2 + delta3' * a2(i,:);  
 
    Delta1 = Delta1 + delta2' * a1(i,:);  
end;  
  
Theta2_grad = Delta2/m;  
Theta1_grad = Delta1/m;  
  
Theta2_grad(:,2:end) = Theta2_grad(:,2:end) + lambda * Theta2(:,2:end) / m;  
Theta1_grad(:,2:end) = Theta1_grad(:,2:end) + lambda * Theta1(:,2:end) / m;

randInitializeWeights

epsilon_init = 0.12; 
W = rand(L_out, 1 + L_in) * 2 * epsilon_init-epsilon_init;

sigmoidGradient

tmp = ones(size(z));
g = tmp./(1+exp(-z)) .* (1 - tmp./(1+exp(-z)));

這章主要是學習反向傳播算法。我在學習的時候一直有一個疑問，爲什麼要用反向傳播來算偏導呢，後面在知乎上找到答案。在layer和unit數量巨大的時候，用一開始的直接計算梯度值方法計算量巨大。用反向傳播計算，會大大指減少計算量。