Coursera-MachineLearning-NeuralNetwork

Coursera機器學習-吳恩達
背景：判別手寫體數字，分別實現邏輯迴歸和神經網絡，並對比正確率

% 1.讀取訓練集，並隨機打印一下

clear ; close all; clc
input_layer_size  = 400;  		% 20x20 Input Images of Digits
num_labels = 10;          		% 10 labels, from 1 to 10

%讀取訓練集，是手寫體的圖像矩陣，自動將特徵放入X，預測結果放入y
load('ex3data1.mat'); 

m = size(X, 1);
% Randomly select 100 data points to display
rand_indices = randperm(m);

%圖像的矩陣值
sel = X(rand_indices(1:100), :);
displayData(sel);

% 2.邏輯迴歸向量化（Vectorize Logistic Regression）

%用一些測試數據來測試代價函數
theta_t = [-2; -1; 1; 2];

% 將 5*1 向量 和 5*3 的 0.1 至 1.5 矩陣合在一起
X_t = [ones(5,1) reshape(1:15,5,3)/10];

y_t = ([1;0;1;0;1] >= 0.5);

lambda_t = 3;
[J grad] = lrCostFunction(theta_t, X_t, y_t, lambda_t);

% 函數 lrCostFunction：
function [J, grad] = lrCostFunction(theta, X, y, lambda)
m = length(y); 

% 初始化參數
J = 0;
grad = zeros(size(theta));

% 假設函數：
hx = sigmoid(X * theta);

% 計算代價函數和梯度
J = 1 / m * sum((-y) .* log(hx) - (1 - y) .* log(1 - hx)) + lambda / (2 * m) * (sum(theta .^ 2) - (theta(1) ^ 2));
grad = 1 / m * (X' * (hx - y) + lambda * theta);

%這裏注意theta值，theta(1)是不參與的，需要把theta(1)減去
grad(1) = grad(1) - lambda / m * theta(1); 
grad = grad(:);
end

fprintf('\nCost: %f\n', J);
fprintf(' %f \n', grad);

% 3.正式開始訓練模型

% 設置 懲罰係數
lambda = 0.1;
% 計算最優theta
[all_theta] = oneVsAll(X, y, num_labels, lambda);

% 函數 oneVsAll：
function [all_theta] = oneVsAll(X, y, num_labels, lambda)
m = size(X, 1);
n = size(X, 2);

all_theta = zeros(num_labels, n + 1);

X = [ones(m, 1) X];

initial_theta = zeros(n + 1, 1);

options = optimset('GradObj', 'on', 'MaxIter', 50);

for i = 1 : num_labels

	% 設置 c 爲手寫體結果，判斷y，c是否相等
    c = i * ones(size(y));
    
    [theta] = fmincg(@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);
    all_theta(i, :) = theta;
end
end