本文爲原創文章轉載請註明出處，博主博客地址：http://blog.csdn.net/qq_20259459 和
作者（ [email protected] ）信息。
（如有需要可以郵件聯繫我）
上學期神經網最後的project有關CNN的部分我們學習使用了MatConvNet。
從名字我們就知道，Matlab Convolution Neural Network。所以就不做過多介紹了。
正文：
MatConvNet 本身是用Matlab去編譯運行C++文件，所以我們需要搭建相關連接。
這裏我用到的是Matlab2016a和VS2015兩個平臺去實現的。這裏我極力推薦VS2015版本和Matlab2013以後的版本。
因爲我當時嘗試了VS2017不可以，VS2010可以但是必須要按照網上的相關步驟去一步一步的按照順序的安裝SDK，相當的麻煩。所以我強烈推薦大家用VS2015.因爲只需要在安裝的時候選上開發者工具包並一起安裝就可以很簡單的進行二者的搭建。
配置方法：
1. 添加MatConvNet內的相關文件夾至Matlab路徑中。
2. 輸入：mex -setup cpp 等待系統配置。
3. 配置成功之後，輸入：vl_compilenn進行搭建相關文件。
如下圖：
第二步，我們就可以利用MatConvNet自帶的數據集進行訓練了，不過我們當時並沒有使用自帶的cnn_mnist_init.m文件來做訓練，我的請看下面代碼。
下面是我的CNN的編程：
function cnn_mnist_NNclass(varargin)
warning off
% CNN_MNIST Demonstrated MatConNet on MNIST
% run( fullfile(fileparts(mfilename('fullpath')), '../matlab/vl_setupnn.m') ) ;
run('C:\Users\matconvnet-1.0-beta23\matconvnet-1.0-beta23\matlab/vl_setupnn.m') ;
opts.dataDir = 'data/mnist' ;
opts.expDir = 'data/mnist-baseline' ;
opts.imdbPath = fullfile(opts.expDir, 'imdb.mat');
opts.train.batchSize = 100 ;
opts.train.numEpochs = 100 ;
opts.train.continue = true ;
% opts.train.useGpu = [] ;
opts.train.gpus = [];
opts.train.learningRate = 0.001 ;
opts.train.expDir = opts.expDir ;
opts = vl_argparse(opts, varargin);
opts.train.subsetSize = 1e4; % statsogk
% --------------------------------------------------------------------
% Prepare data
% --------------------------------------------------------------------
if exist(opts.imdbPath)
imdb = load(opts.imdbPath) ;
else
imdb = getMnistImdb(opts) ;
mkdir(opts.expDir) ;
save(opts.imdbPath, '-struct', 'imdb') ;
end
% Use a subset of the images for faster training.
if opts.train.subsetSize > 0
imdb = getSubset(imdb,opts);
end
% Define a network similar to LeNet
f=1/100 ;
net.layers = {} ;
net.layers{end+1} = struct('type', 'conv', ...
'filters', f*randn(5,5,1,20, 'single'), ...
'biases', zeros(1, 20, 'single'), ...
'stride', 1, ...
'pad', 0) ;
% net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'pool', ...
'method', 'max', ...
'pool', [2 2], ...
'stride', 2, ...
'pad', 0) ;
net.layers{end+1} = struct('type', 'conv', ...
'filters', f*randn(5,5,20,50, 'single'),...
'biases', zeros(1,50,'single'), ...
'stride', 1, ...
'pad', 0) ;
% net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'pool', ...
'method', 'max', ...
'pool', [2 2], ...
'stride', 2, ...
'pad', 0) ;
net.layers{end+1} = struct('type', 'conv', ...
'filters', f*randn(4,4,50,500, 'single'),...
'biases', zeros(1,500,'single'), ...
'stride', 1, ...
'pad', 0) ;
net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'conv', ...
'filters', f*randn(1,1,500,10, 'single'),...
'biases', zeros(1,10,'single'), ...
'stride', 1, ...
'pad', 0) ;
net.layers{end+1} = struct('type', 'softmaxloss') ;
% --------------------------------------------------------------------
% Train
% --------------------------------------------------------------------
% Take the mean out and make GPU if needed
imdb.images.data = bsxfun(@minus, imdb.images.data, mean(imdb.images.data,4)) ;
% if opts.train.useGpu
if opts.train.gpus
imdb.images.data = gpuArray(imdb.images.data) ;
end
[ net, info ] = cnn_train(net, imdb, @getBatch, opts.train, 'val', find(imdb.images.set == 3)) ;
% --------------------------------------------------------------------
function [im, labels] = getBatch(imdb, batch)
% --------------------------------------------------------------------
im = imdb.images.data(:,:,:,batch) ;
labels = imdb.images.labels(1,batch) ;
% --------------------------------------------------------------------
function imdb = getMnistImdb(opts)
% --------------------------------------------------------------------
files = {'train-images-idx3-ubyte', ...
'train-labels-idx1-ubyte', ...
't10k-images-idx3-ubyte', ...
't10k-labels-idx1-ubyte'} ;
mkdir(opts.dataDir) ;
for i=1:4
if ~exist(fullfile(opts.dataDir, files{i}), 'file')
url = sprintf('http://yann.lecun.com/exdb/mnist/%s.gz',files{i}) ;
fprintf('downloading %s\n', url) ;
gunzip(url, opts.dataDir) ;
end
end
f=fopen(fullfile(opts.dataDir, 'train-images-idx3-ubyte'),'r') ;
x1=fread(f,inf,'uint8');
fclose(f) ;
x1=permute(reshape(x1(17:end),28,28,60e3),[2 1 3]) ;
f=fopen(fullfile(opts.dataDir, 't10k-images-idx3-ubyte'),'r') ;
x2=fread(f,inf,'uint8');
fclose(f) ;
x2=permute(reshape(x2(17:end),28,28,10e3),[2 1 3]) ;
f=fopen(fullfile(opts.dataDir, 'train-labels-idx1-ubyte'),'r') ;
y1=fread(f,inf,'uint8');
fclose(f) ;
y1=double(y1(9:end)')+1 ;
f=fopen(fullfile(opts.dataDir, 't10k-labels-idx1-ubyte'),'r') ;
y2=fread(f,inf,'uint8');
fclose(f) ;
y2=double(y2(9:end)')+1 ;
imdb.images.data = single(reshape(cat(3, x1, x2),28,28,1,[])) ;
imdb.images.labels = cat(2, y1, y2) ;
imdb.images.set = [ones(1,numel(y1)) 3*ones(1,numel(y2))] ;
imdb.meta.sets = {'train', 'val', 'test'} ;
imdb.meta.classes = arrayfun(@(x)sprintf('%d',x),0:9,'uniformoutput',false) ;
% ------------------------------------------------------------------------------
function imdb = getSubset(imdb,opts)
% ------------------------------------------------------------------------------
assert(opts.train.subsetSize <= nnz(imdb.images.set == 1),...
'Subset size is bigger than the total train set size')
inds = find(imdb.images.set == 1); % indices must be from the train set
inds = randsample(inds, length(inds)-opts.train.subsetSize );
imdb.images.labels(inds) = [];
imdb.images.set(inds) = [];
imdb.images.data(:,:,:,inds) = [];下面是我的運行結果一共爲四個（我一共構造了3個網絡外加原始網絡一共是4個）：
1. 原始CNN（LeNets）的訓練結果：
2.減去兩層（layer）之後的訓練結果：
3.增加兩層之後的訓練結果：
4.在原始網絡中添加Dropout layer之後的訓練結果：
實驗結果討論：（在我的project裏面我已經寫了相關的討論（英文），大家可以百度翻譯一下看看，我這裏就不再寫了，直接複製粘貼了。）
1. Thediscussion about the Top1 err and Top5 err.
First,you make a prediction using the CNN and obtain the predicted class multinomialdistribution (∑pclass=1∑pclass=1).
Now, inthe case of top-1 score, you check if the top class (the one having the highestprobability) is the same as the target label.
In thecase of top-5 score, you check if the target label is one of your top 5predictions (the 5 ones with the highest probabilities).
In bothcases, the top score is computed as the times a predicted label matched thetarget label, divided by the number of data-points evaluated.
Finally,when 5-CNNs are used, you first average their predictions and follow the sameprocedure for calculating the top-1 and top-5 scores.
2. Thediscussion about the differences between reduced and increased
|
|
Original LeNets |
Reduced |
Increased |
Dropout |
|
Object |
0.082 |
0.071 |
0.062 |
0.066 |
|
Top1 err |
0.021 |
0.019 |
0.018 |
0.018 |
|
Top5 err |
0.000 |
0.000 |
0.000 |
0.000 |
Generallyif we use more layers, it becomes more complex and more prone to overfitting.To avoid that kind of problem when increasing the layers, I add relu layers.And the result shows that it performs than the original one. But if I addlayers consisting of additional conv-pool , then it would cause overfitting anderror will go up.
Alsowhen decreasing two layers , I got less error than the original one because itreduces the complexity. More complexity means more extract details from thefeature. Through erasing some layers, network becomes nice circumstance andperforms better than the original one.
3. Thediscussion about Dropout Layer.
A dropout layerrandomly sets a layer's input elements to zero with a given probability.
This corresponds to temporarily droppinga randomly chosen unit and all of its connections from the network duringtraining. So, for each new input element, the software randomly selects asubset of neurons, hence forms a different layer architecture. Thesearchitectures use common weights, but because the learning does not depend onspecific neurons and connections, the dropout layer might help preventoverfitting
4. DROPOUT FOR REGULARIZATION
Hinton,Srivastava, Krizhevsky, Sutskever, & Salakhutdinov (2012) introduced thedropout regularization algorithm. Although dropout works in a different waythan L1 and L2, it accomplishes the same goal—the prevention of overfitting.However, the algorithm goes about the task by actually removing neurons and connections—atleast temporarily. Unlike L1 and L2, no weight penalty is added. Dropout doesnot directly seek to train small weights. Most neural network frameworksimplement dropout as a separate layer. Dropout layers function as a regular,densely connected neural network layer. The only difference is that the dropoutlayers will periodically drop some of their neurons during training. You canuse dropout layers on regular feedforward neural networks. Figure 6 showsdropout in action.
本文爲原創文章轉載請註明出處，博主博客地址：http://blog.csdn.net/qq_20259459 和
作者（ [email protected] ）信息。