一、Assignment 1 課程作業要求

具體原版要求見鏈接http://cs231n.github.io/assignments2019/assignment1/ 給不能FQ的同志搬運下

對cifar10 數據進行圖像數據的10分類，分別有以下要求

1: k-Nearest Neighbor classifier (20 points) 求解鏈接

2: Training a Support Vector Machine (25 points) 求解鏈接

3: Implement a Softmax classifier (20 points) 求解鏈接

4: Two-Layer Neural Network (25 points) 求解鏈接

5: Higher Level Representations: Image Features (10 points)

本文僅完成Two-Layer Neural Network 分類器部分，其他部分請點擊上面相關求解鏈接

二、數據集下載及處理

基本的數據集下載，數據簡單批處理，參考https://github.com/Halfish/cs231n/blob/master/assignment1/knn.ipynb中，將下載後的cifar10數據集解壓

data_utils.py:

# -*- coding: utf-8 -*-
"""
Created on Sat Feb 29 12:57:18 2020
data_utils.py
@author: acer
"""
import pickle
import numpy as np 
import os 

def load_cifar_batch(filename): 
    with open(filename,'rb') as f : 
        datadict=pickle.load(f,encoding='bytes') 
        x=datadict[b'data'] 
        y=datadict[b'labels'] 
        x=x.reshape(10000,3,32,32).transpose(0,2,3,1).astype('float') 
        y=np.array(y) 
        #以數組形式輸出
        return x,y 
#root 爲cifar文件根目錄
def load_cifar10(root_path): 
    xs=[] 
    ys=[] 
    # 訓練集1-5
    for b in range(1,2): #這裏只選10000張作爲訓練集
        # 讀取訓練集圖片
        f=os.path.join(root_path,'data_batch_%d' % (b,)) 
        #每批讀取
        x,y=load_cifar_batch(f) 
        #累加 x y 將5批數據合併
        xs.append(x) 
        ys.append(y) 
        #數組拼接
        Xtrain=np.concatenate(xs) 
        Ytrain=np.concatenate(ys) 
    del x ,y 
    # 讀取測試集圖片
    Xtest,Ytest=load_cifar_batch(os.path.join(root_path,'test_batch'))   
    return Xtrain,Ytrain,Xtest,Ytest

建立讀取數據訓練的主文件main.py,調用函數讀取並顯示cifar10圖片

main.py:

import numpy as np
from data_utils import load_cifar10
import matplotlib.pyplot as plt

#提取數據
x_train,y_train,x_test,y_test = load_cifar10('.\cifar-10-python\cifar-10-batches-py')
x_test = x_test[0:2000,:]
y_test = y_test[0:2000]
print('training data shape:',x_train.shape) 
print('training labels shape:',y_train.shape) 
print('test data shape:',x_test.shape) 
print('test labels shape:',y_test.shape)

#載入數據
classes=['plane','car','bird','cat','deer','dog','frog','horse','ship','truck'] 
num_classes=len(classes) 
samples_per_class=3
for y ,cls in enumerate(classes): 
    idxs=np.flatnonzero(y_train==y)
    idxs=np.random.choice(idxs,samples_per_class,replace=False) 
    for i ,idx in enumerate(idxs): 
        plt_idx=i*num_classes+y+1 
        plt.subplot(samples_per_class,num_classes,plt_idx) 
        plt.imshow(x_train[idx].astype('uint8'))
        plt.axis('off') 
        if i==0: 
            plt.title(cls) 
plt.show()

三、2-Layer Neural Network分類器實現

我們要做的就是建立一個2層神經網絡分類器，具體在python中以類的形式實現

3.1 建立神經網絡類

建立NN.py，初始化各參數

class Net2layer:
    # 定義網絡基本參數 初始化權重和偏置參數 輸入層 隱層 輸出層大小
    def __init__(self,input_size,hidden_size,output_size,std = 1e-3):
        self.params = {}
        self.params['w1'] = std * np.random.rand(input_size,hidden_size)
        self.params['b1'] = np.zeros(hidden_size)
        self.params['w2'] = std * np.random.rand(hidden_size,output_size)
        self.params['b2'] = np.zeros(output_size)

3.2 計算2層神經網絡損失值及梯度求解及詳細推導

第一層：第i個輸入，權重參數，偏置，激活函數爲relu

第一層線性輸出爲

$h_{i_1}=W_1^Tx_i$

經過relu之後

$r_i=\max(0,h_{i_1})$

第二層線性輸出:輸入 $r_{i}$ ，權重，偏置，激活函數softmax

$h_{i_2}=W_2^T\cdot r_{i}$

參考softmax分類器中loss和dW的計算方法，預測的樣本標籤爲的損失值爲

$L_i=\sum_{j=1}^K1\cdot (j==y_i)\cdot\log\frac{e^{W_{2_j}^Tr_{i}}}{\sum_{l=1}^Ke^{W_{2_l}^Tr_{i}}}$

加入正則化項，並替換線性表達式

$L_i=\sum_{j=1}^K1\cdot (j==y_i)\cdot\log\frac{e^{W_{2_j}^Tr_{i}}}{\sum_{l=1}^Ke^{W_{2_l}^Tr_{i}}}+\lambda\cdot W_{y_i}^2$

反向傳播求取第二層梯度

$\frac{\partial L_i}{\partial W_2_{y_i}}=-r_{i} \sum_j^K[1\cdot (y_i==j)-P(y_i=j|x_i,W)]+2\lambda\cdot W_2_{y_i}$

$\frac{\partial L_i}{\partial b_2_{y_i}}=\sum_j^K [1\cdot (y_i==j)-P(y_i=j|x_i,W)]$

求第一層梯度，又由鏈式求導法則

$\frac{\partial L_i}{\partial W_1_{y_i}}=\frac{\partial L_i}{\partial r_i}\frac{\partial r_i}{\partial h_{1_i}}\frac{\partial h_{1_i}}{\partial W_1_{y_i}}+2\lambda W_1_{y_i}$

$\frac{\partial L_i}{\partial r_i}=-W_{2_i} \sum_j^K[1\cdot (y_i==j)-P(y_i=j|x_i,W)]$

由relu函數求得梯度

$\frac{\partial r_i}{\partial h_{1_i}} = (h_{1_i}>0)?1:0=1\cdot\{h_{1_i}>0\}$

又 $\frac{\partial r_i}{\partial W_{1_{y_i}}} = x_i$

可得 $\frac{\partial L_i}{\partial b_1_{y_i}}=\frac{\partial L_i}{\partial r_i}\frac{\partial r_i}{\partial h_{1_i}}\frac{\partial h_{1_i}}{\partial b_1_{y_i}}$

由 $\frac{\partial h_{1_i}}{\partial b_1_{y_i}}=1$ 可得 $\frac{\partial L_i}{\partial b_1_{y_i}}$

定義NN.py，定義損失值和梯度的向量形式求解

# 計算前向通道 
    def loss(self,X,y,reg):
        n_sample = X.shape[0]

        w1,b1 = self.params['w1'],self.params['b1']
        w2,b2 = self.params['w2'],self.params['b2']
        
        #第一層輸出(N,H) relu激活
        h1 = np.maximum(0,np.dot(X,w1)+b1)
        #第二層線性輸出 (N,C) 之後接softmax
        score = np.dot(h1,w2)+b2
        
        #經過softmax層計算loss 
        nm = np.max(score,axis =1) #axis = 0 按橫軸求最大 (N,1)
        nm = nm.reshape(-1,1) #-1爲行不確定 1列
        shift_score = score - nm  #(N,C)
        #計算分類爲y[i]時的正確分類的輸出
        softmax_res = np.exp(shift_score)/np.sum(np.exp(shift_score),axis = 1).reshape(-1,1)
        softmax_output = softmax_res[range(n_sample),list(y)]
        loss = -np.sum(np.log(softmax_output))
        loss/=n_sample
        loss+= 0.5*reg*(np.sum(w1*w1)+np.sum(w2*w2))
        
        grads={}
        
        #計算梯度
        dscore = softmax_res
        dscore[range(n_sample),list(y)]-=1
        dscore/=n_sample
        
        #從後往前計算
        grads['w2'] =  h1.T.dot(dscore)+reg*w2
        grads['b2'] = np.sum(dscore,axis=0)
        
        #計算w2經過反向relu層的梯度
        dh = dscore.dot(w2.T)
        dh_relu = (h1>0)*dh
        grads['w1'] =  X.T.dot(dh_relu) + reg*w1
        grads['b1'] =  np.sum(dh_relu,axis=0)  #axis = 0 按縱軸求和
        return loss,grads

3.3 訓練過程

神經網絡訓練過程主要是完成以下工作：

1、接收數據，進行批處理和迭代相關參數的設置

2、喂數據，得到loss和梯度

3、更新神經網絡參數w1，w2，b1，b2

4、設置指數衰減學習率

    def train(self,X,y,X_val,y_val,num_iter,batch_size,learning_rate,learning_rate_decay,reg):
        #初始化參數
        n_sample = X.shape[0]
        #每輪迭代數目
        iter_per_epoch = max(n_sample/batch_size,1)
        loss_his=[]
        train_acc=[]
        val_acc=[]
        
        #開始循環訓練
        for itr in range(num_iter):
            #隨機選擇樣本
            batch_idx = np.random.choice(n_sample,batch_size,replace=True)
            X_batch = X[batch_idx]
            y_batch = y[batch_idx]
            
            loss,grads = self.loss(X_batch,y_batch,reg = reg)
            loss_his.append(loss)
            
            
            #參數更新
            self.params['w2']+=-learning_rate*grads['w2']
            self.params['b2']+=-learning_rate*grads['b2']
            self.params['w1']+=-learning_rate*grads['w1']
            self.params['b1']+=-learning_rate*grads['b1']
            
            if itr%100==0:
                print('iteration %d / %d: loss %f' %(itr,num_iter,loss))
                
            if itr%iter_per_epoch == 20:
                learning_rate*=learning_rate_decay
        
        return {
                'loss_his':loss_his,
                'train_acc':train_acc,
                'val_acc':val_acc
                }

3.4 預測過程

輸入數據，計算一輪前向通道，得到輸出標籤即可

    def predict(self,X):
        y_pred = None
        h=np.maximum(0,X.dot(self.params['w1'])+self.params['b1'])
        scores = h.dot(self.params['w2'])+self.params['b2']
        y_pred = np.argmax(scores,axis=1)
        return y_pred

四、調用Net2Layer分類器進行圖片分類

main.py 補充以下代碼

4.1 數據集分割，及歸零化處理圖片

#截斷數據訓練
num_training = 3000 #訓練集
num_val = 100      #驗證集
num_test=100       #測試集

mask=range(num_training,num_training+num_val) 
x_val=x_train[mask]
y_val=y_train[mask] 

mask=range(num_training) 
x_train=x_train[mask]
y_train=y_train[mask] 


mask=range(num_test) 
x_test=x_test[mask] 
y_test=y_test[mask]

#爲了歐氏距離的計算，我們把得到的圖像數據拉長成行向量32*32*3 -> 1*3072，代碼如下：
x_train = np.reshape(x_train,(x_train.shape[0],-1)) 
x_val   = np.reshape(x_val,(x_val.shape[0],-1)) 
x_test  = np.reshape(x_test,(x_test.shape[0],-1)) 

print(x_train.shape,x_val.shape,x_test.shape)

4.2 調用Net2Layer訓練數據並預測

#所有圖片減去均值 進行批歸一化處理
mean_image = np.mean(x_train,axis = 0)
x_train-= mean_image
x_val  -= mean_image
x_test -= mean_image
# 選用模型 開始訓練
print('init...')
# neuron-net
input_size = x_train.shape[1]
hidden_size =100
num_class = 10

classifier = Net2layer(input_size,hidden_size,num_class,std = 1e-3)

train_states = classifier.train(x_train,y_train,x_test,y_test,num_iter = 1000,batch_size = 200,learning_rate = 1e-3,learning_rate_decay=0.95,reg=0.5)
#val_acc = (Net2layer.predict(x_test)==y_test).mean()

print('predicting...')
y_test_pred=classifier.predict(x_test) 

#print(y_test_pred)
#print(y_test)

#計算準確度
num_correct = np.sum(y_test_pred == y_test)
accu = float(num_correct)/num_test
print('classifier: %f accuracy in %d image test' %(accu,num_test))

最終loss下降過程如下

init...
training...
iteration 0 / 1000: loss 2.336731
iteration 100 / 1000: loss 1.931344
iteration 200 / 1000: loss 1.746192
iteration 300 / 1000: loss 1.720358
iteration 400 / 1000: loss 1.678779
iteration 500 / 1000: loss 1.418102
iteration 600 / 1000: loss 1.431614
iteration 700 / 1000: loss 1.417301
iteration 800 / 1000: loss 1.291479
iteration 900 / 1000: loss 1.369606
predicting...
classifier: 0.432000 accuracy in 1000 image test

C231n Assignment 1 作業(4) 2層神經網絡分類器實現

一、Assignment 1 課程作業要求

二、數據集下載及處理

三、2-Layer Neural Network分類器實現

3.1 建立神經網絡類

3.2 計算2層神經網絡損失值及梯度求解及詳細推導

3.3 訓練過程

3.4 預測過程

四、調用Net2Layer分類器進行圖片分類

4.1 數據集分割，及歸零化處理圖片

4.2 調用Net2Layer訓練數據並預測

一鍵自動化博客發佈工具,用過的人都說好(掘金篇)

「Pygors跨平臺GUI」2：安裝MinGW-w64、MSYS2還是WSL2

[轉帖]

python列出centos7內存使用前50的進程信息

「Pygors跨平臺GUI」1：Pygors跨平臺GUI應用研究

nodejs學習06——小案例

評估統計算法在銀行僞造鈔票檢測中的價值

C# Xmlserializer 程序集內存泄露

Java ThreadPoolShutdown

5月21日相聚上海張江！與文心大模型一起共建大模型產業應用生態圈

C231n Assignment 1 作業(4) 2層神經網絡分類器實現

C231n Assignment 1 作業(1) K Nearest-Neighbour K最近鄰分類器實現

C231n Assignment 1 作業(3) Softmax線性分類器實現

C231n Assignment 1 作業(2) SVM線性分類器實現

從今天開始DSP嵌入式控制系統的研發

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結