一、Assignment 1 课程作业要求

具体原版要求见链接http://cs231n.github.io/assignments2019/assignment1/ 给不能FQ的同志搬运下

对cifar10 数据进行图像数据的10分类，分别有以下要求

1: k-Nearest Neighbor classifier (20 points) 求解链接

2: Training a Support Vector Machine (25 points) 求解链接

3: Implement a Softmax classifier (20 points) 求解链接

4: Two-Layer Neural Network (25 points) 求解链接

5: Higher Level Representations: Image Features (10 points)

本文仅完成Two-Layer Neural Network 分类器部分，其他部分请点击上面相关求解链接

二、数据集下载及处理

基本的数据集下载，数据简单批处理，参考https://github.com/Halfish/cs231n/blob/master/assignment1/knn.ipynb中，将下载后的cifar10数据集解压

data_utils.py:

# -*- coding: utf-8 -*-
"""
Created on Sat Feb 29 12:57:18 2020
data_utils.py
@author: acer
"""
import pickle
import numpy as np 
import os 

def load_cifar_batch(filename): 
    with open(filename,'rb') as f : 
        datadict=pickle.load(f,encoding='bytes') 
        x=datadict[b'data'] 
        y=datadict[b'labels'] 
        x=x.reshape(10000,3,32,32).transpose(0,2,3,1).astype('float') 
        y=np.array(y) 
        #以数组形式输出
        return x,y 
#root 为cifar文件根目录
def load_cifar10(root_path): 
    xs=[] 
    ys=[] 
    # 训练集1-5
    for b in range(1,2): #这里只选10000张作为训练集
        # 读取训练集图片
        f=os.path.join(root_path,'data_batch_%d' % (b,)) 
        #每批读取
        x,y=load_cifar_batch(f) 
        #累加 x y 将5批数据合并
        xs.append(x) 
        ys.append(y) 
        #数组拼接
        Xtrain=np.concatenate(xs) 
        Ytrain=np.concatenate(ys) 
    del x ,y 
    # 读取测试集图片
    Xtest,Ytest=load_cifar_batch(os.path.join(root_path,'test_batch'))   
    return Xtrain,Ytrain,Xtest,Ytest

建立读取数据训练的主文件main.py,调用函数读取并显示cifar10图片

main.py:

import numpy as np
from data_utils import load_cifar10
import matplotlib.pyplot as plt

#提取数据
x_train,y_train,x_test,y_test = load_cifar10('.\cifar-10-python\cifar-10-batches-py')
x_test = x_test[0:2000,:]
y_test = y_test[0:2000]
print('training data shape:',x_train.shape) 
print('training labels shape:',y_train.shape) 
print('test data shape:',x_test.shape) 
print('test labels shape:',y_test.shape)

#载入数据
classes=['plane','car','bird','cat','deer','dog','frog','horse','ship','truck'] 
num_classes=len(classes) 
samples_per_class=3
for y ,cls in enumerate(classes): 
    idxs=np.flatnonzero(y_train==y)
    idxs=np.random.choice(idxs,samples_per_class,replace=False) 
    for i ,idx in enumerate(idxs): 
        plt_idx=i*num_classes+y+1 
        plt.subplot(samples_per_class,num_classes,plt_idx) 
        plt.imshow(x_train[idx].astype('uint8'))
        plt.axis('off') 
        if i==0: 
            plt.title(cls) 
plt.show()

三、2-Layer Neural Network分类器实现

我们要做的就是建立一个2层神经网络分类器，具体在python中以类的形式实现

3.1 建立神经网络类

建立NN.py，初始化各参数

class Net2layer:
    # 定义网络基本参数 初始化权重和偏置参数 输入层 隐层 输出层大小
    def __init__(self,input_size,hidden_size,output_size,std = 1e-3):
        self.params = {}
        self.params['w1'] = std * np.random.rand(input_size,hidden_size)
        self.params['b1'] = np.zeros(hidden_size)
        self.params['w2'] = std * np.random.rand(hidden_size,output_size)
        self.params['b2'] = np.zeros(output_size)

3.2 计算2层神经网络损失值及梯度求解及详细推导

第一层：第i个输入，权重参数，偏置，激活函数为relu

第一层线性输出为

$h_{i_1}=W_1^Tx_i$

经过relu之后

$r_i=\max(0,h_{i_1})$

第二层线性输出:输入 $r_{i}$ ，权重，偏置，激活函数softmax

$h_{i_2}=W_2^T\cdot r_{i}$

参考softmax分类器中loss和dW的计算方法，预测的样本标签为的损失值为

$L_i=\sum_{j=1}^K1\cdot (j==y_i)\cdot\log\frac{e^{W_{2_j}^Tr_{i}}}{\sum_{l=1}^Ke^{W_{2_l}^Tr_{i}}}$

加入正则化项，并替换线性表达式

$L_i=\sum_{j=1}^K1\cdot (j==y_i)\cdot\log\frac{e^{W_{2_j}^Tr_{i}}}{\sum_{l=1}^Ke^{W_{2_l}^Tr_{i}}}+\lambda\cdot W_{y_i}^2$

反向传播求取第二层梯度

$\frac{\partial L_i}{\partial W_2_{y_i}}=-r_{i} \sum_j^K[1\cdot (y_i==j)-P(y_i=j|x_i,W)]+2\lambda\cdot W_2_{y_i}$

$\frac{\partial L_i}{\partial b_2_{y_i}}=\sum_j^K [1\cdot (y_i==j)-P(y_i=j|x_i,W)]$

求第一层梯度，又由链式求导法则

$\frac{\partial L_i}{\partial W_1_{y_i}}=\frac{\partial L_i}{\partial r_i}\frac{\partial r_i}{\partial h_{1_i}}\frac{\partial h_{1_i}}{\partial W_1_{y_i}}+2\lambda W_1_{y_i}$

$\frac{\partial L_i}{\partial r_i}=-W_{2_i} \sum_j^K[1\cdot (y_i==j)-P(y_i=j|x_i,W)]$

由relu函数求得梯度

$\frac{\partial r_i}{\partial h_{1_i}} = (h_{1_i}>0)?1:0=1\cdot\{h_{1_i}>0\}$

又 $\frac{\partial r_i}{\partial W_{1_{y_i}}} = x_i$

可得 $\frac{\partial L_i}{\partial b_1_{y_i}}=\frac{\partial L_i}{\partial r_i}\frac{\partial r_i}{\partial h_{1_i}}\frac{\partial h_{1_i}}{\partial b_1_{y_i}}$

由 $\frac{\partial h_{1_i}}{\partial b_1_{y_i}}=1$ 可得 $\frac{\partial L_i}{\partial b_1_{y_i}}$

定义NN.py，定义损失值和梯度的向量形式求解

# 计算前向通道 
    def loss(self,X,y,reg):
        n_sample = X.shape[0]

        w1,b1 = self.params['w1'],self.params['b1']
        w2,b2 = self.params['w2'],self.params['b2']
        
        #第一层输出(N,H) relu激活
        h1 = np.maximum(0,np.dot(X,w1)+b1)
        #第二层线性输出 (N,C) 之后接softmax
        score = np.dot(h1,w2)+b2
        
        #经过softmax层计算loss 
        nm = np.max(score,axis =1) #axis = 0 按横轴求最大 (N,1)
        nm = nm.reshape(-1,1) #-1为行不确定 1列
        shift_score = score - nm  #(N,C)
        #计算分类为y[i]时的正确分类的输出
        softmax_res = np.exp(shift_score)/np.sum(np.exp(shift_score),axis = 1).reshape(-1,1)
        softmax_output = softmax_res[range(n_sample),list(y)]
        loss = -np.sum(np.log(softmax_output))
        loss/=n_sample
        loss+= 0.5*reg*(np.sum(w1*w1)+np.sum(w2*w2))
        
        grads={}
        
        #计算梯度
        dscore = softmax_res
        dscore[range(n_sample),list(y)]-=1
        dscore/=n_sample
        
        #从后往前计算
        grads['w2'] =  h1.T.dot(dscore)+reg*w2
        grads['b2'] = np.sum(dscore,axis=0)
        
        #计算w2经过反向relu层的梯度
        dh = dscore.dot(w2.T)
        dh_relu = (h1>0)*dh
        grads['w1'] =  X.T.dot(dh_relu) + reg*w1
        grads['b1'] =  np.sum(dh_relu,axis=0)  #axis = 0 按纵轴求和
        return loss,grads

3.3 训练过程

神经网络训练过程主要是完成以下工作：

1、接收数据，进行批处理和迭代相关参数的设置

2、喂数据，得到loss和梯度

3、更新神经网络参数w1，w2，b1，b2

4、设置指数衰减学习率

    def train(self,X,y,X_val,y_val,num_iter,batch_size,learning_rate,learning_rate_decay,reg):
        #初始化参数
        n_sample = X.shape[0]
        #每轮迭代数目
        iter_per_epoch = max(n_sample/batch_size,1)
        loss_his=[]
        train_acc=[]
        val_acc=[]
        
        #开始循环训练
        for itr in range(num_iter):
            #随机选择样本
            batch_idx = np.random.choice(n_sample,batch_size,replace=True)
            X_batch = X[batch_idx]
            y_batch = y[batch_idx]
            
            loss,grads = self.loss(X_batch,y_batch,reg = reg)
            loss_his.append(loss)
            
            
            #参数更新
            self.params['w2']+=-learning_rate*grads['w2']
            self.params['b2']+=-learning_rate*grads['b2']
            self.params['w1']+=-learning_rate*grads['w1']
            self.params['b1']+=-learning_rate*grads['b1']
            
            if itr%100==0:
                print('iteration %d / %d: loss %f' %(itr,num_iter,loss))
                
            if itr%iter_per_epoch == 20:
                learning_rate*=learning_rate_decay
        
        return {
                'loss_his':loss_his,
                'train_acc':train_acc,
                'val_acc':val_acc
                }

3.4 预测过程

输入数据，计算一轮前向通道，得到输出标签即可

    def predict(self,X):
        y_pred = None
        h=np.maximum(0,X.dot(self.params['w1'])+self.params['b1'])
        scores = h.dot(self.params['w2'])+self.params['b2']
        y_pred = np.argmax(scores,axis=1)
        return y_pred

四、调用Net2Layer分类器进行图片分类

main.py 补充以下代码

4.1 数据集分割，及归零化处理图片

#截断数据训练
num_training = 3000 #训练集
num_val = 100      #验证集
num_test=100       #测试集

mask=range(num_training,num_training+num_val) 
x_val=x_train[mask]
y_val=y_train[mask] 

mask=range(num_training) 
x_train=x_train[mask]
y_train=y_train[mask] 


mask=range(num_test) 
x_test=x_test[mask] 
y_test=y_test[mask]

#为了欧氏距离的计算，我们把得到的图像数据拉长成行向量32*32*3 -> 1*3072，代码如下：
x_train = np.reshape(x_train,(x_train.shape[0],-1)) 
x_val   = np.reshape(x_val,(x_val.shape[0],-1)) 
x_test  = np.reshape(x_test,(x_test.shape[0],-1)) 

print(x_train.shape,x_val.shape,x_test.shape)

4.2 调用Net2Layer训练数据并预测

#所有图片减去均值 进行批归一化处理
mean_image = np.mean(x_train,axis = 0)
x_train-= mean_image
x_val  -= mean_image
x_test -= mean_image
# 选用模型 开始训练
print('init...')
# neuron-net
input_size = x_train.shape[1]
hidden_size =100
num_class = 10

classifier = Net2layer(input_size,hidden_size,num_class,std = 1e-3)

train_states = classifier.train(x_train,y_train,x_test,y_test,num_iter = 1000,batch_size = 200,learning_rate = 1e-3,learning_rate_decay=0.95,reg=0.5)
#val_acc = (Net2layer.predict(x_test)==y_test).mean()

print('predicting...')
y_test_pred=classifier.predict(x_test) 

#print(y_test_pred)
#print(y_test)

#计算准确度
num_correct = np.sum(y_test_pred == y_test)
accu = float(num_correct)/num_test
print('classifier: %f accuracy in %d image test' %(accu,num_test))

最终loss下降过程如下

init...
training...
iteration 0 / 1000: loss 2.336731
iteration 100 / 1000: loss 1.931344
iteration 200 / 1000: loss 1.746192
iteration 300 / 1000: loss 1.720358
iteration 400 / 1000: loss 1.678779
iteration 500 / 1000: loss 1.418102
iteration 600 / 1000: loss 1.431614
iteration 700 / 1000: loss 1.417301
iteration 800 / 1000: loss 1.291479
iteration 900 / 1000: loss 1.369606
predicting...
classifier: 0.432000 accuracy in 1000 image test

C231n Assignment 1 作业(4) 2层神经网络分类器实现

一、Assignment 1 课程作业要求

二、数据集下载及处理

三、2-Layer Neural Network分类器实现

3.1 建立神经网络类

3.2 计算2层神经网络损失值及梯度求解及详细推导

3.3 训练过程

3.4 预测过程

四、调用Net2Layer分类器进行图片分类

4.1 数据集分割，及归零化处理图片

4.2 调用Net2Layer训练数据并预测

TDengine docker安装方法

vue项目获取富文本编辑器wangEditor内容导出为word（html转word格式并下载）

dotnet C# 创建 X11 应用时设置窗口背景颜色

vue3组件通信与props

sapui5

Alpine Linux apk add DNS lookup error

部分JDK版本的发布时间

工作中用到的脚本合集

合并代码时Beyond Compare设置

Navicat安装与激活教程

C231n Assignment 1 作業(4) 2層神經網絡分類器實現

C231n Assignment 1 作業(1) K Nearest-Neighbour K最近鄰分類器實現

C231n Assignment 1 作業(3) Softmax線性分類器實現

C231n Assignment 1 作業(2) SVM線性分類器實現

從今天開始DSP嵌入式控制系統的研發

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結