目錄
一、Assignment 1 課程作業要求
具體原版要求見鏈接http://cs231n.github.io/assignments2019/assignment1/ 給不能FQ的同志搬運下
對cifar10 數據進行圖像數據的10分類，分別有以下要求
1: k-Nearest Neighbor classifier (20 points)
2: Training a Support Vector Machine (25 points)
3: Implement a Softmax classifier (20 points)
4: Two-Layer Neural Network (25 points)
5: Higher Level Representations: Image Features (10 points)
本文僅完成Softmax 線性分類器部分
二、數據集下載及處理
基本的數據集下載，數據簡單批處理，參考https://github.com/Halfish/cs231n/blob/master/assignment1/knn.ipynb中，將下載後的cifar10數據集解壓
data_utils.py:
# -*- coding: utf-8 -*-
"""
Created on Sat Feb 29 12:57:18 2020
data_utils.py
@author: acer
"""
import pickle
import numpy as np
import os
def load_cifar_batch(filename):
with open(filename,'rb') as f :
datadict=pickle.load(f,encoding='bytes')
x=datadict[b'data']
y=datadict[b'labels']
x=x.reshape(10000,3,32,32).transpose(0,2,3,1).astype('float')
y=np.array(y)
#以數組形式輸出
return x,y
#root 爲cifar文件根目錄
def load_cifar10(root_path):
xs=[]
ys=[]
# 訓練集1-5
for b in range(1,2): #這裏只選10000張作爲訓練集
# 讀取訓練集圖片
f=os.path.join(root_path,'data_batch_%d' % (b,))
#每批讀取
x,y=load_cifar_batch(f)
#累加 x y 將5批數據合併
xs.append(x)
ys.append(y)
#數組拼接
Xtrain=np.concatenate(xs)
Ytrain=np.concatenate(ys)
del x ,y
# 讀取測試集圖片
Xtest,Ytest=load_cifar_batch(os.path.join(root_path,'test_batch'))
return Xtrain,Ytrain,Xtest,Ytest
建立讀取數據訓練的主文件main.py,調用函數讀取並顯示cifar10圖片
main.py:
import numpy as np
from data_utils import load_cifar10
import matplotlib.pyplot as plt
#提取數據
x_train,y_train,x_test,y_test = load_cifar10('.\cifar-10-python\cifar-10-batches-py')
x_test = x_test[0:2000,:]
y_test = y_test[0:2000]
print('training data shape:',x_train.shape)
print('training labels shape:',y_train.shape)
print('test data shape:',x_test.shape)
print('test labels shape:',y_test.shape)
#載入數據
classes=['plane','car','bird','cat','deer','dog','frog','horse','ship','truck']
num_classes=len(classes)
samples_per_class=3
for y ,cls in enumerate(classes):
idxs=np.flatnonzero(y_train==y)
idxs=np.random.choice(idxs,samples_per_class,replace=False)
for i ,idx in enumerate(idxs):
plt_idx=i*num_classes+y+1
plt.subplot(samples_per_class,num_classes,plt_idx)
plt.imshow(x_train[idx].astype('uint8'))
plt.axis('off')
if i==0:
plt.title(cls)
plt.show()
三、Softmax線性分類器實現
我們要做的就是建立一個SVM線性分類器，具體在python中以類的形式實現，原理見具體Softmax原理
3.1 計算Softmax損失值和梯度
輸入圖片X：N*D，輸出標籤爲y：N*C，線性權值矩陣 W：D*C
線性得分
要計算多類softmax分類器，首先來看看其中二分類logistic分類器，在logistic迴歸中，有樣本
二分類激活函數假設函數
損失函數
![L(W)=-\frac{1}{N}\sum_{i=1}^N[y_i\log{h(X_i)}+(1-y_i)\log(1-h_W(X_i))]](https://pic1.xuehuaimg.com/proxy/csdn/https://private.codecogs.com/gif.latex?L%28W%29%3D-%5Cfrac%7B1%7D%7BN%7D%5Csum_%7Bi%3D1%7D%5EN%5By_i%5Clog%7Bh%28X_i%29%7D+%281-y_i%29%5Clog%281-h_W%28X_i%29%29%5D)
擴展到多分類的情況，得分score不再是{0,1}兩個值，而是估計X_i的每一種分類的概率
softmax中將X_i分類判別到j的概率爲
假設函數
輸出,預測
加入正則化項，並替換線性表達式
取梯度
![\frac{\partial L_i}{\partial W_{y_i}}=-x_i [1\cdot (y_i==j)-P(y_i=j|X_i,W)]+2\lambda\cdot W_{y_i}](https://pic1.xuehuaimg.com/proxy/csdn/https://private.codecogs.com/gif.latex?%5Cfrac%7B%5Cpartial%20L_i%7D%7B%5Cpartial%20W_%7By_i%7D%7D%3D-x_i%20%5B1%5Ccdot%20%28y_i%3D%3Dj%29-P%28y_i%3Dj%7CX_i%2CW%29%5D+2%5Clambda%5Ccdot%20W_%7By_i%7D)
定義softmax.py，定義softmax損失值和梯度的向量形式求解
def loss_softmax(X,y,W,reg):
#softmax classifier 訓練中
# Inputs:
# W: A numpy array of shape (D, C) containing weights.
# X: A numpy array of shape (N, D) containing a minibatch of data.
# y: A numpy array of shape (N, 1) containing training labels; y[i] = p in (0,1)
# P(y = K | x = xi) = e^x_k/sum_j(e^x_j)
# 線性迴歸
# 得分 Score: s = xW
# 1/(1+exp(-f(x,W)))= 1/(1+exp(-xW)) s:N*C x:N*D w:D*C N-圖像數 D-每幅圖維數 C-類別數
# loss_i = -log(e^s_yi/sum_j(e^s_j)) = -s_y_i + log(sum_j(e^s_j))
# dwj: 當 j = y[i] : dwj = x_i*(0-P(y = j | x = x_i))
# 當 j != y[i] : dwj = x_i*(1-P(y = j | x = x_i))
# dW = (-1/n) sum_j(dwj)
#輸出類別
#score = np.dot(X,W)
n_classes = W.shape[1]
n_sample = X.shape[0]
n_dim = X.shape[1]
dW = np.zeros((W.shape[0],W.shape[1]))
loss = [0.0]
for i in range(n_sample):
score = np.dot(X[i],W)
loss_i = -score[y[i]] + np.log(np.sum(np.exp(score)))
loss += loss_i
for j in range(n_classes):
P_i = np.exp(score[j])/np.sum(np.exp(score))
if y[i] == j:
dW[:,j] += X[i]*(1-P_i)
else:
dW[:,j] += X[i]*(-P_i)
loss = loss/n_sample + 0.5*reg*np.sum(W*W)
dW = -dW/n_sample + reg*W
return loss,dW
3.2定義Softmax線性分類器
先定義廣義的線性分類器 LinearClassifier.py，loss值的計算先不定義，後續SVM分類器和softmax分類器均可在此基礎上繼承
3.2.1 定義初始化和訓練函數
# 線性分類器類 設置初始化 訓練 預測三個函數
class LinearClassifier:
def _init_(self):
pass
#self.W = None #D*C
# 其中訓練函數：數據批處理 迭代輪數 計算損失和梯度 更新權值函數
def train(self,X,y,learning_rate,reg,train_iters,batch_size):
#輸出類別
n_class = np.max(y)+1
#通道數
n_dimension = X.shape[1]
#樣本數
n_sample = X.shape[0]
#if self.W==None:
self.W = 0.001*np.random.randn(n_dimension,n_class) #D*C
#批處理數據,計算損失值 梯度 並更新權值
loss_history = []
for it in range(train_iters):
#batch_gen = get_batch([X,y],batch_size,True)
#x_batch, y_batch = next(batch_gen)
#在N中選batch_size個數組成數組
batch_idx = np.random.choice(n_sample,batch_size,replace=True)
x_batch = X[batch_idx]
y_batch = y[batch_idx]
#print(x_batch, y_batch)
#計算loss grad
loss,grad = self.loss(x_batch,y_batch,reg)
loss_history.append(loss)
#更新權值
self.W += - learning_rate*grad
if it%5 ==0:
print('after %d/%d iterations, the algorithm gets %f loss value'%(it,train_iters,loss))
3.2.2 定義測試函數和損失值函數
其中loss函數先不定義，後續專門在線性分類器的繼承類softmax分類器中調用之前定義的loss_softmax
def predict(self,X):
#前向通道 輸出score
y_pred = np.zeros(X.shape[1])
score = np.dot(X,self.W)
y_pred = np.argmax(score,axis=1)
return y_pred
def loss(self,x_batch,y_batch,reg):
pass
3.2.3 定義Softmax線性分類器
#Softmax 繼承類
class SoftmaxClassifier(LinearClassifier):
def loss(self,x_batch,y_batch,reg):
return loss_softmax(x_batch,y_batch,self.W,reg)
四、調用Softmax分類器進行圖片分類
main.py 補充以下代碼
4.1 數據集分割，及歸零化處理圖片
#截斷數據訓練
num_training = 3000 #訓練集
num_val = 100 #驗證集
num_test=100 #測試集
mask=range(num_training,num_training+num_val)
x_val=x_train[mask]
y_val=y_train[mask]
mask=range(num_training)
x_train=x_train[mask]
y_train=y_train[mask]
mask=range(num_test)
x_test=x_test[mask]
y_test=y_test[mask]
#爲了歐氏距離的計算，我們把得到的圖像數據拉長成行向量32*32*3 -> 1*3072，代碼如下：
x_train = np.reshape(x_train,(x_train.shape[0],-1))
x_val = np.reshape(x_val,(x_val.shape[0],-1))
x_test = np.reshape(x_test,(x_test.shape[0],-1))
print(x_train.shape,x_val.shape,x_test.shape)
4.2 調用Softmax訓練數據並預測
#linear Softmax
#所有圖片減去均值 進行批歸一化處理
mean_image = np.mean(x_train,axis = 0)
x_train-= mean_image
x_val -= mean_image
x_test -= mean_image
# 選用模型 開始訓練
print('init...')
classifier = SoftmaxClassifier()
classifier.train(x_train,y_train,learning_rate=1e-8,reg=1e-5,train_iters=100,batch_size = 250)
print('predicting...')
y_test_pred=classifier.predict(x_test)
print(y_test_pred)
print(y_test)
#計算準確度
num_correct = np.sum(y_test_pred == y_test)
accu = float(num_correct)/num_test
print('classifier: %f accuracy in %d image test' %(accu,num_test))