CNN卷積神經網絡CIFAR-10圖片分類

概述

對CIFAR-10 數據集的分類是機器學習中一個公開的基準測試問題，其任務是對一組32x32RGB的圖像進行分類，這些圖像涵蓋了10個類別：
飛機， 汽車， 鳥， 貓， 鹿， 狗， 青蛙， 馬， 船以及卡車。

模型結構

本教程中的模型是一個多層架構，由卷積層和非線性層(nonlinearities)交替多次排列後構成。這些層最終通過全連通層對接到softmax分類器上。這一模型除了最頂部的幾層外，基本跟Alex Krizhevsky提出的模型一致。
在一個GPU上經過幾個小時的訓練後，該模型達到了最高86%的精度。細節請查看下面的描述以及代碼。模型中包含了1,068,298個學習參數，分類一副圖像需要大概19.5M個乘加操作。

模型包括：

CIFAR-10 網絡模型部分的代碼位於 cifar10.py. 完整的訓練圖中包含約765個操作。但是我們發現通過下面的模塊來構造訓練圖可以最大限度的提高代碼複用率:
模型輸入: 包括inputs() 、 distorted_inputs()等一些操作，分別用於讀取CIFAR的圖像並進行預處理，做爲後續評估和訓練的輸入；
模型預測: 包括inference()等一些操作，用於進行統計計算，比如在提供的圖像進行分類； adds operations that perform inference, i.e. classification, on supplied images.
模型訓練: 包括loss() and train()等一些操作，用於計算損失、計算梯度、進行變量更新以及呈現最終結果。

import numpy as np

# 序列化和反序列化
import pickle

from sklearn.preprocessing import OneHotEncoder

import warnings
warnings.filterwarnings('ignore')

import tensorflow as tf

數據加載

def unpickle(file):
    import pickle
    with open(file, 'rb') as fo:
        dict = pickle.load(fo, encoding='ISO-8859-1')
    return dict

# def unpickle(file):
#     import pickle
#     with open(file, 'rb') as fo:
#         dict = pickle.load(fo, encoding='bytes')
#     return dict

labels = []
X_train = []
for i in range(1,6):
    data = unpickle('./cifar-10-python/cifar-10-batches-py/data_batch_%d'%(i))
    labels.append(data['labels'])
    X_train.append(data['data'])
print(X_train[:1])
# 將list類型轉換爲ndarray
y_train = np.array(labels).reshape(-1)
X_train = np.array(X_train)

# reshape
X_train = X_train.reshape(-1,3072)

# 目標值概率
one_hot = OneHotEncoder()
y_train =one_hot.fit_transform(y_train.reshape(-1,1)).toarray()#轉外概率 
display(X_train.shape,y_train.shape)     # 50000張照片  32*32*3

[array([[ 59,  43,  50, ..., 140,  84,  72],
       [154, 126, 105, ..., 139, 142, 144],
       [255, 253, 253, ...,  83,  83,  84],
       ...,
       [ 71,  60,  74, ...,  68,  69,  68],
       [250, 254, 211, ..., 215, 255, 254],
       [ 62,  61,  60, ..., 130, 130, 131]], dtype=uint8)]
(1, 1, 3072)



(50000, 3072)



(50000, 10)

構建神經網絡

X = tf.placeholder(dtype=tf.float32,shape = [None,3072])
y = tf.placeholder(dtype=tf.float32,shape = [None,10])
kp = tf.placeholder(dtype=tf.float32)

### !!! 給成常量了
def gen_v(shape):
    return tf.Variable(tf.truncated_normal(shape = shape))

def conv(input_,filter_,b):
    conv = tf.nn.relu(tf.nn.conv2d(input_,filter_,strides=[1,1,1,1],padding='SAME') + b)
    return tf.nn.max_pool(conv,[1,3,3,1],[1,2,2,1],'SAME')

def net_work(input_,kp):
    
#     形狀改變，4維
    input_ = tf.reshape(input_,shape = [-1,32,32,3])
#     第一層
    filter1 = gen_v(shape = [3,3,3,64])
    b1 = gen_v(shape = [64])
    conv1 = conv(input_,filter1,b1)
#     歸一化
    conv1 = tf.layers.batch_normalization(conv1,training=True)
    
#     第二層
    filter2 = gen_v([3,3,64,128])
    b2 = gen_v(shape = [128])
    conv2 = conv(conv1,filter2,b2)
    conv2 = tf.layers.batch_normalization(conv2,training=True)
    
#     第三層
    filter3 = gen_v([3,3,128,256])
    b3 = gen_v([256])
    conv3 = conv(conv2,filter3,b3)
    conv3 = tf.layers.batch_normalization(conv3,training=True)
    
#     第一層全連接層
    dense = tf.reshape(conv3,shape = [-1,4*4*256])
    fc1_w = gen_v(shape = [4*4*256,1024])
    fc1_b = gen_v([1024])
    fc1 = tf.matmul(dense,fc1_w) + fc1_b
    fc1 = tf.layers.batch_normalization(fc1,training=True)
    fc1 = tf.nn.relu(fc1)
#     fc1.shape = [-1,1024]
    
    
#     dropout
    dp = tf.nn.dropout(fc1,keep_prob=kp)
    
#     第二層全連接層
    fc2_w = gen_v(shape = [1024,1024])
    fc2_b = gen_v(shape = [1024])
    fc2 = tf.nn.relu(tf.layers.batch_normalization(tf.matmul(dp,fc2_w) + fc2_b,training=True))
    
#     輸出層
    out_w = gen_v(shape = [1024,10])
    out_b = gen_v(shape = [10])
    out = tf.matmul(fc2,out_w) + out_b
    return out

損失函數準確率

out = net_work(X,kp)

loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=out))

# 準確率
y_ = tf.nn.softmax(out)

# equal 相當於 == 
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(y,axis = -1),tf.argmax(y_,axis = 1)),tf.float16))
accuracy

WARNING: Logging before flag parsing goes to stderr.
W1214 11:53:00.069432  5584 deprecation.py:323] From <ipython-input-4-1d11df91139b>:18: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.BatchNormalization instead.  In particular, `tf.control_dependencies(tf.GraphKeys.UPDATE_OPS)` should not be used (consult the `tf.keras.layers.batch_normalization` documentation).
W1214 11:53:00.074395  5584 deprecation.py:323] From d:\python37\lib\site-packages\tensorflow_core\python\layers\normalization.py:327: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
W1214 11:53:00.255907  5584 deprecation.py:506] From <ipython-input-4-1d11df91139b>:43: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.





<tf.Tensor 'Mean_1:0' shape=() dtype=float16>

最後化

opt = tf.train.AdamOptimizer().minimize(loss)
opt

<tf.Operation 'Adam' type=NoOp>

開啓訓練

epoches = 50000

saver = tf.train.Saver()

X_train.shape
y_train.shape

(50000, 10)

index = 0
def next_batch(X,y):
    global index
    batch_X = X[index*128:(index+1)*128]
    batch_y = y[index*128:(index+1)*128]
    index+=1
    if index == 390:
        index = 0
    return batch_X,batch_y

test = unpickle('./cifar-10-python/cifar-10-batches-py/test_batch')
y_test = test['labels']
y_test = np.array(y_test)
X_test = test['data']
y_test = one_hot.transform(y_test.reshape(-1,1)).toarray()
y_test[:10]

array([[0., 0., 0., 1., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 1., 0.],
       [0., 0., 0., 0., 0., 0., 0., 0., 1., 0.],
       [1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
       [0., 0., 0., 1., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.]])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(epoches):
        batch_X,batch_y = next_batch(X_train,y_train)
        opt_,loss_ = sess.run([opt,loss],feed_dict = {X:batch_X,y:batch_y,kp:0.5})
        print('----------------------------',loss_)
        if i % 100 == 0:
            score_test = sess.run(accuracy,feed_dict = {X:X_test,y:y_test,kp:1.0})
            score_train = sess.run(accuracy,feed_dict = {X:batch_X,y:batch_y,kp:1.0})
            print('iter count:%d。mini_batch loss:%0.4f。訓練數據上的準確率：%0.4f。測試數據上準確率：%0.4f'%
                  (i+1,loss_,score_train,score_test)) 

最後由於未對數據進行處理,已經使用的是普通的神經網絡,準確率只有50%+