神經網絡算法：ResNet手寫數字識別

ResNet手寫數字識別

Kaiming He博士在ResNet論文中提出了這個思想,用於解決深層神經網絡不好訓練的問題.

上圖是論文中關於傳統神經網絡不好識別的描述,可以看到.隨着神經網絡的深度增大,訓練的錯誤率逐漸上升.於是他提出了利用殘差學習的思想,就是機器不再學習整個分佈,而是學習當前網絡生成的分佈與原始分佈的差異.在CIFAR-10上的網絡結構如下:

他使用如下的誤差訓練公式,y-x即爲殘差:

具體在CIFAR-10上的訓練網絡如下圖最右側所示,

這些跳過來的路徑使得梯度反向傳播時很少出現誤差消失的情況.
我在mnist數據集上參考了幾篇論文,實現了類似的神經網絡,效果比較相似,但是mnist基本上不用做池化操作,因爲畢竟每個樣本的大小隻有28*28.使用keras訓練,最終準確率大概爲0.9906,最終把模型保存爲h5文件.下面直接上訓練的代碼.recognize是使用訓練好的模型做手寫數字識別的模塊,本身和深度學習關係不大.網絡結構大概是這個樣子,keras導出的,使用

keras.utils.plot_model(self.model,to_file='mnist_res.png')

就可以把模型的結構可視化了.

'''用於訓練mnist模型,生成h5模型文件'''
import tensorflow as tf
from tensorflow import keras
import recogonize
'''
識別的效果:
Epoch 1/5
 - 141s - loss: 0.2842 - acc: 0.9086 - val_loss: 0.0605 - val_acc: 0.9814
Epoch 2/5
 - 153s - loss: 0.0776 - acc: 0.9775 - val_loss: 0.0465 - val_acc: 0.9847
Epoch 3/5
 - 147s - loss: 0.0557 - acc: 0.9837 - val_loss: 0.0442 - val_acc: 0.9852
Epoch 4/5
 - 142s - loss: 0.0439 - acc: 0.9872 - val_loss: 0.0319 - val_acc: 0.9898
Epoch 5/5
 - 145s - loss: 0.0370 - acc: 0.9890 - val_loss: 0.0323 - val_acc: 0.9906
loss in test set: 0.032348819018113865
acc in test set: 0.9906
'''
class ResNet:
    def __init__(self,input_shape,classes,img_cols,img_rows):
        self.classes = classes
        self.img_cols = img_cols
        self.img_rows = img_rows
        self.input_shape = img_cols, img_rows, 1
        self.model = None
        (train_images, train_labels), (test_images, test_labels) = keras.datasets.mnist.load_data()
        self.train_images = train_images.reshape((train_images.shape[0], img_cols, img_rows, 1)).astype('float32')
        self.test_images = test_images.reshape((test_images.shape[0], img_cols, img_rows, 1)).astype('float32')
        self.train_images /= 255
        self.test_images /= 255
        self.train_labels = keras.utils.to_categorical(train_labels, classes)
        self.test_labels = keras.utils.to_categorical(test_labels, classes)

    #殘差塊,private
    def __res_block(self,x, channels, i):
        if i==1:
            xstrides = (1,1)
            x_add = x
        else:
            xstrides =(2,2)
            x_add = keras.layers.Conv2D(channels,
                                        kernel_size=3,
                                        activation='relu',
                                        padding='same',
                                        strides=(2,2)
                                        )(x)
        x = keras.layers.Conv2D(channels,
                                kernel_size=3,
                                activation='relu',
                                padding='same',
                                strides=xstrides)(x)
        x = keras.layers.Conv2D(channels,
                                kernel_size=3,
                                padding='same',
                                strides=(1,1))(x)

        x = keras.layers.add([x, x_add])
        x = keras.layers.Activation(keras.backend.relu)(x)
        return x

    def buildModel(self):
        firstin = keras.layers.Input(shape=self.input_shape)
        x = keras.layers.Conv2D(16,
                                kernel_size=7,
                                activation='relu',
                                input_shape=self.input_shape,
                                padding='same'
                                )(firstin)
        for i in range(2):
            x = self.__res_block(x,16,i)

        for i in range(2):
            x = self.__res_block(x,32,i)

        x = keras.layers.AveragePooling2D(pool_size=(7, 7))(x)
        x = keras.layers.Flatten()(x)
        x = keras.layers.Dense(10, activation='softmax')(x)

        self.model = keras.Model(inputs=firstin, outputs=x)
        keras.utils.plot_model(self.model,to_file='mnist_res.png')
        self.model.compile(loss='categorical_crossentropy',
                    optimizer='adadelta',
                    metrics=['accuracy']
                    )

    def train(self):
        self.model.fit(self.train_images,self.train_labels,
                batch_size=10,
                verbose=2,#精細化顯示
                validation_data=(self.test_images,self.test_labels),
                epochs=5
                )
        self.model.save('mnist_res_model.h5')
        json_str = self.model.to_json()
        with open('mnist_res_model.json','w') as file:
            file.write(json_str)
        loss,acc = self.model.evaluate(self.test_images,self.test_labels,verbose=0)
        print("loss in test set:",loss)
        print("acc in test set:", acc)

    def test(self):
        recogonize.main(0)


def main(_):
    tr = ResNet((28,28,1),classes=10,img_cols=28,img_rows=28)
    tr.buildModel()
    tr.train()
    tr.test()

if __name__=="__main__":
    tf.app.run()