keras入門筆記2:我們來玩卷積層（ Convolution Layer）～一層卷積層分類Mnist！

哈囉哈囉～繼之前的：

keras深度學習入門筆記附錄1：讓我們看看有多少種讓搭建好的模型開始跑的方式（fit 和 train on batch）

https://blog.csdn.net/timcanby/article/details/103644089

從代碼案例入門keras1:LeNet對手寫數字Mnist分類

https://blog.csdn.net/timcanby/article/details/103620371

之後，小女子打算整理一個掃盲系列的第二個入門筆記：讓我們來用一層卷積層分類Mnist：

（搬走代碼給github小星星啊）

盆友們應該知道,全聯接層爲flatten後的數據注入類別信息，這就說明什麼呢 ，其實我只要多一點epochs，只用一層fc都可以玩mnist分類 。數據的導入代碼見（從代碼案例入門keras1:LeNet對手寫數字Mnist分類），模型部分的代碼如下：

model = Sequential()
model.add(Flatten(name='flatten',input_shape=(28, 28, 1)))
model.add(Dense(10,activation='softmax',name='fc1'))
model.compile(loss=keras.losses.categorical_crossentropy,
    optimizer=keras.optimizers.Adam(),
    metrics=['accuracy']

完整代碼：https://github.com/timcanby/Kares__StudyNotes/blob/master/oneFCTest.py

網絡的結構就是：

Layer (type)                 Output Shape              Param #   
=================================================================
  history=model.fit(x=X_train,y=Y_train, batch_size=batch_size,nb_epoch=n_epochs)
flatten (Flatten)            (None, 784)               0         
_________________________________________________________________
fc1 (Dense)                  (None, 10)                7850      
=================================================================
Total params: 7,850
Trainable params: 7,850
Non-trainable params: 0
_________________________________________________________________

如此簡單粗暴。換句話說就是數據讀進去就把它拉成一維然後餵給輸出是十個概率的softmax裝載全聯接層。這樣只訓練一道是肯定不行的比如epoch1:

Epoch 1/100

  100/60000 [..............................] - ETA: 1:06 - loss: 14.8443 - acc: 0.0700
 4200/60000 [=>............................] - ETA: 2s - loss: 13.9142 - acc: 0.1276  
 9200/60000 [===>..........................] - ETA: 1s - loss: 11.9051 - acc: 0.2503
14500/60000 [======>.......................] - ETA: 0s - loss: 10.2923 - acc: 0.3507
20700/60000 [=========>....................] - ETA: 0s - loss: 8.9224 - acc: 0.4362 
26400/60000 [============>.................] - ETA: 0s - loss: 8.0683 - acc: 0.4899
31100/60000 [==============>...............] - ETA: 0s - loss: 7.5861 - acc: 0.5201
37200/60000 [=================>............] - ETA: 0s - loss: 6.9932 - acc: 0.5568
43300/60000 [====================>.........] - ETA: 0s - loss: 6.5184 - acc: 0.5865
48900/60000 [=======================>......] - ETA: 0s - loss: 6.1924 - acc: 0.6069
54000/60000 [==========================>...] - ETA: 0s - loss: 5.9157 - acc: 0.6243
59900/60000 [============================>.] - ETA: 0s - loss: 5.6578 - acc: 0.6403
60000/60000 [==============================] - 1s 11us/step - loss: 5.6542 - acc: 0.6406

爲什麼不行？感受一下如果你心不在學習上給你看一遍的東西你能記住嗎？？哈哈哈哈所以我們如果在不去仔細觀察的前提下看得多了我們也可以記住，所以在100個epoch後我們可以看到：

  100/60000 [..............................] - ETA: 1s - loss: 0.9671 - acc: 0.9400
 3500/60000 [>.............................] - ETA: 0s - loss: 1.1487 - acc: 0.9280
 7000/60000 [==>...........................] - ETA: 0s - loss: 1.1750 - acc: 0.9256
10900/60000 [====>.........................] - ETA: 0s - loss: 1.1918 - acc: 0.9249
16100/60000 [=======>......................] - ETA: 0s - loss: 1.2007 - acc: 0.9242
20600/60000 [=========>....................] - ETA: 0s - loss: 1.1664 - acc: 0.9266
24300/60000 [===========>..................] - ETA: 0s - loss: 1.1546 - acc: 0.9272
27700/60000 [============>.................] - ETA: 0s - loss: 1.1421 - acc: 0.9280
31400/60000 [==============>...............] - ETA: 0s - loss: 1.1560 - acc: 0.9270
34700/60000 [================>.............] - ETA: 0s - loss: 1.1615 - acc: 0.9267
38000/60000 [==================>...........] - ETA: 0s - loss: 1.1517 - acc: 0.9273
41300/60000 [===================>..........] - ETA: 0s - loss: 1.1367 - acc: 0.9283
44200/60000 [=====================>........] - ETA: 0s - loss: 1.1420 - acc: 0.9279
47500/60000 [======================>.......] - ETA: 0s - loss: 1.1386 - acc: 0.9282
51200/60000 [========================>.....] - ETA: 0s - loss: 1.1340 - acc: 0.9285
56100/60000 [===========================>..] - ETA: 0s - loss: 1.1314 - acc: 0.9287
60000/60000 [==============================] - 1s 13us/step - loss: 1.1286 - acc: 0.9289

機器看了100遍以後表示記住了92.89%哈哈。當然今天的主題不是這個。我們今天來看的是當你有一點點學習慾望的時候如何學習知識😂

爲什麼這麼說呢，大家可能看到過說卷積層是個抽取局部細節的過濾器，它把圖像抽象到高度抽象的空間在進行分類：

圖片來自：https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

也是這篇文章裏說的一樣：The objective of the Convolution Operation is to extract the high-level featuressuch as edges, from the input image. 

就是說機器在嘗試把圖像的輪廓啊，顏色信息抽象到高緯再進行分類也就是，嘗試去用“腦子理解這張圖”了。所以我們今天就看看用一層的卷積來分類mnist，看看這個卷積究竟神奇在哪裏，今天的模型架構如下：

在剛剛的基礎下：from keras.layers import Conv2D

然後模型部分是：

model = Sequential()
model.add(Conv2D(16, kernel_size=(3,3), strides=(1, 1), padding='same', activation='tanh',name='conv',input_shape=(28, 28, 1)))
model.add(Flatten(name='flatten'))
model.add(Dense(10,activation='softmax',name='fc1'))
model.compile(loss=keras.losses.categorical_crossentropy,
    optimizer=keras.optimizers.Adam(),
    metrics=['accuracy'])

然後跑一下：可以看到只需要一個epoch，batchsize=100

  100/60000 [..............................] - ETA: 1:35 - loss: 2.5583 - acc: 0.0700
  800/60000 [..............................] - ETA: 15s - loss: 1.2045 - acc: 0.6087 
 1500/60000 [..............................] - ETA: 10s - loss: 0.8945 - acc: 0.7080
 2200/60000 [>.............................] - ETA: 8s - loss: 0.7685 - acc: 0.7582 
 2800/60000 [>.............................] - ETA: 7s - loss: 0.7062 - acc: 0.7779
 3500/60000 [>.............................] - ETA: 6s - loss: 0.6357 - acc: 0.8009
 4200/60000 [=>............................] - ETA: 6s - loss: 0.5920 - acc: 0.8174
 4900/60000 [=>............................] - ETA: 6s - loss: 0.5481 - acc: 0.8312
 5600/60000 [=>............................] - ETA: 5s - loss: 0.5217 - acc: 0.8389
 6300/60000 [==>...........................] - ETA: 5s - loss: 0.4976 - acc: 0.8460
 7000/60000 [==>...........................] - ETA: 5s - loss: 0.4744 - acc: 0.8530
 7700/60000 [==>...........................] - ETA: 5s - loss: 0.4502 - acc: 0.8614
 8400/60000 [===>..........................] - ETA: 4s - loss: 0.4379 - acc: 0.8670
 9000/60000 [===>..........................] - ETA: 4s - loss: 0.4287 - acc: 0.8698
 9600/60000 [===>..........................] - ETA: 4s - loss: 0.4154 - acc: 0.8735
10200/60000 [====>.........................] - ETA: 4s - loss: 0.4035 - acc: 0.8769
10900/60000 [====>.........................] - ETA: 4s - loss: 0.3906 - acc: 0.8808
11600/60000 [====>.........................] - ETA: 4s - loss: 0.3851 - acc: 0.8830
12300/60000 [=====>........................] - ETA: 4s - loss: 0.3759 - acc: 0.8863
13000/60000 [=====>........................] - ETA: 4s - loss: 0.3666 - acc: 0.8892
13700/60000 [=====>........................] - ETA: 4s - loss: 0.3585 - acc: 0.8915
14400/60000 [======>.......................] - ETA: 4s - loss: 0.3498 - acc: 0.8942
15000/60000 [======>.......................] - ETA: 4s - loss: 0.3448 - acc: 0.8957
15700/60000 [======>.......................] - ETA: 3s - loss: 0.3383 - acc: 0.8978
16400/60000 [=======>......................] - ETA: 3s - loss: 0.3334 - acc: 0.8995
17100/60000 [=======>......................] - ETA: 3s - loss: 0.3268 - acc: 0.9016
17800/60000 [=======>......................] - ETA: 3s - loss: 0.3210 - acc: 0.9035
18500/60000 [========>.....................] - ETA: 3s - loss: 0.3159 - acc: 0.9049
19200/60000 [========>.....................] - ETA: 3s - loss: 0.3125 - acc: 0.9062
19900/60000 [========>.....................] - ETA: 3s - loss: 0.3088 - acc: 0.9074
20600/60000 [=========>....................] - ETA: 3s - loss: 0.3038 - acc: 0.9091
21300/60000 [=========>....................] - ETA: 3s - loss: 0.3005 - acc: 0.9103
22000/60000 [==========>...................] - ETA: 3s - loss: 0.2969 - acc: 0.9113
22700/60000 [==========>...................] - ETA: 3s - loss: 0.2929 - acc: 0.9126
23400/60000 [==========>...................] - ETA: 3s - loss: 0.2887 - acc: 0.9139
24100/60000 [===========>..................] - ETA: 3s - loss: 0.2858 - acc: 0.9146
24800/60000 [===========>..................] - ETA: 2s - loss: 0.2823 - acc: 0.9156
25500/60000 [===========>..................] - ETA: 2s - loss: 0.2779 - acc: 0.9168
26200/60000 [============>.................] - ETA: 2s - loss: 0.2750 - acc: 0.9176
26900/60000 [============>.................] - ETA: 2s - loss: 0.2724 - acc: 0.9186
27600/60000 [============>.................] - ETA: 2s - loss: 0.2696 - acc: 0.9195
28300/60000 [=============>................] - ETA: 2s - loss: 0.2674 - acc: 0.9203
29000/60000 [=============>................] - ETA: 2s - loss: 0.2655 - acc: 0.9210
29700/60000 [=============>................] - ETA: 2s - loss: 0.2638 - acc: 0.9214
30500/60000 [==============>...............] - ETA: 2s - loss: 0.2614 - acc: 0.9222
31200/60000 [==============>...............] - ETA: 2s - loss: 0.2590 - acc: 0.9229
31800/60000 [==============>...............] - ETA: 2s - loss: 0.2574 - acc: 0.9235
32500/60000 [===============>..............] - ETA: 2s - loss: 0.2553 - acc: 0.9240
33200/60000 [===============>..............] - ETA: 2s - loss: 0.2523 - acc: 0.9249
33900/60000 [===============>..............] - ETA: 2s - loss: 0.2503 - acc: 0.9255
34500/60000 [================>.............] - ETA: 2s - loss: 0.2479 - acc: 0.9263
35200/60000 [================>.............] - ETA: 2s - loss: 0.2463 - acc: 0.9267
35900/60000 [================>.............] - ETA: 1s - loss: 0.2454 - acc: 0.9270
36600/60000 [=================>............] - ETA: 1s - loss: 0.2440 - acc: 0.9275
37300/60000 [=================>............] - ETA: 1s - loss: 0.2432 - acc: 0.9276
38000/60000 [==================>...........] - ETA: 1s - loss: 0.2430 - acc: 0.9280
38700/60000 [==================>...........] - ETA: 1s - loss: 0.2413 - acc: 0.9285
39400/60000 [==================>...........] - ETA: 1s - loss: 0.2393 - acc: 0.9290
40100/60000 [===================>..........] - ETA: 1s - loss: 0.2375 - acc: 0.9296
40800/60000 [===================>..........] - ETA: 1s - loss: 0.2355 - acc: 0.9302
41500/60000 [===================>..........] - ETA: 1s - loss: 0.2336 - acc: 0.9307
42200/60000 [====================>.........] - ETA: 1s - loss: 0.2319 - acc: 0.9311
42900/60000 [====================>.........] - ETA: 1s - loss: 0.2302 - acc: 0.9317
43600/60000 [====================>.........] - ETA: 1s - loss: 0.2284 - acc: 0.9323
44300/60000 [=====================>........] - ETA: 1s - loss: 0.2274 - acc: 0.9324
45000/60000 [=====================>........] - ETA: 1s - loss: 0.2257 - acc: 0.9328
45600/60000 [=====================>........] - ETA: 1s - loss: 0.2244 - acc: 0.9331
46300/60000 [======================>.......] - ETA: 1s - loss: 0.2234 - acc: 0.9334
47000/60000 [======================>.......] - ETA: 1s - loss: 0.2226 - acc: 0.9335
47700/60000 [======================>.......] - ETA: 1s - loss: 0.2212 - acc: 0.9338
48300/60000 [=======================>......] - ETA: 0s - loss: 0.2201 - acc: 0.9341
49000/60000 [=======================>......] - ETA: 0s - loss: 0.2187 - acc: 0.9345
49700/60000 [=======================>......] - ETA: 0s - loss: 0.2176 - acc: 0.9348
50400/60000 [========================>.....] - ETA: 0s - loss: 0.2163 - acc: 0.9351
51100/60000 [========================>.....] - ETA: 0s - loss: 0.2149 - acc: 0.9356
51800/60000 [========================>.....] - ETA: 0s - loss: 0.2135 - acc: 0.9359
52500/60000 [=========================>....] - ETA: 0s - loss: 0.2125 - acc: 0.9362
53200/60000 [=========================>....] - ETA: 0s - loss: 0.2117 - acc: 0.9366
53800/60000 [=========================>....] - ETA: 0s - loss: 0.2107 - acc: 0.9369
54500/60000 [==========================>...] - ETA: 0s - loss: 0.2100 - acc: 0.9372
55200/60000 [==========================>...] - ETA: 0s - loss: 0.2094 - acc: 0.9375
55900/60000 [==========================>...] - ETA: 0s - loss: 0.2088 - acc: 0.9378
56600/60000 [===========================>..] - ETA: 0s - loss: 0.2082 - acc: 0.9378
57300/60000 [===========================>..] - ETA: 0s - loss: 0.2072 - acc: 0.9382
58000/60000 [============================>.] - ETA: 0s - loss: 0.2063 - acc: 0.9384
58700/60000 [============================>.] - ETA: 0s - loss: 0.2053 - acc: 0.9387
59300/60000 [============================>.] - ETA: 0s - loss: 0.2043 - acc: 0.9389
59900/60000 [============================>.] - ETA: 0s - loss: 0.2033 - acc: 0.9393
60000/60000 [==============================] - 5s 82us/step - loss: 0.2030 - acc: 0.9394

所以關於卷積層裏的參數到底有什麼奧祕呢？

Conv2D(16, kernel_size=(3,3), strides=(1, 1), padding='same', activation='tanh',name='conv',input_shape=(28, 28, 1))

我覺得這裏最有疑問的應該就是第一第二個參數了，filter個數以及卷積核的大小。關於這兩個參數的學術意義，很多大佬們都有剖析過，比如：https://blog.csdn.net/jyy555555/article/details/80515562

我覺得作者分析的賊詳細所以，我們這個門在哪都還沒有找到怎麼入門的教程就看看在一層卷積層下這兩個參數之間的關係：

代碼：https://github.com/timcanby/Kares__StudyNotes/blob/master/20200101OneConvtest.py

代碼加入了計算時間的測量，（最後註釋的內容是可以看每一層的輸出，感興趣的朋友自行操作）這裏擺放2*2 3*3 4*4 5*5在1-100個filter下的計算時間和accuracy，以及accuracy最高時候的filter數：

 

	2*2	3*3	4*4	5*5
圖像

 

kernel=2
maxfilter 21
accMax 0.9319500026355187

kenel=3
maxfilter 16
accMax 0.9414833366622527

kernel=4
maxfilter 14
accMax 0.9419666689758499

kernel=5
maxfilter 9
accMax 0.9414000028371811

所以可見在一層的情況下，小核需要的filter數要稍微多一些會收穫比較好的效果，但是看個人的數據我覺得還是各有各的特點。當kernel=5的時候計算量會增大但是在小輸出的情況下就能有比較好的效果。

好的 多餘的分析盆友們可以自己去玩，今天就到這裏。（轉載請註明出處，github給小星星的是好青年）