LeNet 誕生於 1994 年，是最早的卷積神經網絡之一，並且推動了深度學習領域的發展。自從 1988 年開始，在許多次成功的迭代後，這項由 Yann LeCun 完成的開拓性成果被命名爲 LeNet5。
LeNet-5 出自論文 Gradient-Based Learning Applied to Document Recognition，是一種用於手寫體字符識別的非常高效的卷積神經網絡。Lenet-5 是 Yann LeCun 提出的，對 MNIST 數據集的分識別準確度可達 99.2%。
1、LeNet-5 Architecture
The LeNet-5 architecture consists of two sets of convolutional and average pooling layers, followed by a flattening convolutional layer, then two fully-connected layers and finally a softmax classifier.
1.1 First Layer
The input for LeNet-5 is a 32×32 grayscale image which passes through the first convolutional layer with 6 feature maps or filters having size 5×5 and a stride of one. The image dimensions changes from 32x32x1 to 28x28x6.
1.2 Second Layer
Then the LeNet-5 applies average pooling layer or sub-sampling layer with a filter size 2×2 and a stride of two. The resulting image dimensions will be reduced to 14x14x6.
1.3 Third Layer
Next, there is a second convolutional layer with 16 feature maps having size 5×5 and a stride of 1. In this layer, only 10 out of 16 feature maps are connected to 6 feature maps of the previous layer as shown below.
The main reason is to break the symmetry in the network and keeps the number of connections within reasonable bounds. That’s why the number of training parameters in this layers are 1516 instead of 2400 and similarly, the number of connections are 151600 instead of 240000.
1.4 Fourth Layer
The fourth layer (S4) is again an average pooling layer with filter size 2×2 and a stride of 2. This layer is the same as the second layer (S2) except it has 16 feature maps so the output will be reduced to 5x5x16.
1.5 Fifth Layer
The fifth layer (C5) is a fully connected convolutional layer with 120 feature maps each of size 1×1. Each of the 120 units in C5 is connected to all the 400 nodes (5x5x16) in the fourth layer S4.
1.6 Sixth Layer
The sixth layer is a fully connected layer (F6) with 84 units.
1.7 Output Layer
Finally, there is a fully connected softmax output layer ŷ with 10 possible values corresponding to the digits from 0 to 9.
2、Summary of LeNet-5 Architecture
3、Implementation of LeNet-5 Using Tensorflow2.0
3.1 導入相關包
import tensorflow as tf
from tensorflow.keras import Sequential, layers, losses, optimizers, datasets
from tensorflow.keras.callbacks import TensorBoard
3.2 加載數據集並進行預處理
數據預處理函數：
def preprocess(x, y):
"""
預處理函數
"""
x = tf.cast(x, dtype=tf.float32) / 255
x = tf.expand_dims(x, axis=3)
y = tf.cast(y, dtype=tf.int32)
y = tf.one_hot(y, depth=10)
return x,y
加載手寫數據集：
# 加載手寫數據集
(x, y), (x_test, y_test) = datasets.mnist.load_data()
print(x.shape, y.shape, x_test.shape, y_test.shape)
輸出：
(60000, 28, 28) (60000,) (10000, 28, 28) (10000,)
轉化爲Dataset數據集：
batchsz = 1000
# 轉化爲Dataset數據集
train_db = tf.data.Dataset.from_tensor_slices((x, y))
# 隨機打散
train_db = train_db.shuffle(10000)
train_db = train_db.batch(batchsz)
# 數據預處理
train_db = train_db.map(preprocess)
test_db = tf.data.Dataset.from_tensor_slices((x_test, y_test))
test_db = test_db.batch(batchsz).map(preprocess)
sample = sample = next(iter(train_db))
print('batch:', sample[0].shape, sample[1].shape)
print('batch:', sample[0].shape, sample[1].shape)
輸出：
batch: (1000, 28, 28, 1) (1000, 10)
batch: (1000, 28, 28, 1) (1000, 10)
3.3 創建網絡
model = Sequential([
layers.Conv2D(6, kernel_size=5, strides=1, activation="relu"), # Conv Layer 1
layers.MaxPool2D(pool_size=2, strides=2), # Pooling Layer 2
layers.Conv2D(16, kernel_size=5, strides=1, activation="relu"), # Conv Layer 3
layers.MaxPool2D(pool_size=2, strides=2), # Pooling Layer 4
layers.Flatten(), # flatten 層，方便全連接處理
layers.Dense(120, activation="relu"), # Fully connected layer 1
layers.Dense(84, activation="relu"), # Fully connected layer 2
layers.Dense(10) # Fully connected layer
])
打印網絡結構：
model.build(input_shape=(None, 28, 28, 1))
model.summary()
輸出：
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) multiple 156
_________________________________________________________________
max_pooling2d (MaxPooling2D) multiple 0
_________________________________________________________________
conv2d_1 (Conv2D) multiple 2416
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 multiple 0
_________________________________________________________________
flatten (Flatten) multiple 0
_________________________________________________________________
dense (Dense) multiple 30840
_________________________________________________________________
dense_1 (Dense) multiple 10164
_________________________________________________________________
dense_2 (Dense) multiple 850
=================================================================
Total params: 44,426
Trainable params: 44,426
Non-trainable params: 0
_________________________________________________________________
3.4 模型訓練與驗證
模型裝配：
model.compile(
optimizer=optimizers.Adam(lr=1e-3),
loss=tf.losses.CategoricalCrossentropy(from_logits=True),
metrics=['accuracy']
)
模型訓練：
model.fit(
train_db,
epochs=5,
validation_data=test_db,
validation_freq=2
)
輸出：
Train for 60 steps, validate for 10 steps
Epoch 1/5
60/60 [==============================] - 15s 253ms/step - loss: 1.0019 - accuracy: 0.7417
Epoch 2/5
60/60 [==============================] - 15s 246ms/step - loss: 0.2996 - accuracy: 0.9113 - val_loss: 0.2260 - val_accuracy: 0.9320
Epoch 3/5
60/60 [==============================] - 14s 232ms/step - loss: 0.2050 - accuracy: 0.9399
Epoch 4/5
60/60 [==============================] - 15s 243ms/step - loss: 0.1517 - accuracy: 0.9545 - val_loss: 0.1196 - val_accuracy: 0.9625
Epoch 5/5
60/60 [==============================] - 14s 228ms/step - loss: 0.1231 - accuracy: 0.9631
模型驗證：
model.evaluate(test_db)
輸出：
10/10 [==============================] - 1s 71ms/step - loss: 0.0971 - accuracy: 0.9701
轉載自： https://engmrk.com/lenet-5-a-classic-cnn-architecture/
