paddle深度學習基礎之訓練調試與優化

前言

上一節咱們討論了四種不同的優化算法，這一節，咱們討論訓練過程中的優化問題。本次代碼修改模型全是在卷積神經網絡

文章目錄

五、可視化庫

網絡結構

優化思路

計算分類準確率，觀測模型訓練效果
檢查模型訓練過程，通過輸出訓練過程中的某些參數或者中間結果，識別潛在問題
加入校驗或測試，更好評價模型效果
加入正則化項，避免模型過擬合
可視化分析

一、計算模型的分類準確率

通過計算訓練的準確度，能夠比較直接的反應模型的精準程度。

在paddle框架中，我們可以使用自帶的準確率計算方法：

fluit.layers.accuracy(prediction,lable)

第一個參數是預測值，第二個參數是實際標籤值。下面是代碼中需要修改的地方：

    def forward(self, inputs,label):
        conv1 = self.conv1(inputs)
        pool1 = self.pool1(conv1)
        conv2 = self.conb2(pool1)
        pool2 = self.pool2(conv2)
        pool2 = fluid.layers.reshape(pool2, [pool2.shape[0], -1])
        outputs = self.linear(pool2)
        if label is not None:#添加
            acc = fluid.layers.accuracy(input=outputs,label=label)#添加
            return outputs,acc
        else:
            return outputs

輸出結果：

epoch: 0, batch: 0, loss is: [2.796657], acc is [0.04]
epoch: 0, batch: 200, loss is: [0.50403804], acc is [0.88]
epoch: 0, batch: 400, loss is: [0.2659506], acc is [0.92]
epoch: 1, batch: 0, loss is: [0.22079289], acc is [0.92]
epoch: 1, batch: 200, loss is: [0.23240374], acc is [0.92]
epoch: 1, batch: 400, loss is: [0.16370663], acc is [0.95]
epoch: 2, batch: 0, loss is: [0.37291032], acc is [0.92]
epoch: 2, batch: 200, loss is: [0.23772442], acc is [0.92]
epoch: 2, batch: 400, loss is: [0.18071894], acc is [0.95]
epoch: 3, batch: 0, loss is: [0.15938215], acc is [0.95]
epoch: 3, batch: 200, loss is: [0.21112804], acc is [0.92]
epoch: 3, batch: 400, loss is: [0.05794979], acc is [0.99]
epoch: 4, batch: 0, loss is: [0.24466723], acc is [0.93]
epoch: 4, batch: 200, loss is: [0.14045799], acc is [0.96]
epoch: 4, batch: 400, loss is: [0.12366832], acc is [0.94]

二、檢查模型訓練過程

在我們訓練模型時，時常會出現結果和我們預期有很大差距。此時，我們就想了解訓練過程中數據的變化過程。恰巧，paddle深度學習框架支持這些功能，我們一起去看看如何做的：

class MNIST(fluid.dygraph.Layer):
    def __init__(self):
        super(MNIST, self).__init__()
        # self.linear1 = Linear(input_dim=28*28,output_dim=10,act=None)
        # self.linear2 = Linear(input_dim=10,output_dim=10,act='sigmoid')
        # self.linear3 = Linear(input_dim=10,output_dim=1,act='sigmoid')
        self.conv1 = Conv2D(num_channels=1, num_filters=20, filter_size=5, stride=1, padding=2, act='relu')
        self.pool1 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
        self.conb2 = Conv2D(num_channels=20, num_filters=20, filter_size=5, stride=1, padding=2, act='relu')
        self.pool2 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
        self.linear = Linear(input_dim=980, output_dim=10, act='softmax')
    def forward(self, inputs,label,check_shape=False,check_content=False):
        conv1 = self.conv1(inputs)
        pool1 = self.pool1(conv1)
        conv2 = self.conb2(pool1)
        pool2 = self.pool2(conv2)
        pool21 = fluid.layers.reshape(pool2, [pool2.shape[0], -1])
        outputs = self.linear(pool21)
        # hidden1 = self.linear1(inputs)
        # hidden2 = self.linear2(hidden1)
        # outputs = self.linear3(hidden2)
        if(check_shape):
            print("\n------------打印各個層設置的網絡超參數的尺寸 -------------")
            print("conv1-- kernel_size:{}, padding:{}, stride:{}".format(self.conv1.weight.shape, self.conv1._padding, self.conv1._stride))
            print("conv2-- kernel_size:{}, padding:{}, stride:{}".format(self.conv2.weight.shape, self.conv2._padding, self.conv2._stride))
            print("pool1-- pool_type:{}, pool_size:{}, pool_stride:{}".format(self.pool1._pool_type, self.pool1._pool_size, self.pool1._pool_stride))
            print("pool2-- pool_type:{}, poo2_size:{}, pool_stride:{}".format(self.pool2._pool_type, self.pool2._pool_size, self.pool2._pool_stride))
            print("liner-- weight_size:{}, bias_size_{}, activation:{}".format(self.fc.weight.shape, self.fc.bias.shape, self.fc._act))

            print("\n------------打印各個層的形狀 -------------")
            print("inputs_shape: {}".format(inputs.shape))
            print("outputs1_shape: {}".format(conv1.shape))
            print("outputs2_shape: {}".format(pool1.shape))
            print("outputs3_shape: {}".format(conv2.shape))
            print("outputs4_shape: {}".format(pool2.shape))
            print("outputs5_shape: {}".format(outputs.shape))

        if check_content:
            # 打印卷積層的參數-卷積核權重，權重參數較多，此處只打印部分參數
            print("\n########## print convolution layer's kernel ###############")
            print("conv1 params -- kernel weights:", self.conv1.weight[0][0])
            print("conv2 params -- kernel weights:", self.conv2.weight[0][0])

            # 創建隨機數，隨機打印某一個通道的輸出值
            idx1 = np.random.randint(0, conv1.shape[1])
            idx2 = np.random.randint(0, conv1.shape[1])
            # 打印卷積-池化後的結果，僅打印batch中第一個圖像對應的特徵
            print("\nThe {}th channel of conv1 layer: ".format(idx1), conv1[0][idx1])
            print("The {}th channel of conv2 layer: ".format(idx2), conv1[0][idx2])
            print("The output of last layer:", conv1[0], '\n')
        if label is not None:
            acc = fluid.layers.accuracy(input=outputs,label=label)
            return outputs,acc
        else:
            return outputs

輸出結果：

------------打印各個層設置的網絡超參數的尺寸 -------------
conv1-- kernel_size:[20, 1, 5, 5], padding:[2, 2], stride:[1, 1]
conv2-- kernel_size:[20, 20, 5, 5], padding:[2, 2], stride:[1, 1]
pool1-- pool_type:max, pool_size:[2, 2], pool_stride:[2, 2]
pool2-- pool_type:max, poo2_size:[2, 2], pool_stride:[2, 2]
liner-- weight_size:[980, 10], bias_size_[10], activation:softmax

------------打印各個層的形狀 -------------
inputs_shape: [20, 1, 28, 28]
outputs1_shape: [20, 20, 28, 28]
outputs2_shape: [20, 20, 14, 14]
outputs3_shape: [20, 20, 14, 14]
outputs4_shape: [20, 20, 7, 7]
outputs5_shape: [20, 10]

三、加入校驗或者測試，更好的評價模型

通常在我們獲取數據集時，通常將數據集分成三個部分，分別是訓練集，校驗集，測試集。

訓練集 ：用於訓練模型的參數，即訓練過程中主要完成的工作。
校驗集 ：用於對模型超參數的選擇，比如網絡結構的調整、正則化項權重的選擇等。
測試集 ：用於模擬模型在應用後的真實效果。因爲測試集沒有參與任何模型優化或參數訓練的工作，所以它對模型來說是完全未知的樣本。在不以校驗數據優化網絡結構或模型超參數時，校驗數據和測試數據的效果是類似的，均更真實的反映模型效果。

四、加入正則化項，避免模型過擬合

過擬合現象

對於樣本量有限、但需要使用強大模型的複雜任務，模型很容易出現過擬合的表現，即在訓練集上的損失小，在驗證集或測試集上的損失較大，如圖2 所示。

[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-jGYNAyE0-1585745105788)(https://ai-studio-static-online.cdn.bcebos.com/e291cf2ea42947a7b671a71f862b96005270ef0f4b31478f9ef155581622b616)]

圖2：過擬合現象，訓練誤差不斷降低，但測試誤差先降後增

反之，如果模型在訓練集和測試集上均損失較大，則稱爲欠擬合。過擬合表示模型過於敏感，學習到了訓練數據中的一些誤差，而這些誤差並不是真實的泛化規律（可推廣到測試集上的規律）。欠擬合表示模型還不夠強大，還沒有很好的擬合已知的訓練樣本，更別提測試樣本了。因爲欠擬合情況容易觀察和解決，只要訓練loss不夠好，就不斷使用更強大的模型即可，因此實際中我們更需要處理好過擬合的問題。

導致過擬合原因

造成過擬合的原因是模型過於敏感，而訓練數據量太少或其中的噪音太多。

如圖3 所示，理想的迴歸模型是一條坡度較緩的拋物線，欠擬合的模型只擬合出一條直線，顯然沒有捕捉到真實的規律，但過擬合的模型擬合出存在很多拐點的拋物線，顯然是過於敏感，也沒有正確表達真實規律。

[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-jNca1VU7-1585745105789)(https://ai-studio-static-online.cdn.bcebos.com/53ffb08a6a4a4e92ac2e622953dceda225b0452418fe4835863d1814bbe4f744)]

圖3：迴歸模型的過擬合，理想和欠擬合狀態的表現

如圖4 所示，理想的分類模型是一條半圓形的曲線，欠擬合用直線作爲分類邊界，顯然沒有捕捉到真實的邊界，但過擬合的模型擬合出很扭曲的分類邊界，雖然對所有的訓練數據正確分類，但對一些較爲個例的樣本所做出的妥協，高概率不是真實的規律。

[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-7OfBRW1N-1585745105790)(https://ai-studio-static-online.cdn.bcebos.com/d3361a54e3524701ad993c2bb0e341d22c1277f575a7422798ed6fb7a4e5c626)]

圖4：分類模型的欠擬合，理想和過擬合狀態的表現

正則化項

前面咱們提到，過擬合現象就是因爲模型過於複雜，對數據太敏感。所以，我們在無法獲取更多數據時，只能降低模型的複雜度。

具體來說，在模型的優化目標（損失）中人爲加入對參數規模的懲罰項。當參數越多或取值越大時，該懲罰項就越大。通過調整懲罰項的權重係數，可以使模型在“儘量減少訓練損失”和“保持模型的泛化能力”之間取得平衡。泛化能力表示模型在沒有見過的樣本上依然有效。正則化項的存在，增加了模型在訓練集上的損失。

代碼實現

 optimizer = fluid.optimizer.AdamOptimizer(learning_rate=0.01, regularization=fluid.regularizer.L2Decay(regularization_coeff=0.1),
                                              parameter_list=model.parameters())

主要是通過這一句：

regularization=fluid.regularizer.L2Decay(regularization_coeff=0.1)

我們可以調節正則化項的權重，也就是regularization_coeff的值。

五、可視化庫

訓練模型時，經常需要觀察模型的評價指標，分析模型的優化過程，以確保訓練是有效的。可視化分析有兩種工具：Matplotlib庫和tb-paddle。

Matplotlib庫：Matplotlib庫是Python中使用的最多的2D圖形繪圖庫，它有一套完全仿照MATLAB的函數形式的繪圖接口，使用輕量級的PLT庫（Matplotlib）作圖是非常簡單的。
tb-paddle：如果期望使用更加專業的作圖工具，可以嘗試tb-paddle。tb-paddle能夠有效地展示飛槳框架在運行過程中的計算圖、各種指標隨着時間的變化趨勢以及訓練中使用到的數據信息。

Matplotlib 顯示

通常，我們會把模型訓練的的損失值和訓練次數存進列表中，再通過matplotlib 相關函數進行可視化顯示。

 if batch_id % 100 == 0:
                print("epoch: {}, batch: {}, loss is: {}, acc is {}".format(epoch_id, batch_id, avg_loss.numpy(), acc.numpy()))
                iters.append(iter)
                losses.append(avg_loss.numpy())
                iter = iter + 100

#畫出訓練過程中Loss的變化曲線
plt.figure()
plt.title("train loss", fontsize=24)
plt.xlabel("iter", fontsize=14)
plt.ylabel("loss", fontsize=14)
plt.plot(iters, losses,color='red',label='train loss') 
plt.grid()
plt.show()

[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-sizcOGq6-1585745105794)(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYkAAAEjCAYAAADHWv01AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XeYlNXd//H3F1iWsgoqunTBHlsUcNFoEI0aOxpTNMYaRVNMHtErsaLxSSy/RH00+jxKbNHYkmgiKhY0rCWKCooFFEEhiqCiILB02O/vj3MPOzs7s0zbe3Z3Pq/rmmtm7rnLObOz85lzzl3M3REREUmnQ6kLICIirZdCQkREMlJIiIhIRgoJERHJSCEhIiIZKSRERCQjhYRICjO7y8zczC6PcZsjo23OjWubItnoVOoCiCSY2anAIOCf7j6ttKUREVBISOtyKrA/MBcoZUgsAGYCX5SwDCKtgkJCJIW7XwhcWOpyiLQGGpMQEZGMFBJScmZ2qpk5oasJ4M5oENdTB3MT85pZbfT8RDN7zsy+jKYfE03vaGaHmdmtZjbVzD4zszVmNt/M/mFmBzZTnrQD12Y2KFGm6PmuZvaAmX1qZqvM7D0zu9TMOhf1DWrY/gFm9nC0vTXR/cbqsklUpqlmtizpPZhiZr83s13TLLO/mf3dzOZF8y8xs1lm9k8zO8vM9L1RRtTdJK3BSuAzYHOgAlgaTUtYmG4hM7sROAeoB5ZE9wlfAyYkPV8KrAH6AMcAx5jZRe5+VT4FNrNDgH8CXaNtVwA7AlcAQ6NtFI2Z/Ra4OHrq0Ta3oqEuV0fdZMnL9ABeAnaOJiXep2rC+zAUWA9ckLTMaODWpNWsADoC20W3UcCfgVVFrJ60YvpFICXn7g+6e2/CFxrAL929d9JtrzSLDQV+DlwGbOHumwObJa1jDXAH8G2gh7v3cPcqwhfkpYQvx9+Z2fA8i/0g8Cgw2N17ApsSxjEcGGVmh+e53ibM7HgaAuImYCt33wzYEvhjNP0CM/tRyqK/JATEQuBIoDJ6n7oAOxDC4YOk7XQDro2e3gEMdPfu0fu2BXAYcD+Nw1jaO3fXTbdWcQNqCV+ypzYzz6nRPA5cWcC2Lo3WcWea1+6KXrs8ZfqgpG0/DViaZR+NXr8jx/KMjJabmzLdgFnRa/dnWPa+6PU5QIek6ROi6b/Osgw10fx1QMdSfx50ax03tSSkrVoPXFfA8o9G9/vmufzV7p7uYiz/jO6b9PXnaQ9CNw/AbzPM85vofhDhiz5haXTfJ8ttJeavILQcRBQS0mbNdvdmj2Mws65mdq6Z1ZrZ52a2Nmng+Y1otr55bv+1DNM/ie43y3O9qYZE9wvdfXq6Gdx9ZtJ2hyS9lBiT+YWZ3RMN5G/SzLZmRbfOwMvRe7eTmVkB5Zc2TiEhbVXawewEM+tDOCDvOsJeU1sCq6PlPqPhQLnu+Wzc3ZdleCkxoFuRz3rT2DK6/6TZuWBeyvy4+93AOEKX1Y8IofGVmb1hZldE7xFJ868HfhhtaxvCe/cu8IWZ/c3MjlZglB+FhLRV6zfy+v8QBmc/BI4DNnf3KnffysMg+d4tXcAi65LPQu5+FqHr6wrCmM9qQhfWpcAsMzs4Zf4pwPaEULmb8P5tDnwXeAR43Mw65lcFaYsUEtLuRMcpjIqenujuD7v74pTZqmMuVr4SLaYBG5mvf8r8G7j7dHe/zN0PAHoCRwFvE1pRfzazipT5V7r7ve5+irtvS2hVXEUY1D4MODvv2kibo5CQ1iSxa2WhXRq9gMro8RsZ5jmowG3E5fXovruZ1aSbwcx2APqlzJ+Wu69x98eA70WT+hBaDs0tM8fdLyLs9gsNBz1KGVBISGuS2LumZ4HrWUb41QuwW+qLUV/8OQVuIy7TgNnR44syzHN5dD8XeDUxcSNHficfrFiZxfzJy1Q2O5e0KwoJaU0Se+98JzpaOC/RoPLk6OkdZrYHgJl1MLNvAc9ReGslFtFutpdET0eZ2R/NbAsAM9siOur8hOj1S9w9+UC3Z8zsRjMbYWZdExPNbBfCsSAQznj7dvT4cDN72czONLOtk+bvZmZnAidGk54qZh2lddNpOaQ1uQc4H9iPsEfN58BaYJ6775fjus4FJhFaEm+Y2XLCj6KuwCLgdBqOaWjV3P1BM9uNcNT1z4GfmtkSoAcNP/Sudvd7UxbdlNBiOgeoj5bpSsMg+ArgJHdfl7TM3tENM1tJ2FurJw2hOoGwx5SUCbUkpNVw9/eAg4EnCecY6g1sTcOgbC7regXYhxAEiwm7pH5OOC/RHsCbxSl1PNz9EuBbhD2MvgCqgC+B8cBBnnLepsgZhNOWTAI+IgQEwHuE03vs6u7PJs3/L+AkwrmZ3iaEyCbRdiYCJwNHpYSKtHOW/qBRERERtSRERKQZCgkREclIISEiIhkpJEREJKPYdoE1swGEc8FUEw50GufuN6TMM5Kw98acaNLD7n5Fc+vt1auXDxo0KK8yLV++nO7d8zq/W5tXrnVXvcuL6p3Z1KlTv3D3LZudiXiPk1gHnOfur0enK55qZhPdfUbKfC+4+5HZrnTQoEFMmTIlrwLV1tYycuTIvJZt68q17qp3eVG9MzOz/2Szrti6m9x9gbu/Hj1eRjgFcb/mlxIRkVIqyXESZjYIeJ5wMM/SpOkjgYcI58afD5yf7kIr0cXaRwNUV1cPfeCBB/IqR11dHVVVVXkt29aVa91V7/Kiemd2wAEHTHX3YRtdWdzXSyUcKToV+E6a1zYFqqLHhwOzNra+oUOHer4mTZqU97JtXbnWXfUuL6p3ZsAUb23XuI7OW/8QcK+7P5z6ursvdfe66PEEoMLMesVZRhERaRBbSESXPbwdeNfd017A3sx6Jy6PGJ07vwPhvDEiIlICce7dtC/h5GFvm9m0aNpFwEAAd7+FcInEn5jZOsK564+PmkUiIlICsYWEu7/IRs7h7+43Ec5OKSIirUD5HnH99tsMvv12+OKLUpdERKTVKt+QeP99tv7LX2D+/FKXRESk1SrfkEgcsl5XV9pyiIi0YuUbEokDTZYvL205RERaMYWEWhIiIhkpJBQSIiIZlW9IJMYk1N0kIpJR+YaEWhIiIhtVviHRrVu4V0iIiGRUviHRsSPrKysVEiIizSjfkADWd+2qMQkRkWYoJNSSEBHJSCGhkBARyai8Q6JLF3U3iYg0o7xDQi0JEZFmKSQUEiIiGSkkFBIiIhmVdUjUa0xCRKRZZR0SakmIiDRPIbFiBdTXl7ooIiKtUnmHRJcu4cGKFaUtiIhIK1XeIdG1a3igLicRkbQUEqCQEBHJQCEBCgkRkQzKOyQSYxLaDVZEJK3yDgm1JEREmqWQAIWEiEgG5R0S6m4SEWlWeYeEWhIiIs1SSIBCQkQkg7IOifrKSjBTSIiIZFDWIUGHDtCtm8YkREQyiC0kzGyAmU0ysxlmNt3MfplmHjOzG81stpm9ZWZDWrxgVVVqSYiIZNApxm2tA85z99fNbBNgqplNdPcZSfMcBmwf3YYD/xfdtxyFhIhIRrG1JNx9gbu/Hj1eBrwL9EuZbRRwtweTgZ5m1qdFC9a9u7qbREQyKMmYhJkNAvYEXkl5qR/wcdLzeTQNkuJSS0JEJKM4u5sAMLMq4CHgv9x9aZ7rGA2MBqiurqa2tjavstTV1bFozRo6LV7M63muo62qq6vL+31ry1Tv8qJ6Fy7WkDCzCkJA3OvuD6eZ5RNgQNLz/tG0Rtx9HDAOYNiwYT5y5Mi8ylNbW8vmAwfCzJnku462qra2tuzqDKp3uVG9Cxfn3k0G3A686+7XZZhtPHBytJfT3sASd1/QogXTmISISEZxtiT2BU4C3jazadG0i4CBAO5+CzABOByYDawATmvxUmlMQkQko9hCwt1fBGwj8zjws3hKFFFIiIhkVN5HXEPoblq1CtavL3VJRERaHYVEVVW417iEiEgTColESKjLSUSkCYWEQkJEJCOFRPfu4V7dTSIiTSgk1JIQEclIIaGQEBHJSCGR6G5SSIiINKGQ0C6wIiIZKSTU3SQikpFCQiEhIpKRQqJLFzBTd5OISBoKCTOd5E9EJAOFBCgkREQyUEiAQkJEJAOFBOjqdCIiGSgkQC0JEZEMFBKgkBARyUAhAepuEhHJQCEBakmIiGSgkACFhIhIBgoJUEiIiGSgkIAwJrFmDaxdW+qSiIi0KgoJ0OnCRUQyUEiAzgQrIpKBQgIark6nloSISCMKCVBLQkQkA4UEKCRERDJQSIBCQkQkA4UEaExCRCQDhQSoJSEikoFCAhQSIiIZKCRA3U0iIhnEFhJmdoeZfW5m72R4faSZLTGzadFtbFxlo7ISOnZUS0JEJEWnGLd1F3ATcHcz87zg7kfGU5wkZjrJn4hIGrG1JNz9eWBRXNvLmUJCRKSJOFsS2djHzN4E5gPnu/v0dDOZ2WhgNEB1dTW1tbV5bayurm7DsjUdOlA3Zw4z8lxXW5Nc93KiepcX1bsI3D22GzAIeCfDa5sCVdHjw4FZ2axz6NChnq9JkyY1PBkyxP2II/JeV1vTqO5lRPUuL6p3ZsAUz+I7ttXs3eTuS929Lno8Aagws16xFUDdTSIiTbSakDCz3mZm0eMaQtm+jK0A3btrF1gRkRSxjUmY2f3ASKCXmc0DLgMqANz9FuC7wE/MbB2wEjg+ahLFo6oK5syJbXMiIm1BbCHh7ids5PWbCLvIloa6m0REmmg13U0lp5AQEWlCIZGgMQkRkSYUEglVVbB2LaxZU+qSiIi0GgqJBJ0JVkSkCYVEgs4EKyLShEIiQS0JEZEmFBIJCgkRkSYUEgkKCRGRJnIKCTPb0sy2THq+m5n91syaPVCuTdCYhIhIE7m2JP4KHAUQnXzveeBY4BYzO6/IZYuXWhIiIk3kGhK7A5Ojx98FZrv7LsDJwFnFLFjsFBIiIk3kGhJdgcS36EHA+Ojx68CAYhWqJNTdJCLSRK4hMQv4jpkNAA4Bno6mVwNfFbNgsVNLQkSkiVxD4jfANcBcYLK7vxJN/zbwRhHLFb/OnaGiQiEhIpIkp1OFu/vDZjYQ6Au8mfTSM8BDxSxYSehMsCIijeR8PQl3/wz4LPHczLYD3nT3VcUsWEnoTLAiIo3kepzElWZ2SvTYzGwi8D6wwMyGt0QBY6WWhIhII7mOSZwIzIweHwbsAewN3A1cXcRylYZCQkSkkVy7m6qBedHjw4G/uvurZrYImFLUkpVCVZW6m0REkuTakvgS2Dp6fAjwbPS4E2DFKlTJdO+uloSISJJcWxIPAfeZ2fvA5sBT0fQ9gNnFLFhJqLtJRKSRXENiDPAfYCDwK3dP9M30Af6vmAUrCYWEiEgjuR4nsQ64Ns3064tWolLSLrAiIo3kfJyEmVUDPwN2BhyYAdzs7p8XuWzxS7Qk3MHa/hCLiEihcj1OYl/C2MMPgZXAKsJusbPNbJ/iFy9mVVWwfj2sXl3qkoiItAq5tiT+ANwPnO3u9QBm1gG4hdAN9Y3iFi9mySf569KltGUREWkFct0Fdg/g2kRAAESPrwP2LGbBSkKnCxcRaSTXkFgCDE4zfTBt/VThoNOFi4ikyLW76QHgdjP7FfBSNG1fwunD7y9mwUpCISEi0kiuIfErwpHVd9BwlPUawjESFxS3aCWg7iYRkUZyPU5iDfBLM7sQ2Daa/IG7ryh6yUpBLQkRkUY2GhJmNj6LeQBw96OLUKbSUUiIiDSSTUviy2JsyMzuAI4EPnf3XdO8bsANhLPLrgBOdffXi7HtrCkkREQa2WhIuPtpRdrWXcBNhGtPpHMYsH10G04Y54j3QkYakxARaSTXXWDz5u7PA4uamWUUcLcHk4GeZtYnntJFEiGhloSICJDHuZtaUD/g46Tn86JpC1JnNLPRwGiA6upqamtr89pgXV1dk2VHVFQwb8YMPsxznW1FurqXA9W7vKjehWtNIZE1dx8HjAMYNmyYjxw5Mq/11NbW0mTZTTZh4BZbMDDPdbYVaeteBlTv8qJ6Fy627qYsfAIMSHreP5oWL11TQkRkg9YUEuOBky3YG1ji7k26mlqcQkJEZIPYupvM7H5gJNDLzOYBlwEVAO5+CzCBsPvrbMIusMXaqyo3CgkRkQ1iCwl3P2EjrzvhYkalpavTiYhs0Jq6m1oHtSRERDZQSKRSSIiIbKCQSKXuJhGRDRQSqdSSEBHZQCGRKhES7qUuiYhIySkkUlVVhYBYubLUJRERKTmFRCqdCVZEZAOFRCpdU0JEZAOFRCqFhIjIBgqJVOpuEhHZQCGRSi0JEZENFBKpFBIiIhsoJFIlQuKrr0pbDhGRVkAhkWrgQOjdG+6+u9QlEREpOYVEqspKuOACmDQJnnuu1KURESkphUQ6o0dDnz5w2WWlLomISEkpJNLp2hUuvDC0JCZNKnVpRERKRiGRyZlnQr9+MHasTvYnImVLIZFJly6hNfHii/Dss6UujYhISSgkmnPGGdC/fxibUGtCRMqQQqI5lZVw0UXw0kswcWKpSyMiEjuFxMacfjoMGKDWhIiUJYXExlRWwsUXw+TJ8NRTpS6NiEisFBLZOO002HprtSZEpOwoJLLRuXNoTbz6KjzxRKlLIyISG4VEtk49NZzT6Z57Sl0SEZHYKCSyVVEB++0XWhMiImVCIZGLmhr48ENYuLDUJRERiYVCIhfDh4f7114rbTlERGKikMjFkCHQoQO88kqpSyIiEguFRC6qqmDXXTUuISJlQyGRq5qaEBI6XkJEyoBCIlc1NbBoEXzwQalLIiLS4mINCTM71MxmmtlsM7sgzeunmtlCM5sW3c6Is3xZSQxeq8tJRMpAbCFhZh2Bm4HDgJ2BE8xs5zSzPujue0S32+IqX9Z23hm6ddPgtYiUhThbEjXAbHf/0N3XAA8Ao2LcfnF06gRDh6olISJloVOM2+oHfJz0fB4wPM18x5nZCOB94Fx3/zh1BjMbDYwGqK6upra2Nq8C1dXV5bXsNn370v/hh3lh4kS8oiKvbZdavnVv61Tv8qJ6F4G7x3IDvgvclvT8JOCmlHm2ACqjx2cB/9rYeocOHer5mjRpUn4L/vWv7uD+2mt5b7vU8q57G6d6lxfVOzNgimfx3R1nd9MnwICk5/2jaRu4+5fuvjp6ehswNKay5UaD1yJSJuIMideA7c1ssJl1Bo4HxifPYGZ9kp4eDbwbY/myN2AAVFdr8FpE2r3YxiTcfZ2Z/Rx4CugI3OHu083sCkKzZzzwCzM7GlgHLAJOjat8OTFrOKhORKQdi3PgGnefAExImTY26fGFwIVxlilvw4fDo4/CkiXQo0epSyMi0iJ0xHW+amrCvc4IKyLtmEIiX3vtFe7V5SQi7ZhCIl89e8KOO2rwWkTaNYVEIWpqQkjojLAi0k4pJAoxfDh89hnMm1fqkoiItAiFRCESg9fqchKRdkohUYjdd4fOnTV4LSLtlkKiEJWVsOeeCgkRabcUEoWqqYEpU2D9+vSva1BbRNowhUShhg+H5cthxoyGafX14WjsAw4I53l6663SlU9EpAAKiUIlD16vXAm33hquXnf00eE62O4hLN54o7TlFBHJg0KiUNttB5ttBtdfDwMHwtlnQ1UV3HdfCIkXXgjPDzwwdEuJiLQhColCmcE3vwnvvgvf+AY891w4n9MJJ0BFBWyzTZjWsyccdFD+u8uuW6fxDRGJnUKiGO68E+bOhUcegREjQnAkGzQoBMUWW8DBB8NLL+W2/vfeg+23hyOOgLVri1VqEZGNUkgUw+abh66m5gwcGIKid2/49rfhxRezW/crr8B++8HixfDEE3DWWS3Toli7NnSZvflm8dctIm2WQiJO/ftDbS306xeC4tprYfXqzPM/+WQYy+jRA6ZOhbFjQ6vld78rbrkWLQrlGTMmBNIzzxR3/SLSZikk4ta3bwiKESPg/PPDnlB/+1vT1sFf/gJHHQU77BC6p7bdFi6/HE46CS69NLxeDO+/D3vvDf/+N9xwAwweDIcfDg8+WJz1i0ibppAohd69Q9fRU09Bt27w/e+HX/CTJ4fXr7suhME3vxm6qKqrw3QzuO22sEvt6aeHsEnHPQTLH/8Is2dnLsezz4bjPBYvDo9/8Qt4/vkQGiecEJaX7H34IZxzDsyfX+qSiBSNQqKUDjkEpk2DcePC7rL77BO+oM87D447DiZMgE03bbxM587w0ENh19tjjw17VSXU1YXjNPbcE/bdN3zpb789DBsWuraSzlbbZ/z40MXUr184rch++4UXevYM4XX00WH5Sy7RXlXZmD49vIc33RTeuxUrSl0ikaJQSJRax45w5pkwa1b4Qn77bfjpT0N3T5cu6ZfZbLMQIJWVcNhh4df/OeeErqyzzw7z3HorzJwJf/hDaIGcf344+nvECDjxRHa8/voQUi+9FLqYknXtCn//O5xxRhj/GD067IIr6U2ZAvvvH8L02mvh9dfhlFPCkfcihVi+HK65Br74onRlcPc2fRs6dKjna9KkSXkv22LWrs1+3tdec+/WzR3cO3d2/9GP3F96yb2+vum877/v/t//7b7zzu7gHx933Ma3VV/vfsklYf3f/GZYd1u0fr377be777ijL/76190vvdT96afdly0rfN21te6bbOI+aJD77Nlh2u9/H96zsWMLX3+RtMrPegzS1nvpUvfPPou9LHk57bTwWfrxj3NaLJu/NzDFs/iOLfmXfKG3dhcSuXruOffrrnP//PPsl6mry63ud9zhvtVW4eNy5JHu06blXMy8TZjg/uST+S//8svue+0Vyj5smC/ZcUf3Dh3C844d3Wtq3M87z33mzNzX/fjj7l26uH/ta+7z5jVMr69v+Oe+7778y15ERfmsf/KJ+403ui9YUPi6YtKk3h995L7NNu59+7rX1ZWkTFm7997wGRo0KHxmp0/PelGFhEKiYDnXfdky9yuvdO/ZM3xsfvAD9/fea3h96VL3qVPd77/f/Yor3M89N/zKXr8+vwLW14ftgXtFhfu//53b8vPnu598cli+b1/3v/zFvb4+1Hvp0hA8F18cWkidO7tvuaX7jBnZr/+BB9w7dXIfMiR9QK9aFdZdWek+eXJuZW8BBX/Wn346vEcQ6nTWWQ0tp1asUb0TAdG9e6jHFVeUrFwbNXt2aKHuu6/7p5+6b7qp+9FHZ724QkIhUbC86754cfhy7d49/LoZPty9T5/wUUq+VVY2/Aq69FL3WbOy38bate5nntkQRttsE7aRzS/YtWvdr7nGvaoqfPlfeGGjbqW09Z450726OoTJBx80v/76evcbbnA3CyHw1VeZ51240H3w4LDujz5q+vr69aEFsnr1xutVoLz/3uvWuV92WajvLru4P/VUCIjOncPf//jj3d94o5hFLaoN9U4ExKabur/yivuxx4bPSGvsdlq92n3YsPCDbO7cMC3xg+mFF7JahUJCIVGwguv+2WfuY8aEL8rTTnO/6ir3v//d/a233FescF++PPx6P/jg8AUD4VfRuHHuixZlXu/Spe6HHhrmv+ii8EX65pvuXbu677ef+5o1mZddvjx0h4H7UUelDaaM9X77bffNNw+hlu4LPbH+H/0orH/UqPB8Y6ZPD19MX/+6+223uf/61+ELatddQ1cVuG+7bWiFtaC8/t6ffur+rW+FMp58cuPumfnzQ1022SS8fuihoWuvlZk0aVLTgHAPreCOHd1/9rOSli+t888P7+lDDzVMW748/FD6xjfSjzmmUEgoJAoWa90//jiEyE47+YZB9lGj3B98sPEX7bx57nvsEf55x41rvI777gvL/uIX6bfx5Zfu++wTAul//zdjUZqt95Qp4Ytkhx3CF2SyDz4IX/RmYQeAXLrRnniiYRykosJ9xx1DmI0ZE8aT+vcP78nNN2f1BZCz9ev95fvvD11sN9zg/pOfuB94YNju4MHuxxzjfvnl7v/8p/ucOaEMtbXuvXuHILv99szlWrw4/G0TXVFHHx1+KLQSLz34YNOASDj77NBl+P77pSlcOk88Ed7Hs89u+tq4ceG1f/xjo6tRSCgkClaSutfXu7/6ahivSHRRVVWFX+d//nP40qqqCv8o6Zx7bljmnnsaT//oozB43LlzaM00Y6P1fuGFsMfYbruF4HEPX66bbRaa/48/nl1dU330UQiadHuULVzofvjhoW7f+17zXVjZ+uKL8D59//vuPXp4o67AHj3CgP1JJ4XXd9ihobUHoZ4dOrhvv31oxWVj2TL33/0urNvM/cQTSz9m8dFHvqJv3/QB4R66L7t3D+95rpYtC+Nkf/pT8UJxwYKwg8iuu4bWeKq1a8MPrZ122uieiQoJhUTBSl73devc//Uv9zPOaBgM79u3+f7tNWvc998/dD0l5ps+PYTLppu6Z1GnrOo9cWIInL32cv/Nb8KX3m67teyX3vr1YSylY8f8up/q693fecf96qtDt1yi5VJd7X766f7emDFhT7hPP03fKqirC91Ft9wSfsWOGeO+ZEnu9fjyS/cLLgh/o06dwvhF8p5fLWXlyrBL+K23hm3W1LhXVvra7t3TB0TC2LHhfWpu54L6evdnngktph/8oGmogvtBB4U98fLdUWP9+rCOrl3D3zGTf/wjbO9Pf2p2dQoJhUTBWlXdV60KgZHNbryffurer1/oJnnssfALv3fvrHfLzbre48eHLzlwP+GE+HaXfPHFhu6niy92f+SRMLCe7pfjnDmhK+jEExvvPLDnnmFngVdf3fClFfvfe/5895/+NLyHXbqE8YvmxqLysX59aDnW1DT8rRItpQMOcB8zxl+9/fbm17F0afj1PmJE+vD89NOGMbLEjhjHHBN+PIwf7/7uuyE8+vYNr++0UwjabMar3ENX1+WXh+CBpt2sqerrQ7dq377NbkMhoZAoWJuu++TJ4UsU3Lfbzv3DD7NeNKd6P/WU+513tsw4QXMWLgx9+8m/VCsqQpfasceGQeTBgxteq64OQTZuXBj/SaNkf+8PPgghZhZajFdfnf0XaCb19eELeo89Gr6YL7rI/W9/C9tL+ntlVe+bbw7refTRxtMfeyyMtXTpEsZymgu51avDjhpDhoR1bbFF6M4bOzYcZzRpUgj1tWtDt9L11zccv2PmPnJbU4mKAAAI4UlEQVRk6HLN5rP2/PNhuSuvzDiLQkIhUbA2X/d77w2/6HLchbFN1Xvx4hCId90VunCOOSZ8IW61VXh8442ha6JIe7u0qGnTGsZd+vYN3ULN7amWTn19CO6aGt+wV9g994SuywyyqveaNWH8ZZddwrpWrAh7PYH77rs33/2TrozPPef+ne+EFmFqt1THjg1dgXvu6f6HP2QM9mYddVToYv3ii7QvFzMkOpXobCAihfnhD8OtPevZM5yld/jwUpekcF//Ojz+eDjP2IUXhotnnXNOOCNynz6Nbz17wqpV4SSJK1eG+xUrYMYMePnlcAGv226Dk08OlwguVEUFXHklfO97cPHF8OijYVvnnhumZzqHWjpm4fxoI0aE52vWwMcfhytXzpkT7jt3Dtv62tfyL/NVV8Huu4fyXXtt/uvJQqwhYWaHAjcAHYHb3P3qlNcrgbuBocCXwA/cfW6cZRSRFjRiRLgq44QJITAWLAi3WbPC80WLGs9fWRlOONmtW7gC5M03w49/HKYX03HHhTC+5poQXE8+Gc6SXKjOncO1YLbdtvB1JdtlF7jllnBRshYWW0iYWUfgZuBgYB7wmpmNd/cZSbP9GFjs7tuZ2fHANcAP4iqjiMTALFyv/Ygjmr62ejUsWxaCoUuXcJbkuMp0223hyo8XXABbbhnPdgtx5pmxbCbOU4XXALPd/UN3XwM8AIxKmWcU8Ofo8d+Bb5mZxVhGESmlykro1Qu6d48vIBJ23TV03bSFgIhRnN1N/YCPk57PA1I7WzfM4+7rzGwJsAXQ6GTqZjYaGA1QXV1NbaYrtG1EXV1d3su2deVad9W7vKjehWuTA9fuPg4YBzBs2DAfOXJkXuupra0l32XbunKtu+pdXlTvwsXZ3fQJMCDpef9oWtp5zKwT0IMwgC0iIiUQZ0i8BmxvZoPNrDNwPDA+ZZ7xwCnR4+8C/4r25xURkRKIrbspGmP4OfAUYRfYO9x9upldQTioYzxwO3CPmc0GFhGCRERESiTWMQl3nwBMSJk2NunxKuB7cZZJREQyi7O7SURE2hiFhIiIZGRtfVzYzBYC/8lz8V6kHINRRsq17qp3eVG9M9va3Td65GCbD4lCmNkUdx9W6nKUQrnWXfUuL6p34dTdJCIiGSkkREQko3IPiXGlLkAJlWvdVe/yonoXqKzHJEREpHnl3pIQEZFmKCRERCSjsg0JMzvUzGaa2Wwzu6DU5SmUmd1hZp+b2TtJ0zY3s4lmNiu63yyabmZ2Y1T3t8xsSNIyp0TzzzKzU9JtqzUxswFmNsnMZpjZdDP7ZTS9XdfdzLqY2atm9mZU799E0web2StR/R6MTqaJmVVGz2dHrw9KWteF0fSZZlaEa3a2PDPraGZvmNlj0fN2X28zm2tmb5vZNDObEk1r+c+5u5fdjXCCwQ+AbYDOwJvAzqUuV4F1GgEMAd5Jmvb/gAuixxcA10SPDweeAAzYG3glmr458GF0v1n0eLNS120j9e4DDIkebwK8D+zc3uselb8qelwBvBLV56/A8dH0W4CfRI9/CtwSPT4eeDB6vHP0+a8EBkf/Fx1LXb8s6j8GuA94LHre7usNzAV6pUxr8c95ubYksrmUapvi7s8TzpybLPlysH8GjkmafrcHk4GeZtYH+DYw0d0XuftiYCJwaMuXPn/uvsDdX48eLwPeJVzhsF3XPSp/XfS0Iro5cCDh0r/QtN7pLg08CnjA3Ve7+xxgNuH/o9Uys/7AEcBt0XOjDOqdQYt/zss1JNJdSrVficrSkqrdfUH0+FOgOnqcqf5t+n2JuhL2JPyqbvd1j7pcpgGfE/7ZPwC+cvd10SzJdWh0aWAgcWngNldv4H+AXwH10fMtKI96O/C0mU21cAlniOFz3iYvXyq5c3c3s3a7v7OZVQEPAf/l7kvDj8Wgvdbd3dcDe5hZT+AfwE4lLlKLM7Mjgc/dfaqZjSx1eWK2n7t/YmZbARPN7L3kF1vqc16uLYlsLqXaHnwWNTGJ7j+Ppmeqf5t8X8ysghAQ97r7w9Hksqg7gLt/BUwC9iF0KyR+/CXXIdOlgdtavfcFjjazuYRu4gOBG2j/9cbdP4nuPyf8KKghhs95uYZENpdSbQ+SLwd7CvBI0vSToz0g9gaWRE3Wp4BDzGyzaC+JQ6JprVbUv3w78K67X5f0Uruuu5ltGbUgMLOuwMGE8ZhJhEv/QtN6p7s08Hjg+GgvoMHA9sCr8dQid+5+obv3d/dBhP/bf7n7ibTzeptZdzPbJPGY8Pl8hzg+56UesS/VjTD6/z6hH/fiUpenCPW5H1gArCX0M/6Y0Pf6LDALeAbYPJrXgJujur8NDEtaz+mEQbzZwGmlrlcW9d6P0Ff7FjAtuh3e3usO7A68EdX7HWBsNH0bwpfdbOBvQGU0vUv0fHb0+jZJ67o4ej9mAoeVum45vAcjadi7qV3XO6rfm9FteuI7K47PuU7LISIiGZVrd5OIiGRBISEiIhkpJEREJCOFhIiIZKSQEBGRjBQSIjkys7sSZx8Vae+0C6xIjsysB+F/5yszqyWceffnJS6WSIvQuZtEcuTuS4q9TjPr7OGMxCKtiloSIjkys7uAXsAXNJwSIWGwu881s52B3xOu87GScFTsue7+aco6XgDOATq7+1axVEAkBxqTEMnfL4GXgTsJFz/qA3wcnWjtecLpMmqAg4Aq4BEzS/6f259weo1DgW/FWG6RrKm7SSRP7r7EzNYAKxItBAAz+wnwprv/OmnayYSLQg2j4URyq4DT3X11jMUWyYlCQqT4hgIjzKwuzWvb0hAS7yggpLVTSIgUXwfgceD8NK99lvR4eTzFEcmfQkKkMGuAjinTXge+D/zH3dfGXySR4tHAtUhh5gI1ZjbIzHpFA9M3E66A9qCZDTezbczsIDMbl7hwjEhboZAQKcwfCK2JGcBCYKC7zydcZrMeeJJwkZibgdXRTaTN0HESIiKSkVoSIiKSkUJCREQyUkiIiEhGCgkREclIISEiIhkpJEREJCOFhIiIZKSQEBGRjP4/FMStuIqsa7IAAAAASUVORK5CYII=)]

tb-paddle

步驟1：引入tb_paddle庫，定義作圖數據存儲位置（供第3步使用），本案例的路徑是“log/data”。

from tb_paddle import SummaryWriter
data_writer = SummaryWriter(logdir="log/data")

步驟2：在訓練過程中插入作圖語句。當每100個batch訓練完成後，將當前損失作爲一個新增的數據點(scalar_x和loss的映射對)存儲到第一步設置的文件中。使用變量scalar_x記錄下已經訓練的批次數，作爲作圖的X軸座標。

data_writer.add_scalar("train/loss", avg_loss.numpy(), scalar_x)
data_writer.add_scalar("train/accuracy", avg_acc.numpy(), scalar_x)
scalar_x = scalar_x + 100

步驟3：命令行啓動 tensorboard。

使用“tensorboard --logdir [數據文件所在文件夾路徑] 的命令啓動Tensor board。在Tensor board啓動後，命令行會打印出可用瀏覽器查閱圖形結果的網址。

$ tensorboard --logdir log/data

步驟4：打開瀏覽器，查看作圖結果，如圖6 所示。

查閱的網址在第三步的啓動命令後會打印出來（如TensorBoard 2.0.0 at http://localhost:6006/），將該網址輸入瀏覽器地址欄刷新頁面的效果如下圖所示。除了右側對數據點的作圖外，左側還有一個控制板，可以調整諸多作圖的細節。

注意：這裏需要安裝tensorboard，要不然無法打開這個這個。

令行會打印出可用瀏覽器查閱圖形結果的網址。

$ tensorboard --logdir log/data

步驟4：打開瀏覽器，查看作圖結果，如圖6 所示。

注意：這裏需要安裝tensorboard，要不然無法打開這個這個。

paddle深度學習基礎之訓練調試與優化