【TensorFlow快速入門系列01】基礎操作

文章目錄

1 簡介

TF 是 Google 開發的 DL 開源框架，對於深度神經網絡的訓練助力極大。TF 使用圖 graphs 表示計算任務，在會話 session 中執行圖，圖中的節點是 op，一個 op 獲得 0個或多個張量 tensor，計算產生0個或多個張量 tensor；通過變量 variable 維護狀態，使用 feed 和 fetch 賦值和取值，如下圖所示。

2 會話

import tensorflow as tf

# 創建一個常量1x2
m1 = tf.constant([[3,3]])
# 創建一個常量2x1
m2 = tf.constant([[2],[3]])
# 矩陣乘法op
product = tf.matmul(m1, m2)

# 定義會話方式1
sess = tf.Session()
# 調用sess中的run方法來執行矩陣乘法op
result = sess.run(product)
print(result)
sess.close()

[[15]]

#定義會話方式2
with tf.Session() as sess:
    # 調用sess中的run方法來執行矩陣乘法op
    result = sess.run(product)
    print(result)

[[15]]

3 變量

import tensorflow as tf

# 定義一個變量
x = tf.Variable([1,2])
# 定義一個常量
a = tf.constant([3,3])
# 減法op
sub = tf.subtract(x, a)

# 所有變量初始化
init = tf.global_variables_initializer()

with tf.Session() as sess:
    # 執行變量初始化
    sess.run(init)
    print(sess.run(sub))

[-2 -1]

4 Feed 與 Fetch

import tensorflow as tf

# Fetch:可以在session中同時計算多個tensor或執行多個操作
# 定義三個常量
input1 = tf.constant(3.0)
input2 = tf.constant(2.0)
input3 = tf.constant(5.0)
# 加法op
add = tf.add(input2,input3)
# 乘法op
mul = tf.multiply(input1, add)

with tf.Session() as sess:
	# 多個操作並行
    res1,res2 = sess.run([mul, add])
    print(res1,res2)

21.0 7.0

# Feed:先定義佔位符，等需要的時候再傳入數據
input1 = tf.placeholder(tf.float32)
input2 = tf.placeholder(tf.float32)
# 乘法op
output = tf.multiply(input1, input2)

with tf.Session() as sess:
    print(sess.run(output, feed_dict={input1:8.0,input2:2.0})) # 字典形式

16.0

5 案例

使用 TF 實現 MNIST 手寫數字識別，注意激活函數，損失函數，優化器的使用。

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

# 載入數據集
mnist = input_data.read_data_sets("MNIST_data", one_hot=True)

# 一個批次大小
batch_size = 64
# 計算一個週期一共有多少個批次
n_batch = mnist.train.num_examples // batch_size

# 定義兩個佔位符
x = tf.placeholder(tf.float32,[None,784])
y = tf.placeholder(tf.float32,[None,10])

# 創建一個簡單的神經網絡:784-10
W = tf.Variable(tf.truncated_normal([784,10], stddev=0.1)) # 權重
b = tf.Variable(tf.zeros([10]) + 0.1) # 偏置項
prediction = tf.nn.softmax(tf.matmul(x,W)+b) # softmax激活函數

# MSE損失
loss = tf.losses.mean_squared_error(y, prediction)
# # 交叉熵損失
# loss = tf.losses.softmax_cross_entropy(y, prediction)
# GD優化器
train = tf.train.GradientDescentOptimizer(0.3).minimize(loss)
# # Adam優化器
# train_step = tf.train.AdamOptimizer(0.001).minimize(loss)

# 結果存放在一個布爾型列表中
correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1)) # 查找one-hot最大值所在位置,即把10個值變爲1個值
# 求準確率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

with tf.Session() as sess:
    # 變量初始化
    sess.run(tf.global_variables_initializer())
    # 週期epoch：所有數據訓練一次，就是一個週期
    for epoch in range(21):
        for batch in range(n_batch):
            # 獲取一個批次的數據和標籤
            batch_xs,batch_ys = mnist.train.next_batch(batch_size)
            sess.run(train,feed_dict={x:batch_xs,y:batch_ys})
        # 每訓練一個週期做一次測試
        acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})
        print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc))

Iter 0,Testing Accuracy 0.8513
Iter 1,Testing Accuracy 0.8849
Iter 2,Testing Accuracy 0.8954
Iter 3,Testing Accuracy 0.9016
Iter 4,Testing Accuracy 0.9051
Iter 5,Testing Accuracy 0.9088
Iter 6,Testing Accuracy 0.9112
Iter 7,Testing Accuracy 0.9128
Iter 8,Testing Accuracy 0.9145
Iter 9,Testing Accuracy 0.9153
Iter 10,Testing Accuracy 0.9164
Iter 11,Testing Accuracy 0.9163
Iter 12,Testing Accuracy 0.9172
Iter 13,Testing Accuracy 0.9183
Iter 14,Testing Accuracy 0.9187
Iter 15,Testing Accuracy 0.9192
Iter 16,Testing Accuracy 0.9186
Iter 17,Testing Accuracy 0.9196
Iter 18,Testing Accuracy 0.9209
Iter 19,Testing Accuracy 0.9203
Iter 20,Testing Accuracy 0.9205

6 防止過擬合

6.1 Dropout

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

# 載入數據集
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)

# 每個批次的大小
batch_size = 64
# 計算一共有多少個批次
n_batch = mnist.train.num_examples // batch_size

# 定義三個佔位符
x = tf.placeholder(tf.float32,[None,784])
y = tf.placeholder(tf.float32,[None,10])
keep_prob=tf.placeholder(tf.float32)

# NN：784-1000-500-10
W1 = tf.Variable(tf.truncated_normal([784,1000],stddev=0.1))
b1 = tf.Variable(tf.zeros([1000])+0.1)
L1 = tf.nn.tanh(tf.matmul(x,W1)+b1) # tanh激活函數
L1_drop = tf.nn.dropout(L1,keep_prob) # dropout

W2 = tf.Variable(tf.truncated_normal([1000,500],stddev=0.1))
b2 = tf.Variable(tf.zeros([500])+0.1)
L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2) # tanh激活函數
L2_drop = tf.nn.dropout(L2,keep_prob)  # dropout

W3 = tf.Variable(tf.truncated_normal([500,10],stddev=0.1))
b3 = tf.Variable(tf.zeros([10])+0.1)
prediction = tf.nn.softmax(tf.matmul(L2_drop,W3)+b3) # softmax激活函數

# 交叉熵損失
loss = tf.losses.softmax_cross_entropy(y,prediction)
# GD優化器
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(loss)

# 初始化變量
init = tf.global_variables_initializer()

# 結果存放在一個布爾型列表中
correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1)) # 返回一維張量中最大的值所在的位置
# 求準確率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

with tf.Session() as sess:
    sess.run(init)
    for epoch in range(31):
        for batch in range(n_batch):
            batch_xs,batch_ys =  mnist.train.next_batch(batch_size)
            sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys,keep_prob:0.5})
        
        test_acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels,keep_prob:1.0})
        train_acc = sess.run(accuracy,feed_dict={x:mnist.train.images,y:mnist.train.labels,keep_prob:1.0})
        print("Iter " + str(epoch) + ",Testing Accuracy " + str(test_acc) +",Training Accuracy " + str(train_acc))

Iter 0,Testing Accuracy 0.9477,Training Accuracy 0.9524364
Iter 1,Testing Accuracy 0.9621,Training Accuracy 0.96927273
Iter 2,Testing Accuracy 0.9663,Training Accuracy 0.97612727
Iter 3,Testing Accuracy 0.968,Training Accuracy 0.97958183
Iter 4,Testing Accuracy 0.9696,Training Accuracy 0.9835455
Iter 5,Testing Accuracy 0.9737,Training Accuracy 0.98785454
Iter 6,Testing Accuracy 0.976,Training Accuracy 0.9895273
Iter 7,Testing Accuracy 0.9773,Training Accuracy 0.99045455
Iter 8,Testing Accuracy 0.9765,Training Accuracy 0.9915636
Iter 9,Testing Accuracy 0.9783,Training Accuracy 0.99236363
Iter 10,Testing Accuracy 0.9788,Training Accuracy 0.99274546
Iter 11,Testing Accuracy 0.9789,Training Accuracy 0.99316365
Iter 12,Testing Accuracy 0.9806,Training Accuracy 0.9935091
Iter 13,Testing Accuracy 0.9792,Training Accuracy 0.9938545
Iter 14,Testing Accuracy 0.98,Training Accuracy 0.9941818
Iter 15,Testing Accuracy 0.9809,Training Accuracy 0.99447274
Iter 16,Testing Accuracy 0.9801,Training Accuracy 0.9946727
Iter 17,Testing Accuracy 0.9797,Training Accuracy 0.9948909
Iter 18,Testing Accuracy 0.98,Training Accuracy 0.9950909
Iter 19,Testing Accuracy 0.9803,Training Accuracy 0.9952545
Iter 20,Testing Accuracy 0.9806,Training Accuracy 0.9953091
Iter 21,Testing Accuracy 0.9807,Training Accuracy 0.9954727
Iter 22,Testing Accuracy 0.9814,Training Accuracy 0.9956
Iter 23,Testing Accuracy 0.9812,Training Accuracy 0.9956545
Iter 24,Testing Accuracy 0.9808,Training Accuracy 0.99570906
Iter 25,Testing Accuracy 0.981,Training Accuracy 0.9958
Iter 26,Testing Accuracy 0.9811,Training Accuracy 0.9958364
Iter 27,Testing Accuracy 0.9809,Training Accuracy 0.99594545
Iter 28,Testing Accuracy 0.9812,Training Accuracy 0.99603635
Iter 29,Testing Accuracy 0.9812,Training Accuracy 0.9960909
Iter 30,Testing Accuracy 0.9815,Training Accuracy 0.9962

6.2 正則化

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

# 載入數據集
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)

# 每個批次的大小
batch_size = 64
# 計算一共有多少個批次
n_batch = mnist.train.num_examples // batch_size

# 定義三個佔位符
x = tf.placeholder(tf.float32,[None,784])
y = tf.placeholder(tf.float32,[None,10])
keep_prob=tf.placeholder(tf.float32)

# NN:784-1000-500-10
W1 = tf.Variable(tf.truncated_normal([784,1000],stddev=0.1))
b1 = tf.Variable(tf.zeros([1000])+0.1)
L1 = tf.nn.tanh(tf.matmul(x,W1)+b1) # tanh激活函數
L1_drop = tf.nn.dropout(L1,keep_prob) # dropout

W2 = tf.Variable(tf.truncated_normal([1000,500],stddev=0.1))
b2 = tf.Variable(tf.zeros([500])+0.1)
L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2)
L2_drop = tf.nn.dropout(L2,keep_prob) 

W3 = tf.Variable(tf.truncated_normal([500,10],stddev=0.1))
b3 = tf.Variable(tf.zeros([10])+0.1)
prediction = tf.nn.softmax(tf.matmul(L2_drop,W3)+b3) # softmax激活函數

# 正則項
l2_loss = tf.nn.l2_loss(W1) + tf.nn.l2_loss(b1) + tf.nn.l2_loss(W2) + tf.nn.l2_loss(b2) + tf.nn.l2_loss(W3) + tf.nn.l2_loss(b3)

# 交叉熵損失
loss = tf.losses.softmax_cross_entropy(y,prediction) + 0.0005*l2_loss
# GD優化器
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(loss)

# 初始化變量
init = tf.global_variables_initializer()

# 結果存放在一個布爾型列表中
correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1)) # 返回一維張量中最大的值所在的位置
# 求準確率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

with tf.Session() as sess:
    sess.run(init)
    for epoch in range(31):
        for batch in range(n_batch):
            batch_xs,batch_ys =  mnist.train.next_batch(batch_size)
            sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys,keep_prob:1.0})
        
        test_acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels,keep_prob:1.0})
        train_acc = sess.run(accuracy,feed_dict={x:mnist.train.images,y:mnist.train.labels,keep_prob:1.0})
        print("Iter " + str(epoch) + ",Testing Accuracy " + str(test_acc) +",Training Accuracy " + str(train_acc))

Iter 0,Testing Accuracy 0.9451,Training Accuracy 0.94643635
Iter 1,Testing Accuracy 0.9529,Training Accuracy 0.9566909
Iter 2,Testing Accuracy 0.96,Training Accuracy 0.96574545
Iter 3,Testing Accuracy 0.9608,Training Accuracy 0.9655455
Iter 4,Testing Accuracy 0.9644,Training Accuracy 0.96776366
Iter 5,Testing Accuracy 0.9644,Training Accuracy 0.96772724
Iter 6,Testing Accuracy 0.9612,Training Accuracy 0.9637455
Iter 7,Testing Accuracy 0.9647,Training Accuracy 0.96952724
Iter 8,Testing Accuracy 0.9635,Training Accuracy 0.9685091
Iter 9,Testing Accuracy 0.9655,Training Accuracy 0.97016364
Iter 10,Testing Accuracy 0.9631,Training Accuracy 0.96703637
Iter 11,Testing Accuracy 0.9649,Training Accuracy 0.96965456
Iter 12,Testing Accuracy 0.9673,Training Accuracy 0.9712909
Iter 13,Testing Accuracy 0.9669,Training Accuracy 0.97174543
Iter 14,Testing Accuracy 0.9644,Training Accuracy 0.9681818
Iter 15,Testing Accuracy 0.9657,Training Accuracy 0.9709273
Iter 16,Testing Accuracy 0.9655,Training Accuracy 0.97154546
Iter 17,Testing Accuracy 0.966,Training Accuracy 0.9701818
Iter 18,Testing Accuracy 0.9635,Training Accuracy 0.96852726
Iter 19,Testing Accuracy 0.9665,Training Accuracy 0.9719818
Iter 20,Testing Accuracy 0.9679,Training Accuracy 0.9732909
Iter 21,Testing Accuracy 0.9683,Training Accuracy 0.9747273
Iter 22,Testing Accuracy 0.9664,Training Accuracy 0.9724
Iter 23,Testing Accuracy 0.9684,Training Accuracy 0.97367275
Iter 24,Testing Accuracy 0.9666,Training Accuracy 0.9719091
Iter 25,Testing Accuracy 0.9655,Training Accuracy 0.97212726
Iter 26,Testing Accuracy 0.9682,Training Accuracy 0.9728
Iter 27,Testing Accuracy 0.9676,Training Accuracy 0.97221816
Iter 28,Testing Accuracy 0.9669,Training Accuracy 0.97238183
Iter 29,Testing Accuracy 0.9675,Training Accuracy 0.97327274
Iter 30,Testing Accuracy 0.9665,Training Accuracy 0.9725091

【TensorFlow快速入門系列01】基礎操作

文章目錄

1 簡介

2 會話

3 變量

4 Feed 與 Fetch

5 案例

6 防止過擬合

6.1 Dropout

6.2 正則化

35K*14 薪，入職了！這公司只要不裁員，我能一直呆下去！

【ML從入門到入土系列03】K近鄰

一文搞定YOLO3訓練自己的數據集

【ML從入門到入土系列09】HMM

【劍指Offer系列68-2】二叉樹的最近公共祖先

一文搞定ML從入門到入土（附網盤鏈接）

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結