tensorflow 選擇參數進行訓練的方法

1 一般的訓練過程

train的過程其實就是修改計算圖中的tf.Variable的過程。

import tensorflow as tf
 
label = tf.constant(1,dtype = tf.float32)
prediction_to_train = tf.Variable(3,dtype=tf.float32)  # 指定了待訓練的變量（參數）
manual_compute_loss = tf.square(prediction_to_train - label)  # 指定損失函數爲待訓練參數與常量1的平方

optimizer = tf.train.GradientDescentOptimizer(0.01)  # 選擇優化器和對應的參數
train_step = optimizer.minimize(manual_compute_loss)  # 使用梯度下降優化損失函數
 
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    for _ in range(10):
        print('variable is ', sess.run(prediction_to_train), ' and the loss is ',sess.run(manual_compute_loss))
        sess.run(train_step)

variable is  3.0  and the loss is  4.0
variable is  2.96  and the loss is  3.8416002
variable is  2.9208  and the loss is  3.6894724
variable is  2.882384  and the loss is  3.5433698
variable is  2.8447363  and the loss is  3.403052
variable is  2.8078415  and the loss is  3.268291
 
。。。。。。。
。。。
variable is  2.0062745  and the loss is  1.0125883
variable is  1.986149  and the loss is  0.9724898
variable is  1.966426  and the loss is  0.9339792
 
。。。。
。。。
 
variable is  1.0000029  and the loss is  8.185452e-12
variable is  1.0000029  and the loss is  8.185452e-12
variable is  1.0000029  and the loss is  8.185452e-12
variable is  1.0000029  and the loss is  8.185452e-12
variable is  1.0000029  and the loss is  8.185452e-12

2 限定訓練的變量

根據train是修改計算圖中tf.Variable（默認是計算圖中所有tf.Variable，可以通過var_list指定）的事實，可以使用tf.constant或者python變量的形式來規避常量被訓練，這也是遷移學習要用到的技巧。

label = tf.constant(1,dtype = tf.float32)  # 標籤
x = tf.placeholder(dtype = tf.float32)
w1 = tf.Variable(4,dtype=tf.float32)  # 待訓練參數1
w2 = tf.Variable(4,dtype=tf.float32)  # 待訓練參數2
w3 = tf.constant(4,dtype=tf.float32)  # 常量權值
y_predict = w1*x+w2*x+w3*x  # 函數的輸出
 
make_up_loss = tf.square(y_predict - label)
optimizer = tf.train.GradientDescentOptimizer(0.01)
train_step = optimizer.minimize(make_up_loss)
 
init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    for _ in range(2):
        w1_,w2_,w3_,loss_ = sess.run([w1,w2,w3,make_up_loss],feed_dict={x:1})
        print('variable is w1:',w1_,' w2:',w2_,' w3:',w3_, ' and the loss is ',loss_)
        sess.run(train_step,{x:1})

variable is w1: -1.4999986  w2: -1.4999986  w3: 4.0  and the loss is  8.185452e-12
variable is w1: -1.4999986  w2: -1.4999986  w3: 4.0  and the loss is  8.185452e-12

3 使用var_list指定訓練變量

label = tf.constant(1,dtype = tf.float32)
x = tf.placeholder(dtype = tf.float32)
w1 = tf.Variable(4,dtype=tf.float32)
w2 = tf.Variable(4,dtype=tf.float32)
w3 = tf.constant(4,dtype=tf.float32)
 
y_predict = w1*x+w2*x+w3*x
 
#define losses and train
make_up_loss = tf.square(y_predict - label)
optimizer = tf.train.GradientDescentOptimizer(0.01)
train_step = optimizer.minimize(make_up_loss,var_list = w2)  # 指定待訓練的變量爲w2
 
init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    for _ in range(7):
        w1_,w2_,w3_,loss_ = sess.run([w1,w2,w3,make_up_loss],feed_dict={x:1})
        print('variable is w1:',w1_,' w2:',w2_,' w3:',w3_, ' and the loss is ',loss_)
        sess.run(train_step,{x:1})

variable is w1: 4.0  w2: -6.99948  w3: 4.0  and the loss is  2.7063857e-07
variable is w1: 4.0  w2: -6.9994903  w3: 4.0  and the loss is  2.5983377e-07
variable is w1: 4.0  w2: -6.9995003  w3: 4.0  and the loss is  2.4972542e-07
variable is w1: 4.0  w2: -6.9995103  w3: 4.0  and the loss is  2.398176e-07
variable is w1: 4.0  w2: -6.9995203  w3: 4.0  and the loss is  2.3011035e-07
variable is w1: 4.0  w2: -6.99953  w3: 4.0  and the loss is  2.2105178e-07
variable is w1: 4.0  w2: -6.9995394  w3: 4.0  and the loss is  2.1217511e-07

若無可訓練的 tf.Variable 變量，則會報錯：

ValueError: No variables to optimize.

用var_list指定一個 tf.constant 常量，也無法實現：

NotImplementedError: ('Trying to update a Tensor ', <tf.Tensor 'Const_1:0' shape=() dtype=float32>)

4 使用tf.getCollection

label = tf.constant(1,dtype = tf.float32)
x = tf.placeholder(dtype = tf.float32)
w1 = tf.Variable(4,dtype=tf.float32)

with tf.name_scope(name='selected_variable_to_trian'):
    w2 = tf.Variable(4,dtype=tf.float32)
   
w3 = tf.constant(4,dtype=tf.float32)
y_predict = w1*x+w2*x+w3*x
 
make_up_loss = (y_predict - label)**3
optimizer = tf.train.GradientDescentOptimizer(0.01)
 
output_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='selected_variable_to_trian')  # 指定訓練的變量域
train_step = optimizer.minimize(make_up_loss,var_list = output_vars)
 
init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    for _ in range(3000):
        w1_,w2_,w3_,loss_ = sess.run([w1,w2,w3,make_up_loss],feed_dict={x:1})
        print('variable is w1:',w1_,' w2:',w2_,' w3:',w3_, ' and the loss is ',loss_)
        sess.run(train_step,{x:1})

variable is w1: 4.0  w2: -6.988893  w3: 4.0  and the loss is  1.3702081e-06
variable is w1: 4.0  w2: -6.988897  w3: 4.0  and the loss is  1.3687968e-06
variable is w1: 4.0  w2: -6.9889007  w3: 4.0  and the loss is  1.3673865e-06
variable is w1: 4.0  w2: -6.9889045  w3: 4.0  and the loss is  1.3659771e-06
variable is w1: 4.0  w2: -6.9889083  w3: 4.0  and the loss is  1.3645688e-06
variable is w1: 4.0  w2: -6.988912  w3: 4.0  and the loss is  1.3631613e-06
variable is w1: 4.0  w2: -6.988916  w3: 4.0  and the loss is  1.3617548e-06
variable is w1: 4.0  w2: -6.9889197  w3: 4.0  and the loss is  1.3603493e-06

轉自：https://mp.csdn.net/mdeditor/97611750