【Tensorflow】slim.repeat和stack的使用

一：常規做法

在搭建網絡時，TF-Slim 提供 repeat 和 stack，允許用戶重複執行相同的 操作，方便網絡構建，例如：

net = ...
net = slim.conv2d(net, 256, [3, 3], scope='conv3_1')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_2')
net = slim.conv2d(net, 256, [3, 3], scope='conv3_3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

常見的作法：可用循環減少工作 

net = ...
for i in range(3):
net = slim.conv2d(net, 256, [3, 3], scope='conv3_%d' % (i+1))
net = slim.max_pool2d(net, [2, 2], scope='pool2')

二：TF-Slim 中的 repeat 操作

使用 TF-Slim 中的 repeat 操作替代上邊的定義：

net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
net = slim.max_pool2d(net, [2, 2], scope='pool2')

slim.repeat 會自動給每一個卷積層的 scopes 命名爲’conv3/conv3_1’, ’conv3/conv3_2’ 和’conv3/conv3_3’

三：TF-Slim 中的 stack操作
TF-Slim 的 slim.stack 操作允許用戶用不同的參數重複調用同一種操作 slim.stack 也爲每一個被創建的操作創建一個新的 tf.variable_scope

# Verbose way:原始操作
x = slim.fully_connected(x, 32, scope='fc/fc_1')
x = slim.fully_connected(x, 64, scope='fc/fc_2')
x = slim.fully_connected(x, 128, scope='fc/fc_3')
 
# Equivalent , TF-Slim way using slim.stack:等價操作
slim.stack(x, slim.fully_connected , [32, 64, 128], scope='fc')
 
# slim.stack 調用了 slim.fully_connected 三次

看到repeat和stack在細節上有差異。具體查看文檔

例子，定義vgg

def vgg16(inputs):
    with slim.arg_scope([slim.conv2d, slim.fully_connected],
    activation_fn=tf.nn.relu,
    weights_initializer=tf.truncated_normal_initializer(0.0, 0.01),
    weights_regularizer=slim.l2_regularizer(0.0005)):  # with之下
        net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1')
        net = slim.max_pool2d(net, [2, 2], scope='pool1')
        net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2')
        net = slim.max_pool2d(net, [2, 2], scope='pool2')
        net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
        net = slim.max_pool2d(net, [2, 2], scope='pool3')
        net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4')
        net = slim.max_pool2d(net, [2, 2], scope='pool4')
        net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5')
        net = slim.max_pool2d(net, [2, 2], scope='pool5')
        net = slim.fully_connected(net, 4096, scope='fc6')
        net = slim.dropout(net, 0.5, scope='dropout6')
        net = slim.fully_connected(net, 4096, scope='fc7')
        net = slim.dropout(net, 0.5, scope='dropout7')
        net = slim.fully_connected(net, 1000, activation_fn=None, scope='fc8')
    return net

基於slim的vgg框架：

import tensorflow as tf
import tensorflow.contrib.slim.nets as nets
 
slim = tf.contrib.slim
vgg = nets.vgg
 
...
 
train_log_dir = ...
if not tf.gfile.Exists(train_log_dir):
    tf.gfile.MakeDirs(train_log_dir)
 
with tf.Graph().as_default():
    # Set up the data loading:
    images, labels = ...
 
    # Define the model:
    predictions = vgg.vgg_16(images, is_training=True)
 
    # Specify the loss function:
    slim.losses.softmax_cross_entropy(predictions , labels)
 
    total_loss = slim.losses.get_total_loss()
    tf.summary.scalar('losses/total_loss', total_loss)
 
    # Specify the optimization scheme:
    optimizer = tf.train.GradientDescentOptimizer(learning_rate=.001)
 
    # create_train_op that ensures that when we evaluate it to get the loss,
    # the update_ops are done and the gradient updates are computed.
    train_tensor = slim.learning.create_train_op(total_loss , optimizer)
 
    # Actually runs training.
    slim.learning.train(train_tensor , train_log_dir)