【tensorflow 學習】給LSTM加上L2正則化

Reference
https://stackoverflow.com/questions/41841050/tensorflow-adding-regularization-to-lstm?noredirect=1&lq=1

1. 首先，用tf.trainable_variables()得到所有weights和bias，
2. 然後，用tf.nn.l2_loss()計算L2 norm，
3. 求和之後作爲正則項加給原來的cost function

tv = tf.trainable_variables()#得到所有可以訓練的參數，即所有trainable=True 的tf.Variable/tf.get_variable
regularization_cost = 0.001* tf.reduce_sum([ tf.nn.l2_loss(v) for v in tv ]) #0.001是lambda超參數
cost = original_cost_function + regularization_cost
optimizer = tf.train.AdamOptimizer(learning_rate = 0.01).minimize(cost)