TensorFlow Autodiff自動微分

with tf.GradientTape(persistent=True) as tape:
    z1 = f(w1, w2 + 2.)
    z2 = f(w1, w2 + 5.)
    z3 = f(w1, w2 + 7.)
    z = [z1,z3,z3]
[tape.gradient(z, [w1, w2]) for z in (z1, z2, z3)]

輸出結果

[[<tf.Tensor: id=56906, shape=(), dtype=float32, numpy=40.0>,
  <tf.Tensor: id=56898, shape=(), dtype=float32, numpy=10.0>],
 [<tf.Tensor: id=56919, shape=(), dtype=float32, numpy=46.0>,
  <tf.Tensor: id=56911, shape=(), dtype=float32, numpy=10.0>],
 [<tf.Tensor: id=56932, shape=(), dtype=float32, numpy=50.0>,
  <tf.Tensor: id=56924, shape=(), dtype=float32, numpy=10.0>]]

with tf.GradientTape(persistent=True) as tape:
    z1 = f(w1, w2 + 2.)
    z2 = f(w1, w2 + 5.)
    z3 = f(w1, w2 + 7.)
    z = [z1,z2,z3]
tape.gradient(z, [w1, w2])

輸出結果

[<tf.Tensor: id=57075, shape=(), dtype=float32, numpy=136.0>,
 <tf.Tensor: id=57076, shape=(), dtype=float32, numpy=30.0>]

總結：如果對一個listz=[z1,z2,z3]求微分，其結果將自動求和，而不是返回z1、z2和z3各自對[w1,w2]的微分。