Cov2d反向傳播梯度的計算過程

我們用一個例子來說明：

令
$x*w=y$
並且
$x=\begin{bmatrix} x_{11} & x_{12} & x_{13}\\ x_{21} & x_{22} & x_{23}\\ x_{31} & x_{32} & x_{33} \end{bmatrix}, \quad w=\begin{bmatrix} w_{11} & w_{12}\\ w_{21} & w_{22} \end{bmatrix}, \quad y=\begin{bmatrix} y_{11} & y_{12}\\ y_{21} & y_{22} \end{bmatrix}$

則
$\left\{\begin{aligned} &y_{11} = w_{11}x_{11} + w_{12}x_{12}+w_{21}x_{21}+w_{22}x_{22}\\ &y_{12} = w_{11}x_{12} + w_{12}x_{13}+w_{21}x_{22}+w_{22}x_{23}\\ &y_{21} = w_{11}x_{21} + w_{12}x_{22}+w_{21}x_{31}+w_{22}x_{32}\\ &y_{22} = w_{11}x_{22} + w_{12}x_{23}+w_{21}x_{32}+w_{22}x_{33} \end{aligned}\right.$

梯度
$\frac{\partial L}{\partial x} = \frac{\partial L}{\partial y}\frac{\partial y}{\partial x}=\begin{bmatrix} \delta_{11}w_{11} & \delta_{11}w_{12} + \delta_{12}w_{11} & \delta_{12}w_{12}\\ \delta_{11}w_{21} + \delta_{21}w_{11} & \delta_{11}w_{22} + \delta_{12}w_{21} + \delta_{21}w_{12} + \delta_{22}w_{11} & \delta_{12}w_{22} + \delta_{22}w_{12}\\ \delta_{21}w_{21} & \delta_{21}w_{22} + \delta_{22}w_{21} & \delta_{22}w_{22} \end{bmatrix}$
其計算過程如下圖所示（其中不同顏色塊對應位置的值相加）：

另外，可以用卷積的形式計算$x$的梯度：
$\frac{\partial L}{\partial x} = \left( \begin{array}{ccc} 0&0&0&0 \\ 0&\delta_{11}& \delta_{12}&0 \\ 0&\delta_{21}&\delta_{22}&0 \\ 0&0&0&0 \end{array} \right) * \left( \begin{array}{ccc} w_{22}&w_{21}\\ w_{12}&w_{11} \end{array} \right)$
此時卷積核$w$需要旋轉180度，也即上下翻轉一次，然後左右翻轉一次。輸入矩陣需要增加padding。

設正向卷積的padding爲$p$，反向卷積的padding爲$p'$，則需要增加的padding爲：
$p' = k-p-1$

其中$k$是卷積核的大小，另外假設正向卷積的stride爲1。