【記錄】爲什麼在CNN的Batch Norm中對C個channel進行歸一化

下面是cs231n-assignment2中的Spatial Batch Norm的介紹中對於此處空間歸一化的解釋,爲什麼對shape爲NCHW的圖像數據進行Batch Norm時需要計算NHW維度的mean和var

If the feature map was produced using convolutions, then we expect every feature channel’s statistics e.g. mean, variance to be relatively consistent both between different images, and different locations within the same image – after all, every feature channel is produced by the same convolutional filter! Therefore spatial batch normalization computes a mean and variance for each of the C feature channels by computing statistics over the minibatch dimension N as well the spatial dimensions H and W.

如果使用卷積生成feature map,那麼我們期望每個feature channel的統計數據,例如不同圖像之間,以及同一圖像中的不同位置之間的均值、方差相對一致 - 畢竟,每個feature channel都是由相同的filter卷積產生的!因此,空間批量歸一化通過計算小批量維度N以及空間維度H和W的統計量來計算每個C feature channel的均值和方差。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章