全局平均池

全連接層上的全局平均池的一個優點是,通過強制特徵映射和類別之間的對應,它對卷積結構更加本地化。因此,特徵圖可以容易地解釋爲類別置信度圖。另一個優點是在全局平均池中沒有優化參數,因此在該層避免過度擬合。

沒什麼好處,只是比fc參數少。cv裏面不如一直卷積,nlp裏面不如用rnn

最新的網絡中通常都是最後一個全局平均池化加一個全連接
至於原因,一是主流模型採用此種方法確實提升了精度(當然不僅這個改變,沒有做對比實驗不好下結論)。二是LeCun還是Bengio大神說過這倆效果一樣。

1,如果分類的確要使用全fc的話,fc的size可以根據具體任務調整。一般層數爲2層( fc_size-label_size),,第一層保持較高通道數(512或1024),第二層就按最終的類別數輸出(10,21,81,1000.....)。imagenet級別任務一般fc_size爲4096級別,並且配合dropout使用,mnist任務級別一般fc_size 512左右,依次類推。
2,目前分類任務中,主流網絡結構都拋棄了fc,原因有兩個: 參數太多; cpu實現起來實際gflops很低。用global avg+1*1 conv(這時候相當於fc)代替。

3.2 Global Average Pooling Conventional convolutional neural networks perform convolution in the lower layers of the network. For classification, the feature maps of the last convolutional layer are vectorized and fed into fully connected layers followed by a softmax logistic regression layer [4] [8] [11]. This structure bridges the convolutional structure with traditional neural network classifiers. It treats the convolutional layers as feature extractors, and the resulting feature is classified in a traditional way. However, the fully connected layers are prone to overfitting, thus hampering the generalization ability of the overall network. Dropout is proposed by Hinton et al. [5] as a regularizer which randomly sets half of the activations to the fully connected layers to zero during training. It has improved the generalization ability and largely prevents overfitting [4]. In this paper, we propose another strategy called global average pooling to replace the traditional fully connected layers in CNN. The idea is to generate one feature map for each corresponding category of the classification task in the last mlpconv layer. Instead of adding fully connected layers on top of the feature maps, we take the average of each feature map, and the resulting vector is fed directly into the softmax layer. One advantage of global average pooling over the fully connected layers is that it is more native to the convolution structure by enforcing correspondences between feature maps and categories. Thus the feature maps can be easily interpreted as categories confidence maps. Another advantage is that there is no parameter to optimize in the global average pooling thus overfitting is avoided at this layer. Futhermore, global average pooling sums out the spatial information, thus it is more robust to spatial translations of the input. We can see global average pooling as a structural regularizer that explicitly enforces feature maps to be confidence maps of concepts (categories). This is made possible by the mlpconv layers, as they makes better approximation to the confidence maps than GLMs.

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章