利用動態生成的卷積核加速卷積神經網絡
ICLR2020文章：
DYNET: DYNAMIC CONVOLUTION FOR ACCELERATING CONVOLUTION NEURAL NETWORKS
（一）文章的核心思想：
adaptively generate convolution kernels based on image contents
通過 image的內容動態的生成卷積核
經過在mobilenet shufflenet resnet18 resnet50上對比試驗，在相同的精度下可以減少40.0%, 56.7%, 68.2% 72.4%的 FLOPs
加速效果非常明顯
（二）
For example, Liu
et al. (2019) compress MobileNetV2 to 124M, but the accuracy drops by 5.4% on ImageNet. We
theoretically analyze above observation, and find that this phenomenon is caused by the nature of
static convolution, where correlated kernels are cooperated to extract noise-irrelevant features. Thus
it is hard to compress the fixed convolution kernels without information loss. We also find that if we
linearly fuse several convolution kernels to generate one dynamic kernel based on the input, we can
obtain the noise-irrelevant features without the cooperation of multiple kernels, and further reduce
the computation cost of convolution layer remarkably
卷積核之間協同提取可以抑制噪聲的特徵，如果進行壓縮，會丟失信息，造成準確度下降；同時，如果基於輸入的image去線性加權合併現有的卷積核來生成新的動態卷積核，可以在保證準確性的同時，進一步減小計算量
cofficient prediction 是根據輸入的特徵來生成權值，通過對fixed conv kernel 進行加權來生成動態的卷積核（明顯的，動態生成的卷積核數量少很多，可以減小計算量）
上圖就是生成加權的權重過程
GAP是 global avg pooling
fc是以softmax爲激活函數的全連接層（之所以選擇softmax，是因爲輸出權值得和爲1）
