模型參數量與FLOPs計算

模型的參數量與FLOPS計算

• 模型的參數量
  卷積層計算
  $模型的參數量 =[卷積核的長 * 卷積核的寬 * 卷積核的高（即通道，由上一層的輸出通道決定）]*\\ 卷積核的數量 + 偏置參數（其等於卷積核的數量）$
  全連接層
  由於不存在權值共享，它的FLOPs數目即是該層參數數目：$N_{in}∗N_{out}+N_{out}$​ 。

• 模型FLOPs
  卷積層計算
  $FLOPs數量 = 參數量 * 該層輸出特徵圖的大小 \\ 該層輸出特徵圖的大小: h * w$
  
  
  全連接層 由於不存在權值共享，它的FLOPs數目即是該層參數數目：
  $N_{in}∗N_{out}+N_{out}$ ​

深度學習中parameters個數和FLOPS計算（以CNN中經典的AlexNet網絡結構爲例） - Never-Giveup的博客 - CSDN博客
https://blog.csdn.net/qq_36653505/article/details/86700885


大話CNN經典模型：AlexNet - 雪餅的個人空間 - OSCHINA
https://my.oschina.net/u/876354/blog/1633143

• 矩陣乘法次數的計算過程
  矩陣乘法次數的計算過程 - Kellbook的博客 - CSDN博客 https://blog.csdn.net/qq_30622831/article/details/82730986

在計算FLOPs中，有

        kernel_ops = self.kernel_size[0] * self.kernel_size[1] * (self.in_channels / self.groups) 

見

rethinking-network-pruning/compute_flops.py at master · Eric-mingjie/rethinking-network-pruning https://github.com/Eric-mingjie/rethinking-network-pruning/blob/master/cifar/l1-norm-pruning/compute_flops.py

其中，self.group
pytorch的函數中的group參數的作用 - 慢行厚積 - 博客園 https://www.cnblogs.com/wanghui-garcia/p/10775851.html

例子1(VGG16 on imagenet)

圖中計算，忽略了偏置參數（其等於卷積核的數量）

• LayerID1卷積覈計算
  參數量
  $Conv1\_1 =\\ [ (LayerID1, Patch Size(Height)) *\\ (LayerID1, Patch Size(Width))* \\ (LayerID0, Output Size(Channel)) ]*\\ (LayerID1, Output Size(Channel))\\ = (3*3*3）* 64 \\ =1728$
  
  
  FLOPs計算
  $Conv1\_1 =\\ [ (LayerID1, Patch Size(Height)) *\\ (LayerID1, Patch Size(Width))* \\ (LayerID0, Output Size(Channel)) ]*\\ (LayerID1, Output Size(Height))*\\ (LayerID1, Output Size(Height))\\ = (3*3*3）* 64 *224*224\\ =1728 * 224*224 \\ = 86,704,128$

• LayerID12卷積覈計算
  參數量
  $Conv4\_1 =\\ [ (LayerID12, Patch Size(Height)) *\\ (LayerID12, Patch Size(Width))* \\ (LayerID11, Output Size(Channel)) ]*\\ (LayerID12, Output Size(Channel))\\ = (3*3*256）* 512\\ =1,179,648‬$
  
  
  FLOPs計算
  $Conv4\_1 =\\ [ (LayerID12, Patch Size(Height)) *\\ (LayerID12, Patch Size(Width))* \\ (LayerID11, Output Size(Channel)) ]*\\ (LayerID12, Output Size(Height))*\\ (LayerID12, Output Size(Height))\\ = (3*3*256）* 512 *28*28\\ =1,179,648‬ * 28*28\\ = ‭924,844,032‬$

• LayerID22全連接層計算
  參數量
  $FC1 =\\ [ (LayerID21, Output Size(Height)) *\\ (LayerID21, Output Size(Width))* \\ (LayerID21, Output Size(Channel)) ]*\\ [ (LayerID22, Output Size(Height)) *\\ (LayerID22, Output Size(Width))* \\ (LayerID22, Output Size(Channel)) ] \\ = [(7*7*512) ]* [1*1*4096]\\ =‭102,760,448‬$
  
  
  FLOPs計算
  $FC1 =\\ [ (LayerID21, Output Size(Height)) *\\ (LayerID21, Output Size(Width))* \\ (LayerID21, Output Size(Channel)) ]*\\ [ (LayerID22, Output Size(Height)) *\\ (LayerID22, Output Size(Width))* \\ (LayerID22, Output Size(Channel)) ]\\ = [(7*7*512) ]* [1*1*4096]\\ =‭102,760,448‬$

• LayerID23全連接層計算
  參數量
  $FC1 =\\ [ (LayerID22, Output Size(Height)) *\\ (LayerID22, Output Size(Width))* \\ (LayerID22, Output Size(Channel)) ]*\\ [ (LayerID23, Output Size(Height)) *\\ (LayerID23, Output Size(Width))* \\ (LayerID23, Output Size(Channel)) ]\\ = [(1*1*4096) ]* [1*1*4096]\\ =‭‭16,777,216‬$
  
  
  FLOPs計算
  $FC1 =\\ [ (LayerID22, Output Size(Height)) *\\ (LayerID22, Output Size(Width))* \\ (LayerID22, Output Size(Channel)) ]*\\ [ (LayerID23, Output Size(Height)) *\\ (LayerID23, Output Size(Width))* \\ (LayerID23, Output Size(Channel)) ]\\ = [(1*1*4096) ]* [1*1*4096]\\ =‭‭16,777,216‬$