Faster R-CNN系列（四）：RPN的實現之神經網絡部分的實現

RPN的實現之神經網絡部分的實現

RPN的框架：

• 圖解

  • 上支路是所有anchor的修正量（4*9）
  • 下支路是前景背景9個anchor的分類得分(前景 背景)

• rpn_match

  • label爲1的anchor： 當一個anchor與真實bounding box的最大IOU超過閾值Vt1(0.7)
  • label爲-1的anchor ： 當一個anchor與真實bounding box的最大IOU低於閾值Vt2(0.3)
  • label爲0的anchor ： 當一個anchor與真實bounding box的最大IOU介於Vt2與Vt1之間
  • Negative anchor 與 Positive anchor 的數量之和是一個人爲設置的常數

• rpn_bbox

  • Input_rpn_bbox 是anchor和真實bbox之間的偏移量，RPN網絡計算的也是偏移量!
  • 只有positive anchor纔有對應的Input_rpn_bbox

ResNet

• block
  • 保證跳遠連接層和最後一層輸入層的長寬以及通道數一樣，可以實現相加操作
    
• 架構
  

import keras.layers as KL
from keras.models import Model
import keras.backend as K
import tensorflow as tf

構建block

def building_block(filters,block):
    if block !=0 :
        stride=1
    else:
        stride=2
    
    def f(x):
        # 主通路
        y=KL.Conv2D(filters,(1,1),strides=stride)(x)
        y=KL.BatchNormalization(axis=3)(y)
        y=KL.Activation("relu")(y)
        
        y=KL.Conv2D(filters,(3,3),padding="same")(y)
        y=KL.BatchNormalization(axis=3)(y)
        y=KL.Activation("relu")(y)
        
        y=KL.Conv2D(4*filters,(1,1))(y)
        y=KL.BatchNormalization(axis=3)(y)
        
        
        if block==0 :
            shortcut=KL.Conv2D(4*filters,(1,1),strides=stride)(x)
            shortcut=KL.BatchNormalization(axis=3)(shortcut)
        else:
            shortcut=x
            
        y=KL.Add()([y,shortcut])
        y=KL.Activation("relu")(y)
        
        return y
    return f

構建resnet

def resNet_featureExtractor(inputs):
    x=KL.Conv2D(64,(3,3),padding="same")(inputs)
    x=KL.BatchNormalization(axis=3)(x)
    x=KL.Activation("relu")(x)
    
    filters=64
    
    # 每一個stage的block的個數  每個stage中 第一個block是block1 其他的是block2
    blocks=[3,6,4]
    
    for i,block_num in enumerate(blocks):
        for block_id in range(block_num):
            x=building_block(filters,block_id)(x)
        filters *=2
    return x
        

x=KL.Input((64,64,3))
y=resNet_featureExtractor(x)
model=Model([x],[y])
model.summary()

__________________________________________________________________________________________________
Total params: 6,902,656
Trainable params: 6,875,136
Non-trainable params: 27,520
__________________________________________________________________________________________________

from keras.utils.vis_utils import plot_model
plot_model(model,to_file="images/rpn_resnet_model.png",show_shapes=True)

CNN網絡構建完成後，實現後續rpn

def rpn_net(inputs,k):
    shared_map=KL.Conv2D(256,(3,3),padding="same")(inputs)
    shared_map=KL.Activation("linear")(shared_map)
    
    # 下支路
    rpn_class=KL.Conv2D(2*k,(1,1))(shared_map)
    rpn_class=KL.Lambda(lambda x:tf.reshape(x,[tf.shape(rpn_class)[0],-1,2]))(rpn_class)
    rpn_class=KL.Activation("linear")(rpn_class)
    rpn_prob=KL.Activation("softmax")(rpn_class)
    
    
    #上支路
    y=KL.Conv2D(4*k,(1,1))(shared_map)
    y=KL.Activation("linear")(y)
    rpn_bbox=KL.Lambda(lambda x:tf.reshape(x,[tf.shape(x)[0],-1,4]))(y)
    
    return rpn_class,rpn_prob,rpn_bbox

x=KL.Input((64,64,3))
fp=resNet_featureExtractor(x)
rpn_class,rpn_prob,rpn_bbox=rpn_net(fp,9)
rpn_model=Model([x],[rpn_class,rpn_prob,rpn_bbox])
rpn_model.summary()

__________________________________________________________________________________________________

Total params: 9,276,086
Trainable params: 9,248,566
Non-trainable params: 27,520
__________________________________________________________________________________________________

plot_model(rpn_model,to_file="images/rpn_model.png",show_shapes=True)