nms_soft-nms

NMS and soft-NMS

date:2019-09-15

Table of Contents

1. nms和soft-nms
   1. nms
      1. nms代碼實現
   2. soft-nms
      1. soft-nms代碼實現

nms和soft-nms

nms

主要是將各個重疊的box清理,得到score最高的主要box,代碼步驟實現:

• 將所有的box框根據score進行升序排列
• 判斷box序列是否爲空
  • 保存當前的box序號
  • 計算最高score與其餘的box的iou
  • 將iou大於閾值的box去除,得到餘下的box序列
  • 更新box序列

nms代碼實現

#coding:utf-8
import numpy as np

def py_cpu_nms(dets, thre):
    #輸入:dets:爲輸入的box numpy原始序列:x1,y1,x2,y2,score
    #輸入:thre:iou閾值
    xx1 = dets[:, 0]
    yy1 = dets[:, 1]
    xx2 = dets[:, 2]
    yy2 = dets[:, 3]
    scores = dets[:, 4]
    #+1是因爲默認頂點所在的輪廓邊也爲目標區域
    areas = (xx2 - xx1 + 1) * (yy2 -yy1 + 1)
    out = []
    #按照score排序
    order = scores.argsort()[::-1]
    while len(order):
        _index = order[0]
        out.append(_index)
        #計算index對應的box和剩餘box的iou,進行篩選
        _xx1 = np.maximum(xx1[_index], xx1[order[1:]])
        _yy1 = np.maximum(yy1[_index], yy1[order[1:]])
        _xx2 = np.maximum(xx2[_index], xx2[order[1:]])
        _yy2 = np.maximum(yy2[_index], yy2[order[1:]])
        _w = np.maximum(0, _xx2 - _xx1 + 1)
        _h = np.maximum(0, _yy2 - _yy1 + 1)
        _areas_inter = _w * _h
        _overs = _areas_inter / (areas[_index] + areas[order[1:]] - _areas_inter)
        #更新order,_inds:+1,因爲len(_overs)+1=len(order)
        #np.where(),只有condition的情況下,返回各維度的index tuple,這裏只有一維:[0]
        _inds = np.where(_overs <= thre)[0]
        order = order[_inds + 1]

soft-nms

傳統的nms對於多個物體重疊的情況來說,會把低分的物體過濾掉,處理過於粗暴,參考上圖;soft-nms的方法是將計算得到的iou和box本身的score的輸入參數,重新計算box的置信度,最後根據新的置信度判斷是否去除這個box,計算公式爲:

線性加權:

高斯加權:

代碼實現步驟如下:

• 因爲過程中需要對score進行更新,需要不能提前對box的score進行排序,需要遍歷box,得到max_score的box序號:
  • 得到最高score box序號,交換當前序號和最大score的box參數,包括score,box位置
  • 遍歷剩下的box與當前最高score的box進行iou的計算:
    • 根據定義的更新規則(線性加權或者高斯加權)更新score
    • 將score低於閾值的box交換到隊列最後面,並更新隊列數目,不再遍歷這些box
    • 更新序號,循環遍歷

soft-nms代碼實現

#coding:utf-8
import numpy as np
#coding:utf-8
#coding:utf-8

def soft-nms(dets, thre=0.001, Nt=0.1, sigma=0.5, method=1):
    #輸入:dets:爲輸入的box numpy原始序列
    #thre:更新權重後score閾值,Nt爲iou閾值,sigma爲高斯權重的參數
    #method,0:nms,1:線性權重;2:高斯權重
    N = dets.shape[0]
    for i in range(N):
        #pos爲與i比較的box序號
        x1 = dets[i][0]
        y1 = dets[i][1]
        x2 = dets[i][2]
        y2 = dets[i][3]
        score = dets[i][4]
        max_score = dets[i][4]
        pos_tmp = i + 1
        pos = i
        #得到最大的score box序號
        while (pos_tmp < N):
            if max_score < dets[pos_tmp][4]:
                max_score = dets[pos_tmp][4]
                pos = pos_tmp
            pos_tmp += 1
        #swap pos and i index box
        dets[i] = dets[pos]
        dets[pos][0] = x1
        dets[pos][1] = y1
        dets[pos][2] = x2
        dets[pos][3] = y2
        dets[pos][4] = score

        #更新max score box的參數
        x1 = dets[i][0]
        y1 = dets[i][1]
        x2 = dets[i][2]
        y2 = dets[i][3]
        area = (x2 - x1 + 1) * (y2 - y1 + 1)
        pos_tmp = i + 1
        while (pos_tmp < N):
            xt1 = dets[pos_tmp][0]
            yt1 = dets[pos_tmp][1]
            xt2 = dets[pos_tmp][2]
            yt2 = dets[pos_tmp][3]
            scoret = dets[pos_tmp][4]
            areat = (xt2 - xt1 + 1) * (yt2 - yt1 + 1)
            #計算iou
            w = max(0, min(xt2, x2) - max(xt1, x1) + 1)
            h = max(0, min(yt2, y2) - max(yt1, y1) + 1)
            #if inter_area > 0
            if w*h > 0:
                iou = w*h / (area + areat - w*h)
                if iou > Nt:
                    if method == 1:
                        weight = 1 - iou
                    else if method == 2:
                        weight = -np.exp(iou * iou / sigma)
                    else:
                        weight = 1
                    #更新score
                    dets[pos_tmp][4] *= weight
            #如果score小於閾值,則將box扔到最後面廢棄,pos_tmp-=1重新計算
            if dets[pos_tmp][4] < thre:
                dets[pos_tmp] = dets[N-1]
                dets[N-1][0] = xt1
                dets[N-1][1] = yt1
                dets[N-1][2] = xt2
                dets[N-1][3] = yt2
                dets[N-1][4] = scoret
                N = N - 1
                pos_tmp -= 1
            pos_tmp += 1
    out = [_ for _ in N]
    return out

論文參考:soft-nms論文