one/few-shot segmentation 筆記

Task

對某一類別c1,選擇已有標註的樣本作爲support,待測樣本爲 query,model 以 support 爲參考去分割 query。
C-way K-shot:C爲類別數;分割每個 query 使用 K 個 support images

Solution

提取support 和 query 的 feature,將兩者的 feature 進行比對。訓練時藉助其他類別的標註數據去訓練model。

數據集

pascal-5i

每個sub-dataset中,trainset來自pascal trainset,testset來自pascal valset;trainset 和 testset 的類別無交集

 

One-Shot Learning for Semantic Segmentation   (2017.09)

dataset: pascal-5i
train時,support 和 query 均從Dtrain中抽樣,class相同
test時,sample N = 1000 examples 
Metric:mIoU

結構:conditioning branch 輸出參數 θ 用於segmentation branch 輸出feature 的分類
for k-shot:對每個support單獨得到的mask做logical OR

 

SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation (2018.11)

query image 送入 Guidance Branch 和 Segmentation Branch,通過concate 來使 Guidance Branch 輸出指導Segmentation Branch。
CosineSimilarty map :as guidance to teach the segmentation branch to discover the desired object regions.

for k-shot:
1、  Y is the predicted semantic label
2、use the averaged vector to guide the segmentation process

 

CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning (2019.03)

Dense Comparison Module:

Feature Extractor:ResNet-50,使用中間 block 的 feature(middle-level features that may
constitute object parts shared by unseen classes)
use dilated convolutions in layers after block2
Dense Comparison:concatenated feature maps go through another convolutional block with 256 3 × 3 convolutional filters for comparison.

Iterative Optimization Module
循環迭代,alternatively use predicted masks in the last epoch and empty masks as the input to IOM(dropout)

for k-shot:The result from the attention branch serves as the weight λ, λ 通過 softmax 作爲 sum 權重

使用弱標籤

 

PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment (2019.08 ICCV)

與檢測的 PANet 無關,重名了
support 和 query image 公用 feature extractor,對 support feature 做 mask GAP 得到 prototypes。
query feature 與 prototypes 計算 cosine distance
Prototype alignment regularization (PAR):將support 和 query 互換,taking the query and the predicted mask as the new support to learn to segment the support images.


for k-shot:所有 support 一起做 Mask GAP

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章