增量學習——《Insights from the Future for Continual Learning》——arxiv202006

Abstract

提出了一個持續學習的新情景,prescient continual learning(測試模型不僅past classes and current classes,還需要考慮future classes)。基於zero-shot learning的啓發,提出了Ghost Model,表徵空間的生成模型與損失函數的小心微調

Introduction

the future classes(no training samples),作者認爲這個setting需要讓模型know the classes and have some prior information about them。類似的考慮future classes的工作《Selfless sequential learning,ICLR2019》; 《Automatically discovering and learning new visual categories with ranking statistics,ICLR2020》提出一個setting,所有類的訓練樣本一開始就能全部得到,但標籤是增量式地獲取到的。而prescient continual learning setting與之相反,標籤一開始能全部得到,樣本是增量地獲取到的。舉了一個實際應用的例子,fashion product classification。然後介紹方法的insight,integrate the continual and zero-shot learning seamlessly
Zero-shot learning借鑑的兩篇文章《Learning to detect unseen object classes by
between-class attribute transfer,CVPR2009》《Zero-shot learning—a comprehensive
evaluation of the good, the bad and the ugly, TPAMI2019》

Method

ghost model是什麼?三個部件(feature extractor, feature generator, classifier),核心就是feature generator的設計,the generator learns the distribution of the features for all classes, aiming to generate plausible samples of features for the future classes.
For the feature generation to work, we must have exploitable prior information about the classes, more precisely, we must be able to map the class labels c into a class attribute space that makes semantic sense.
The exact way to perform that mapping will be data-dependent, but most often, either we will have an explicit set of attributes linked to each class (color, size, material, provenance, etc.), or we will be able to extract a latent semantic vector, using a technique like Word2vec [30, 32].
類別c與一個屬性特徵關聯,對於feature generator,給定類別c的屬性特徵,輸出類別c的樣本特徵,要與當前模型的特徵逼近。

Conclusion

a new setting, trains on a sequence of tasks, each introducing new classes, but has access to prior information about the classes.

Key points:prescient continual learning setting, 需要知道過去的類、當前出現類、未來出現的類具體是什麼這些prior information;核心就是generate a representation for the future classes,感覺有點ill-defined;方法設計地有點easy,直接將zero-shot拿過來用;工程實現起來的複雜程度,在複雜數據集比較難work

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章