論文速覽：Multi-source Domain Adaptation for Semantic Segmentation

Multi-source Domain Adaptation for Semantic Segmentation

[NeurIPS 2019] [2020: MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation] [github]

目錄

Multi-source Domain Adaptation for Semantic Segmentation

Abstract

Problem Setup

MADAN

Overview

Dynamic Adversarial Image Generation

Adversarial Domain Aggregation

Feature-aligned Semantic Segmentation

MADAN Learning

---

Abstract

Simulation-to-real domain adaptation for semantic segmentation has been actively studied for various applications such as autonomous driving. Existing methods mainly focus on a single-source setting, which cannot easily handle a more practical scenario of multiple sources with different distributions. In this paper, we propose to investigate multi-source domain adaptation for semantic segmentation. Specifically, we design a novel framework, termed Multi-source Adversarial Domain Aggregation Network (MADAN), which can be trained in an end-to-end manner. First, we generate an adapted domain for each source with dynamic semantic consistency while aligning at the pixel-level cycle-consistently towards the target. Second, we propose sub-domain aggregation discriminator and cross-domain cycle discriminator to make different adapted domains more closely aggregated. Finally, feature-level alignment is performed between the aggregated domain and target domain while training the segmentation network. Extensive experiments from synthetic GTA and SYNTHIA to real Cityscapes and BDDS datasets demonstrate that the proposed MADAN model outperforms state-of-the-art approaches.

第一句，研究意義：在自動駕駛等多種應用中，針對語義分割的仿真-真實域適應問題進行了研究。

第二句，提出問題：現有的方法主要集中在單一源域設置上，這很難處理具有不同分佈的、多個源的、更實際的場景。

第三句，本文工作：一句話概括本文做的內容，即 提出研究語義分割的多源域適應問題。

第四-七句，具體方法：本文提出的模型，Multi-source Adversarial Domain Aggregation Network (MADAN)，包括三個部分：

First，爲每個源生成一個具有動態語義一致性的自適應域，同時在像素級循環上對齊目標；

Second，出了子域聚集判別器和跨域循環判別器，以使不同的適應域更緊密地聚集在一起；

Finally，在訓練分割網絡的同時，對聚集的域和目標域進行特徵級對齊。

第八句，實驗結論：從合成的數據集 GTA 和 SYNTHIA 到真實的數據集 Cityscapes 和 BDDS 上的廣泛實驗表明，所提出的 MADAN 模型優於最先進的方法。

 

Problem Setup

 

 

MADAN

Overview

 

Figure 1: The framework of the proposed Multi-source Adversarial Domain Aggregation Network (MADAN). The colored solid arrows represent generators, while the black solid arrows indicate the segmentation network F. The dashed arrows correspond to different losses.

Dynamic Adversarial Image Generation

Adversarial Domain Aggregation

Feature-aligned Semantic Segmentation

MADAN Learning