CAP

Cross-Domain Autonomous Driving Perception using Contrastive Appearance Adaptation

Ziqiang Zheng1           Yingshu Chen1           Binh-Son Hua2,3           Yang Wu4           Sai-Kit Yeung1

1The Hong Kong University of Science and Technology
2Trinity College Dublin
3VinAI Research
4Tencent AI Lab

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2023

Framework Overview. The core of our image-level domain adaptation is a cycle of three mutual-beneficial modules: cross-domain content alignment (CDCA), reference-guided image synthesis and contrastive learning. CDCA uses the domain-invariant feature extractor learned by contrastive learning to construct source-reference image pairs for training the reference-guided image synthesis module to produce target-like images. The source and target-like images can be regarded as augmented views for contrastive learning to improve the domain-invariant feature extractor. Final target-like images and source labels can be adopted for downstream perception tasks.

Abstract

Addressing domain shifts for complex perception tasks in autonomous driving has long been a challenging problem. In this paper, we show that existing domain adaptation methods pay little attention to the content mismatch issue between source and target domains, thus weakening the domain adaptation performance and the decoupling of domain-invariant and domainspecific representations. To solve the aforementioned problems, we propose an image-level domain adaptation framework that aims at adapting source-domain images to the target domain with content-aligned source-target image pairs. Our framework consists of three mutually beneficial modules in a cycle: a cross-domain content alignment module to generate source-target pairs with consistent content representations in a selfsupervised manner, a reference-guided image synthesis based on the generated content-aligned source-target image pairs, and a contrastive learning module to self-supervise domain-invariant feature extractor. Our contrastive appearance adaptation is task agnostic and robust to complex perception tasks in autonomous driving. Our proposed method demonstrates state-of-the-art results in cross-domain object detection, semantic segmentation, and depth estimation as well as better image synthesis ability qualitatively and quantitatively.

Materials
article Paper

Citation
        @inproceedings{zheng2023compuda,
          title={Cross-Domain Autonomous Driving Perception using Contrastive Appearance Adaptation},
          author={Ziqiang Zheng and Yingshu Chen and Binh-Son Hua and Yang Wu and Sai-Kit Yeung},
          booktitle={2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
          year={2023}
          organization={IEEE}
        }