Federated Hallucination Translation and Source-Free Regularization Adaptation in Decentralized Domain Adaptation for Foggy Scene Understanding

作     者：Jin, Xiating Bu, Jiajun Yu, Zhi Zhang, Hui Wang, Yaonan 

作者机构：Zhejiang Univ Coll Software Technol Zhejiang Key Lab Accessible Percept & Intelligent Ningbo 315048 Peoples R China Zhejiang Univ Coll Comp Sci Zhejiang Key Lab Accessible Percept & Intelligent Hangzhou 310027 Peoples R China Hunan Univ Natl Engn Res Ctr Robot Visual Percept & Control T Changsha 410082 Peoples R China Hunan Univ Coll Robot Changsha 410082 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON MULTIMEDIA》 (IEEE Trans Multimedia)

年 卷 期：2025年第27卷

页      面：1601-1616页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Natural Science Foundation of China [62372408, 62027810] National Key Research and Development Program of China [2021YFB2701100, 2019YFF0302601] Zhejiang Provincial Key Research and Development Program of China [2021C01105] Ningbo Natural Science Foundation [2022J183] Key Program of the National Natural Science Foundation of China Major Research plan of the National Natural Science Foundation of China Hunan Leading Talent of Technological Innovation [2022RC3063] Hunan Science Fund for Distinguished Young Scholars [2021JJ10025] Joint Open Foundation of State Key Laboratory of Robotics [2021-KF-22-17] 

主　　题：Translation Adaptation models Entropy Atmospheric modeling Generative adversarial networks Bridges Semantics Robots Minimization Distributed databases Decentralization federated domain adaptation source-free adaptation image translation contrastive learning prototypical knowledge foggy scene understanding 

摘      要：Semantic foggy scene understanding (SFSU) emerges a challenging task under out-of-domain distribution (OD) due to uncertain cognition caused by degraded visibility. With the strong assumption of data centralization, unsupervised domain adaptation (UDA) reduces vulnerability under OD scenario. Whereas, enlarged domain gap and growing privacy concern heavily challenge conventional UDA. Motivated by gap decomposition and data decentralization, we establish a decentralized domain adaptation (DDA) framework called Translate thEn Adapt (abbr. TEA) for privacy preservation. Our highlights lie in. (1) Regarding federated hallucination translation, a Disentanglement and Contrastive-learning based Generative Adversarial Network (abbr. DisCoGAN) is proposed to impose contrastive prior and disentangle latent space in cycle-consistent translation. To yield domain hallucination, client minimizes cross-entropy of local classifier but maximizes entropy of global model to train translator. (2) Regarding source-free regularization adaptation, a Prototypical-knowledge based Regularization Adaptation (abbr. ProRA) is presented to align joint distribution in output space. Soft adversarial learning relaxes binary label to rectify inter-domain discrepancy and inner-domain divergence. Structure clustering and entropy minimization drive intra-class features closer and inter-class features apart. Extensive experiments exhibit efficacy of our TEA which achieves 55.26% or 46.25% mIoU in adaptation from GTA5 to Foggy Cityscapes or Foggy Zurich, outperforming other DDA methods for SFSU.