Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning

作     者：Sun, Dongwei Yao, Jing Zhou, Changsheng Cao, Xiangyong Ghamisi, Pedram 

作者机构：School of Computer Science and Technology the Ministry of Education Key Lab for Intelligent Networks and Network Security Xi’an Jiaotong University Xi’an710049 China Aerospace Information Research Institute Chinese Academy of Sciences Beijing100094 China School of Mathematics and Statistics Guangdong University of Technology Guangzhou China Helmholtz-Zentrum Dresden-Rossendorf Freiberg09599 Germany Lancaster Environment Centre Lancaster University LancasterLA1 4YR United Kingdom 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Change detection 

摘      要：Remote sensing image change description represents an innovative multimodal task within the realm of remote sensing processing. This task not only facilitates the detection of alterations in surface conditions, but also provides comprehensive descriptions of these changes, thereby improving human interpretability and interactivity. Generally, existing deep-learning-based methods predominantly utilized a three-stage framework that successively perform feature extraction, feature fusion, and localization from bitemporal images before text generation. However, this reliance often leads to an excessive focus on the design of specific network architectures and restricts the feature distributions to the dataset at hand, which in turn results in limited generalizability and robustness during application. To address these limitations, this paper proposes a novel approach for remote sensing image change detection and description that incorporates diffusion models, aiming to transition the emphasis of modeling paradigms from conventional feature learning to data distribution learning. The proposed method primarily includes a simple multi-scale change detection module, whose output features are subsequently refined by an well-designed diffusion model. Furthermore, we introduce a frequency-guided complex filter module to boost the model performance by managing high-frequency noise throughout the diffusion process. We validate the effectiveness of our proposed method across several datasets for remote sensing change detection and description, showcasing its superior performance compared to existing techniques. The code will be available at MaskApproxNet after a possible publication. Copyright © 2024, The Authors. All rights reserved.