ClassWise-SAM-Adapter: Parameter-Efficient Fine-Tuning Adapts Segment Anything to SAR Domain for Semantic Segmentation

作     者：Pu, Xinyang Jia, Hecheng Zheng, Linghao Wang, Feng Xu, Feng 

作者机构：Fudan Univ Sch Informat Sci & Technol Key Lab Informat Sci Electromagnet Waves Minist Educ Shanghai 200433 Peoples R China 

出 版 物：《IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING》 (IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.)

年 卷 期：2025年第18卷

页      面：4791-4804页

核心收录：

学科分类：0808[工学-电气工程] 1002[医学-临床医学] 08[工学] 0705[理学-地理学] 0816[工学-测绘科学与技术] 

基　　金：Natural Science Foundation of China [61991421  61991422] 

主　　题：Radar polarimetry Foundation models Semantic segmentation Synthetic aperture radar Computer architecture Computational modeling Decoding Transformers Remote sensing Computer vision Adapter tuning landcover classification parameter-efficient fine-tuning segment anything (SA) synthetic aperture radar (SAR) visual foundation model 

摘      要：In the realm of artificial intelligence, the emergence of foundation models, backed by high computing capabilities and extensive data, has been revolutionary. A segment anything model (SAM), built on the vision transformer (ViT) model with millions of parameters and trained on its corresponding large-scale dataset SA-1B, excels in various segmentation scenarios relying on its significance of semantic information and generalization ability. Such achievement of visual foundation model stimulates continuous researches on specific downstream tasks in computer vision. The classwise-SAM-adapter (CWSAM) is designed to adapt the high-performing SAM for landcover classification on space-borne synthetic aperture radar (SAR) images. The proposed CWSAM freezes most of SAM s parameters and incorporates lightweight adapters for parameter-efficient fine-tuning, and a classwise mask decoder is designed to achieve semantic segmentation task. This adapt-tuning method allows for efficient landcover classification of SAR images, balancing the accuracy with computational demand. In addition, the task-specific input module injects low-frequency information of SAR images by MLP-based layers to improve the model performance. Compared to conventional state-of-the-art semantic segmentation algorithms by extensive experiments, CWSAM showcases enhanced performance with fewer computing resources, highlighting the potential of leveraging foundational models such as SAM for specific downstream tasks in the SAR domain.