检索结果-内蒙古大学图书馆

defectSAM: Prototype Prompt Guided SAM for few-shot defect segmentation

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2025年 74卷

作者： Wu, Zhifei Zhao, Shuying Zhang, Yunzhou Jin, Yuxin Northeastern Univ Coll Informat Sci & Engn Shenyang 110819 Peoples R China

Industrial defect segmentation plays a crucial role in ensuring quality and safety within manufacturing processes. Previous research in this field has focused on training models with limited available data to enable the segmentation of new classes. However, the limited and singular training data lead to limited feature extraction capabilities of the models, hindering their generalization to other industrial scenarios. Recently, the segment anything model (SAM), an interactive visual segmentation foundation model, demonstrates remarkable zero-shot generalization capabilities. However, its performance in the specific domain of industrial defect detection is relatively poor. Therefore, we propose an approach that leverages visual foundation model to achieve few-shot defect segmentation. This method extracts prior knowledge from example images, derives multitype robust prompts from the prior knowledge, and fine-tunes SAM with few-shot learning paradigm and low-rank adaptation (LoRA) strategy. Specifically, given a few support image-mask pairs, foreground and background prototypes are extracted, and their similarities with the query image are computed to obtain prototype prior probability maps. Subsequently, a prototype prompt generator specifically for defect data extracts salient information from these prior probability maps, producing robust prototype prompts. These prototype prompts are then used to fine-tune the LoRA layers in SAM's image encoder and the lightweight mask decoder of SAM, enabling SAM to adapt to the few-shot defect segmentation task. Experimental results demonstrate that our proposed method outperforms state-of-the-art few-shot defect segmentation techniques and can be generalized to various industrial scenarios, exhibiting significant application value.

关键词： few-shot defect segmentation low-rank adaptation (LoRA) prototype prompt segment anything visual foundation model

来源：评论

学校读者我要写书评

暂无评论

Multi-modal background-aware for defect semantic segmentation with limited data

引用

JOURNAL OF INTELLIGENT MANUFACTURING 2024年第5期36卷 3313-3325页

作者： Shan, Dexing Zhang, Yunzhou Liu, Shitong Northeastern Univ Coll Informat Sci & Engn Shenyang Peoples R China

Visual defect detection is widely used in intelligent manufacturing to achieve intelligent detection of product quality. Two main challenges remain in industrial applications. One is the scarcity of defect samples and the other is the weak texture variation of industrial defects. The above problems lead to the application of RGB image-based industrial defect segmentation. To this end, we propose a multi-modal background-aware network (MMBA-Net) for few-shot defect (2D+3D) segmentation with limited data, which can segment texture and structural defects in unseen and seen domains (objects). To synthesize the perception capabilities of different imaging conditions, MMBA-Net exploits the point cloud to provide spatial information for the RGB images. Furthermore, we found that background regions are perceptually consistent within an industrial image, which can be leveraged to discriminate between foreground and background regions. To implement this idea, we model correlation learning between multi-modal query samples and multi-modal normal (defect-free) samples as an optimal transport problem, establishing robust multi-modal background correlations between query and normal samples across different modalities. Experiments were conducted on real-world industrial products and food datasets, demonstrating that the proposed method can perform effective base learning and meta-learning on a small number of defective samples (approximately 15-25 defective training samples) to achieve effective segmentation of defects in the seen and unseen domains.

关键词： few-shot defect segmentation Multi-modal fusion Optimal transport Neural network

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：