检索结果-内蒙古大学图书馆

AESeg: Affinity-enhanced segmenter using feature class mapping knowledge distillation for efficient rgb-d semantic segmentation of indoor scenes

引用

NEURAL NETWORKS 2025年 188卷 107438页

作者： Zhou, Wujie Xiao, Yuxiang Qiang, Fangfang dong, Xiena Xu, Caie Yu, Lu Zhejiang Univ Sci & Technol Sch Informat & Elect Engn Hangzhou 310023 Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore 308232 Singapore Zhejiang Univ Coll Informat Sci & Elect Engn Hangzhou 310027 Peoples R China

Recent advances in deep learning for semantic segmentation models have introduced dynamic segmentation methods as opposed to static segmentation methods represented by full convolutional networks. dynamic prediction methods replace static classifiers with learnable class embeddings to achieve global semantic awareness. Although dynamic methods excel in accuracy, the learning and inference of class embeddings is usually accompanied by a tedious computational burden. To address this challenge, we propose an affinity-enhanced semantic segmentation framework that synergistically combines the strengths of static and dynamic methodologies. Specifically, our approach leverages semantic features to obtain preliminary static segmentation results and constructs a binary affinity matrix that explicitly encodes pixel-wise category relationships. This affinity matrix serves as a dynamic classification kernel, effectively integrating global context awareness with static features, achieving comparable performance to purely dynamic approaches but with a substantially reduced computational overhead. Furthermore, we introduce a novel feature-to-category mapping refinement technique. This technique performs feature knowledge migration by learning a linear transformation between the semantic feature space and the segmentation probability space, resulting in improved accuracy without increasing model complexity. Numerous experiments demonstrated that the proposed method achieves the best performance on the widely used NYUv2 and SUN-rgbd datasets. And the effectiveness of our method in different scenes is verified on the outdoor scene dataset CamVid.

关键词： Knowledge distillation Affinity rgb-d semantic segmentation Efficient semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

GANet: geometry-aware network for rgb-d semantic segmentation

引用

APPLIEd INTELLIGENCE 2025年第6期55卷 1-13页

作者： Tian, Chunqi Xu, Weirong Bai, Lizhi Yang, Jun Xu, Yanjun Tongji Univ Dept Comp Sci & Technol Shanghai 100190 Peoples R China

The field of rgb-d semantic segmentation has attracted considerable interest in recent times. The challenge is to develop an effective method for combining rgb images, which capture colour variations, with depth images, which provide robust information about object geometry regardless of lighting conditions. Treating both image types equally through the same convolution operator fails to take into account their inherent differences. Thus, in this paper, we propose a novel approach that combines a geometry-aware convolution (GAConv) module and a multiscale fusion module (MFM) with the aim of enhancing the performance of rgb-d image segmentation. The GAConv module effectively captures fine-grained geometric details from depth images, while the MFM module enables efficient integration of multi-scale features, allowing the network to utilise both spatial and semantic information. Extensive experimentation was conducted on the NYUv2 and SUN rgb-d datasets, wherein our model demonstrated consistent superiority over existing state-of-the-art methods in terms of pixel accuracy and mean intersection over union (mIoU).

关键词： rgb-d semantic segmentation Geometry-aware convolution

来源：评论

学校读者我要写书评

暂无评论

CdMANet: central difference mutual attention network for rgb-d semantic segmentation

引用

JOURNAL OF SUPERCOMPUTING 2025年第1期81卷 1-23页

作者： Ge, Mengjiao Su, Wen Gao, Jinfeng Jia, Guoqiang Zhejiang Sci Tech Univ Sch Informat Sci & Engn Hangzhou 310018 Zhejiang Peoples R China Zhejiang Fubao Intelligent Technol Co Ltd Hangzhou 311103 Zhejiang Peoples R China

rgb-d semantic segmentation utilizes both images and depth maps to classify pixels into different semantic classes. Currently, most methods rely on scale interaction within a single modality or the late fusion of dual-modal information, with little exploration of the correlation between two modalities at the same scale. There is also limited research on cross-scale and cross-modal feature fusion. This paper introduces a central difference mutual attention network for semantic segmentation based on dual-modal information, utilizing a parallel Transformer encoder to extract multi-level features from images and depth maps. To address issues such as blurring of boundaries in global information interaction and effectively provide spatial and semantic information interaction, a central difference mutual attention module is proposed. Finally, the composite cross decoder is employed to capture diverse feature levels while minimizing information loss. Experimental results demonstrate that our method outperforms several typical and state-of-the-art segmentation models on two challenging public benchmark datasets, achieving a mIoU of 51.9%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} and 79.4%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} on the NYU-dv2 and Cityscapes datasets, respectively, thus affirming its efficacy in balancing segmentation accuracy. Code is available at https://***/eG-Sophia/CdMANet.

关键词： rgb-d semantic segmentation Transformer Central difference mutual attention

来源：评论

学校读者我要写书评

暂无评论

HdBFormer: Efficient rgb-d semantic segmentation With a Heterogeneous dual-Branch Framework

引用

IEEE SIGNAL PROCESSING LETTERS 2025年 32卷 91-95页

作者： Wei, Shuobin Zhou, Zhuang Lu, Zhengan Yuan, Zizhao Su, Binghua Beijing Inst Technol Sch Informat Technol Zhuhai 519080 Peoples R China

In rgb-d semantic segmentation for indoor scenes, a key challenge is effectively integrating the rich color information from rgb images with the spatial distance information from depth images. However, most existing methods overlook the inherent differences in how rgb and depth images express information. Properly distinguishing the processing of rgb and depth images is essential to fully exploiting their unique and significant characteristics. To address this, we propose a novel heterogeneous dual-branch framework called HdBFormer, specifically designed to handle these modality differences. For rgb images, which contain rich detail, we employ both a basic and detail encoder to extract local and global features. For the simpler depth images, we propose LdFormer, a lightweight hierarchical encoder that efficiently extracts depth features with fewer parameters. Additionally, we introduce the Modality Information Interaction Module (MIIM), which combines transformers with large kernel convolutions to interact global and local information across modalities efficiently. Extensive experiments show that HdBFormer achieves state-of-the-art performance on the NYUdepthv2 and SUN-rgbd datasets.

关键词： Feature extraction Transformers Convolution semantic segmentation decoding Kernel Fuses Accuracy Training Testing Heterogeneous dual-branch framework modality differences rgb-d semantic segmentation

来源：评论

学校读者我要写书评

暂无评论

Shape-Aware Convolution with Convolutional Kernel Attention for rgb-d Image semantic segmentation

引用

IEICE TRANSACTIONS ON FUNdAMENTALS OF ELECTRONICS COMMUNICATIONS ANd COMPUTER SCIENCES 2025年第2期E108A卷 140-148页

作者： Zhou, Kun Zhang, Zejun Tang, Xu Xu, Wen Xie, Jianxiao Tang, Changbing Zhejiang Normal Univ Coll Phys & Elect Informat Engn Jinhua Peoples R China

rgb-d semantic segmentation has attracted increasing attention over the past few years. The depth feature encodes both the shape of a local geometry as well as the base (whereabout) of it in a larger context. rgb and depth images can be concatenated into one and inputted into a network model, reducing additional computation but resulting in some distractive information as they are multimodal. For the problem, we propose a Shape-aware Convolutional layer with Convolutional Kernel Attention (CKA-ShapeConv) for reducing the distractive information by leveraging each unique input feature to rectify the kernels. Instead of using a single convolution kernel, we aggregate N parallel convolution kernels based on input-dependent attention. Specifically, four sets of attention weights are firstly calculated from each input feature map, next N parallel convolution kernels are weighted and aggregated along different dimensions, which ensure that the generated convolution kernel is more capable of catching semantic information from the input feature map, reducing interference between rgb and depth features. Then the aggregated convolution kernel is decomposed into two components: base and shape, two new learnable weights are introduced to cooperate with them independently, and finally a convolution is applied on the re-weighted combination of these two components. These two components can capture semantic and shape information of regions effectively, respectively. Meanwhile, our CKA-ShapeConv layer can be easily integrated into most existing backbone models with only a small amount of additional computation. Our experiments on NYUdv2 and SUN rgb-d datasets show that the proposed CKA-ShapeConv layer can improve the performance of backbone models effectively.

关键词： rgb-d semantic segmentation input-dependent attention single-stream network dynamic convolution

来源：评论

学校读者我要写书评

暂无评论

Attention-based three-branch network for rgb-d indoor semantic segmentation

引用

dIGITAL SIGNAL PROCESSING 2025年 162卷

作者： Lei, Bo Guo, Peiyan Jia, Shaoyun Xian Univ Posts & Telecommun Sch Commun & Informat Engn Xian 710121 Peoples R China

In indoor scene segmentation, utilizing the complementary information from rgb and depth images has demonstrated robustness and effectiveness in semantic segmentation. However, simple methods such as concatenating rgb and depth features or performing element-wise addition do not fully leverage the potential of multi-modal features. To better integrate these features, an attention-based three-branch rgb-d semantic segmentation network for indoor scenes, named ABTNet is proposed in this paper. First, this network employs a three-branch encoder architecture to extract rgb features, depth features, and fused features, effectively capturing important information while retaining the original rgb-d characteristics. Second, two modules include the Multi-modal Feature Fusion Module (MFFM) and the Multi-level Feature Refinement Module (MFRM) are presented. The MFFM filters rgb and depth features and performs adaptive fusion, while the MFRM achieves high-resolution predictions by integrating features from different levels. Experimental results demonstrate that the proposed model achieves excellent performance on both the NYUdv2 dataset and the more complex SUN-rgbd dataset.

关键词： rgb-d semantic segmentation Attention mechanism Multi-modal feature fusion Multi-level feature refinement

来源：评论

学校读者我要写书评

暂无评论

Cleaning of object surfaces based on deep learning: a method for generating manipulator trajectories using rgb-d semantic segmentation

引用

NEURAL COMPUTING & APPLICATIONS 2023年第12期35卷 8677-8692页

作者： Qi, Lizhe Gan, Zhongxue Hua, Zhongwei du, daming Jiang, Wenxuan Sun, Yunquan Fudan Univ Acad Engn & Appl Technol Handan Rd 220 Shanghai 200433 Peoples R China

A mobile robot with a robotic arm needs to be able to autonomously perceive the operating environment and plan the trajectory of the object's surface in order to perform surface cleaning tasks in a complex, unstructured environment. This study suggests an autonomous trajectory planning technique for cleaning an object's surface based on rgb-d semantic segmentation, which enables the robotic arm to move the cleaning mechanism on the object's surface smoothly and steadily and finish the cleaning process. More particularly, it contains the following: (1) A double Attention Fusion Net (dAFNet) rgb-d semantic segmentation network is proposed, which successfully integrates color texture features and spatial structure features and enhances the semantic segmentation performance of indoor objects. This network is based on the dual attention mechanism (channel attention and spatial attention). (2) The trajectory planning algorithm for the robot arm is created, and the semantically segmented data is clustered using dBCSCAN. In order to achieve autonomous planning of the cleaning trajectory, the target subject is first extracted, and then the working trajectory of the robot arm is generated via the processes of edge detection, slicing, sampling, fitting, etc. We also compare the accuracy of dAFNet semantic segmentation and other algorithms on SUNrgbd and self-built datasets, experiment with trajectory generation for various objects, and evaluate the online surface cleaning procedure. According to the experimental findings, the dAFNet semantic segmentation model is more accurate than the current models. According to the online test, the trajectory generated has a good degree of smoothness and continuity, and the robotic arm is capable of completing the surface cleaning operation effectively.

关键词： rgb-d semantic segmentation Neural networks Point cloud slicing Trajectory planning

来源：评论

学校读者我要写书评

暂无评论

Spatial Information-Guided Adaptive Context-Aware Network for Efficient rgb-d semantic segmentation

引用

IEEE SENSORS JOURNAL 2023年第19期23卷 23512-23521页

作者： Zhang, Yang Xiong, Chenyun Liu, Junjie Ye, Xuhui Sun, Guodong Hubei Univ Technol Sch Mech Engn Wuhan 430068 Peoples R China Nanjing Univ Dept Comp Sci & Technol Natl Key Lab Novel Software Technol Nanjing 210023 Peoples R China

Efficient rgb-d semantic segmentation has received considerable attention in mobile robots, which plays a vital role in analyzing and recognizing environmental information. According to previous studies, depth information can provide corresponding geometric relationships for objects and scenes, but actual depth data usually exist as noise. To avoid unfavorable effects on segmentation accuracy and computation, it is necessary to design an efficient framework to leverage cross-modal correlations and complementary cues. In this article, we propose an efficient lightweight encoder-decoder network that reduces the computational parameters and guarantees the robustness of the algorithm. Working with channel and spatial fusion attention modules, our network effectively captures multi-level rgb-d features. A globally guided local affinity context module is proposed to obtain sufficient high-level context information. The decoder uses a lightweight residual unit (LRU) that combines short- and long-distance information with a few redundant computations. Experimental results on the NYUv2, SUN rgb-d, and Cityscapes datasets show that our method achieves a better tradeoff among segmentation accuracy, inference time, and parameters than the state-of-the-art (SOTA) methods.

关键词： Efficient encoder-decoder framework rgb-d semantic segmentation spatial and channel attention

来源：评论

学校读者我要写书评

暂无评论

Triple fusion and feature pyramid decoder for rgb-d semantic segmentation

引用

MULTIMEdIA SYSTEMS 2024年第5期30卷 1-13页

作者： Ge, Bin Zhu, Xu Tang, Zihan Xia, Chenxing Lu, Yiming Chen, Zhuang Anhui Univ Sci & Technol Coll Comp Sci & Engn Huainan 232001 Peoples R China Univ Wisconsin Madison Madison WI USA

Current rgb-d semantic segmentation networks incorporate depth information as an extra modality and merge rgb and depth features using methods such as equal-weighted concatenation or simple fusion strategies. However, these methods hinder the effective utilization of cross-modal information. Aiming at the problem that existing rgb-d semantic segmentation networks fail to fully utilize rgb and depth features, we propose an rgb-d semantic segmentation network, based on triple fusion and feature pyramid decoding, which achieves bidirectional interaction and fusion of rgb and depth features via the proposed three-stage cross-modal fusion module (TCFM). The TCFM proposes utilizing cross-modal cross-attention to intermix the data from two modalities into another modality. It fuses the rgb attributes and depth features proficiently, utilizing the channel-adaptive weighted fusion module. Furthermore, this paper introduces a lightweight feature pyramidal decoder network to fuse the multi-scale parts taken out by the encoder effectively. Experiments on NYU depth V2 and SUN rgb-d datasets demonstrate that the cross-modal feature fusion network proposed in this study efficiently segments intricate scenes.

关键词： rgb-d semantic segmentation Cross-modal Feature fusion Attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Self-Enhanced Feature Fusion for rgb-d semantic segmentation

引用

IEEE SIGNAL PROCESSING LETTERS 2024年 31卷 3015-3019页

作者： Xiang, Pengcheng Yao, Baochen Jiang, Zefeng Peng, Chengbin Ningbo Univ Coll Informat Sci & Engn Key Lab Mobile Network Applicat Technol Zhejiang P Ningbo 315211 Peoples R China

Effectively fusing depth and rgb information to fully leverage their complementary strengths is essential for advancing rgb-d semantic segmentation. However, when fusing with rgb information, traditional methods often overlook noises in depth data, presuming that they are of high accuracy. To resolve this issue, we propose a self-enhanced feature fusion network (SEFnet) for rgb-d semantic segmentation in this work. It mainly comprises three steps. Firstly, rgb and depth embeddings from the initial layers of the network are fused together. Secondly, the fused features are enhanced by pure rgb embeddings and are progressively guided by semantic edge labels to suppress irrelevant features. Finally, the enhanced features are combined with high-level rgb features and are fed into a normalizing flow decoder to obtain segmentation results. Experimental results demonstrate that the proposed approach can provide accurate predictions, outperforming state-of-the-art methods on benchmark datasets.

关键词： semantics Feature extraction Image edge detection semantic segmentation Training decoding Convolution Fuses Accuracy Vectors rgb-d semantic segmentation noises sementic edge normalizing flow

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：