检索结果-内蒙古大学图书馆

multimodal object detection of UAV Remote Sensing Based on Joint Representation Optimization and Specific Information Enhancement

引用

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING 2024年 17卷 1016-1025页

作者： Wang, Jinpeng Xu, Congan Zhao, Chunhui Gao, Long Wu, Junfeng Yan, Yiming Feng, Shou Su, Nan Harbin Engn Univ Coll Informat & Commun Engn Harbin 150001 Peoples R China Naval Aeronaut Univ Yantai 264000 Peoples R China Minist Ind & Informat Technol Key Lab Adv Marine Commun & Informat Technol Harbin 150001 Peoples R China

With the development of Earth observation technology, it becomes easier and easier to acquire multimodal image data at the same time. To improve the performance of a multimodal remote-sensing detection algorithm, a new fusion feature optimization detection network is proposed. The method is designed to solve the problem of performance degradation caused by the unreliability of single-modal data in multimodal remote-sensing data. The key to obtain high-quality fusion features from multimodal data with interference is to suppress single-modal redundant features and fully integrate multimodal features. The proposed method mainly includes two improvements. First, a novel joint expression optimization module is designed to enhance the target features and suppress the redundant and interference features that affect the fusion effect. In addition, we propose a novel specific information enhancement module to further enhance the discriminative feature information of targets within each modal image. Experiments on the DroneVehicle dataset show that our proposed method is state of the art on this dataset.

关键词： Feature extraction Tensors Remote sensing Optical sensors Optical imaging object detection Optimization Joint expression optimization module (JEOM) multimodal object detection specific information enhancement module (SIEM)

来源：评论

学校读者我要写书评

暂无评论

SeaDATE: Remedy Dual-Attention Transformer With Semantic Alignment via Contrast Learning for multimodal object detection

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2025年第5期35卷 4713-4726页

作者： Dong, Shuhan Xie, Weiying Yang, Danian Li, Yunsong Zhang, Jiaqing Tian, Jiayuan Lei, Jie Xidian Univ State Key Lab Integrated Serv Networks Xian 710071 Peoples R China Univ Technol Sydney Sch Elect & Data Engn Sydney NSW 2007 Australia

multimodal object detection leverages diverse modal information to enhance the accuracy and robustness of detectors. Due to its ability to capture long-range dependencies, the Transformer model provides a powerful mechanism for integrating multimodal features during feature extraction. This capability significantly enhances the accuracy of multimodal object detection by addressing the limitations of local feature extraction inherent in traditional methods. However, current methods merely stack Transformer-guided fusion techniques without exploring their capability to extract features at various depth layers of network, thus limiting the improvements in detection performance. In this paper, we introduce an accurate and efficient multimodal object detection method named SeaDATE. Initially, we propose a novel dual attention Feature Fusion (DTF) module that, under Transformer's guidance, integrates local and global information through a dual attention mechanism, strengthening the fusion of modal features from orthogonal perspectives using spatial and channel tokens. Meanwhile, our theoretical analysis and empirical validation demonstrate that the Transformer-guided fusion method, treating images as sequences of pixels for fusion, performs better on shallow features' detail information compared to deep semantic information. To address this, we designed a contrastive learning (CL) module aimed at learning features of multimodal samples, remedying the shortcomings of Transformer-guided fusion in extracting deep semantic features, and effectively utilizing cross-modal information. Extensive experiments and ablation studies on the FLIR, LLVIP, and M3FD datasets have proven our method to be effective, achieving state-of-the-art detection performance.

关键词： Feature extraction Transformers object detection Semantics Accuracy Contrastive learning Circuits and systems Detectors Data mining Robustness multimodal object detection transformer attention mechanism contrastive learning

来源：评论

学校读者我要写书评

暂无评论

Low saliency crack detection based on improved multimodal object detection network: an example of wind turbine blade inner surface

引用

JOURNAL OF ELECTRONIC IMAGING 2023年第3期32卷

作者： Gao, Yinfeng Dai, Shijie Ji, Wenbin Wang, Ruiqin Hebei Univ Technol Tianjin Peoples R China Hebei Univ Technol Sch Mech Engn Tianjin Peoples R China

Accurate identification of cracks is of great significance for maintaining the health of the equipment. However, the low saliency of cracks in some composite or metal surfaces affects the detection accuracy of object detection algorithms. For example, small cracks on the inner surface of wind turbine blade (WTB) may be similar in color to the substrate or face complex background textures. Taking WTB cracks as low saliency crack samples, we propose a multimodal object detection convolutional neural network that fuses infrared images with visible images to detect cracks more accurately. The proposed network contains the CenterNet network with an existing fast and efficient mid-level fusion structure. First, we optimized the fusion structure to make it more suitable for extracting crack features. To address the problem that severe background interference in multimodal images affects the detection performance, we add channel attention to the fusion structure and train the improved network using a stepwise training method to enhance the framework's ability to filter background interference information. Finally, the effectiveness of the improvements was verified by ablation experiments and feature map analysis, and the phenomena of wrong detection, missed detection, and repeated detection were reduced. The evaluation results show that the proposed multimodal object detection network is able to detect the low saliency WTB cracks more effectively, and the improvement of the network also results in a 6.22% increase in average precision. In addition, this method can be extended to other materials or scenes to identify very inconspicuous objects, replacing manual inspection in more challenging defect detection tasks. (C) 2023 SPIE and IS&T

关键词： multimodal object detection low saliency cracks wind turbine blade convolutional neural network infrared thermography

来源：评论

学校读者我要写书评

暂无评论

Weakly Aligned Feature Fusion for multimodal object detection

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021年第3期36卷 4209-4219页

作者： Zhang, Lu Liu, Zhiyong Zhu, Xiangyu Song, Zhan Yang, Xu Lei, Zhen Qiao, Hong Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100086 Peoples R China Chinese Acad Sci Ctr Excellence Brain Sci & Intelligence Technol Shanghai 200031 Peoples R China Chinese Acad Sci Ctr Biometr & Secur Res Inst Automat Beijing 100190 Peoples R China Chinese Acad Sci Natl Lab Pattern Recognit Inst Automat Beijing 100190 Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Shenzhen 518055 Guangdong Peoples R China Chinese Acad Sci Ctr Artificial Intelligence & Robot Hong Kong Inst Sci & Innovat Hong Kong Peoples R China

To achieve accurate and robust object detection in the real-world scenario, various forms of images are incorporated, such as color, thermal, and depth. However, multimodal data often suffer from the position shift problem, i.e., the image pair is not strictly aligned, making one object has different positions in different modalities. For the deep learning method, this problem makes it difficult to fuse multimodal features and puzzles the convolutional neural network (CNN) training. In this article, we propose a general multimodal detector named aligned region CNN (AR-CNN) to tackle the position shift problem. First, a region feature (RF) alignment module with adjacent similarity constraint is designed to consistently predict the position shift between two modalities and adaptively align the cross-modal RFs. Second, we propose a novel region of interest (RoI) jitter strategy to improve the robustness to unexpected shift patterns. Third, we present a new multimodal feature fusion method that selects the more reliable feature and suppresses the less useful one via feature reweighting. In addition, by locating bounding boxes in both modalities and building their relationships, we provide novel multimodal labeling named KAIST-Paired. Extensive experiments on 2-D and 3-D object detection, RGB-T, and RGB-D datasets demonstrate the effectiveness and robustness of our method.

关键词： object detection Feature extraction Detectors Robustness Cameras Automation Training Deep learning feature fusion multimodal object detection pedestrian detection

来源：评论

学校读者我要写书评

暂无评论

Adaptive multimodal feature fusion with frequency domain gate for remote sensing object detection

引用

REMOTE SENSING LETTERS 2024年第2期15卷 133-144页

作者： Sun, Xu Yu, Yinhui Cheng, Qing Jilin Univ Sch Commun Engn Changchun Peoples R China Jilin Univ Sch Commun Engn 5372 Nanhu Rd Changchun 130012 Peoples R China

Fusing complementary information of visible and infrared radiation modalities can improve object detection performance for unmanned aerial vehicle (UAV) remote sensing images under insufficient illumination conditions. Although previous works have conducted some studies in this field, they have rarely considered the adaptive ability of multimodal feature fusion, which limits the performance improvement space for multimodal detectors. To this end, we propose an adaptive multimodal feature fusion method with a frequency domain gate based on DINO (detection transformer with improved denoising anchor boxes), called multimodal DINO. In our approach, a multimodal feature encoder with underlying feature sharing is designed, which efficiently extracts common and differential features through RGB-guided infrared radiation data transformation. Additionally, an adaptive frequency domain gate is introduced to dynamically learn the degree of dependence on frequency-filtered features of each modality when processing different samples. We evaluate the proposed method on the two multimodal detection remote sensing image datasets, VEDAI and DroneVehicle. Extensive experiments demonstrate that our approach achieves superior performance compared to basic detectors and existing multimodal detection methods. Our code is available at https://***/cq100/multimodalDINO.

关键词： Unmanned aerial vehicles remote sensing image multimodal object detection frequency domain gate

来源：评论

学校读者我要写书评

暂无评论

Sec-CLOCs: multimodal Back-End Fusion-Based object detection Algorithm in Snowy Scenes

引用

SENSORS 2024年第22期24卷 7401页

作者： Gong, Rui Fan, Xiangsuo Cai, Dengsheng Lu, You Guangxi Univ Sci & Technol Sch Automat Liuzhou 545006 Peoples R China Guangxi Univ Sci & Technol Guangxi Collaborat Innovat Ctr Earthmoving Machine Liuzhou 545006 Peoples R China Guangxi Liugong Machinery Co Ltd Liuzhou 545006 Peoples R China

LiDAR and cameras, often regarded as the "eyes" of intelligent driving vehicles, are vulnerable to adverse weather conditions like haze, rain, and snow, compromising driving safety. In order to solve this problem and enhance the environmental sensing capability under severe weather conditions, this paper proposes a multimodal back-end fusion object detection method, Sec-CLOCs, which is specifically optimized for vehicle detection under heavy snow. This method achieves object detection by integrating an improved YOLOv8s 2D detector with a SECOND 3D detector. First, the quality of image data is enhanced through the Two-stage Knowledge Learning and Multi-contrastive Regularization (TKLMR) image processing algorithm. Additionally, the DyHead detection head and Wise-IOU loss function are introduced to optimize YOLOv8s and improve 2D detection *** LIDROR algorithm preprocesses point cloud data for the SECOND detector, yielding 3D object detection results. The CLOCs back-end fusion algorithm is then employed to merge the 2D and 3D detection outcomes, thereby enhancing overall object detection capabilities. The experimental results show that the Sec-CLOCs algorithm achieves a vehicle detection accuracy of 82.34% in moderate mode (30-100 m) and 81.76% in hard mode (more than 100 m) under heavy snowfall, which demonstrates the algorithm's high detection performance and robustness.

关键词： DyHead LIDROR multimodal object detection Sec-CLOCs Wise-IOU YOLOv8s

来源：评论

学校读者我要写书评

暂无评论

RTM-UAVDet: A Real-Time multimodal UAV Detector

引用

IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS 2025年第1期61卷 473-489页

作者： Wang, Guorui Jiang, Qian Jin, Xin Wozniak, Michal Wang, Puming Yao, Shaowen Yao Yunnan Univ Kunming 650031 Peoples R China Wroclaw Univ Sci & Technol PL-50370 Wroclaw Poland

As unmanned aerial vehicles (UAVs) become more prevalent, the need for accurate and reliable detection algorithms becomes increasingly important. UAV detection algorithms are essential for maintaining security, managing airspace, safeguarding privacy, responding to emergencies, protecting critical infrastructure, ensuring regulatory compliance, and enabling responsible UAV usage across various domains. The multiscene images and videos based on visible light (RGB) and thermal infrared (TIR) remote sensing of UAVs are considered crucial data sources for public safety. However, as imaging techniques move from single to multimodality, using object detection algorithms to detect RGB and TIR modalities simultaneously in real -time is a significant challenge. This study proposes a multimodal UAV object detection framework for static images. Based on convolutional neural network architecture, the multimodal dynamic convolution model extracts features from ground TIR images and videos of forward-looking infrared and visible light cameras. Our approach circumvents the constraints of conventional multimodal fusion methods by effectively detecting UAVs within single-frame, multimodal static images without requiring image alignment. The results show that the average precision 0.5:0.95 (AP(0.5:0.95)) of the UAV instance in the validation task is 66.1%. Furthermore, RTM-UAVDet operates at a speed of 72.4 frames per second, fully satisfying the requirements for real-time processing and effectively recognizing UAVs larger than 48 (6 x 8) pixels.

关键词： Autonomous aerial vehicles Feature extraction Accuracy detection algorithms Real-time systems Radar detection Aerospace and electronic systems multimodal object detection multimodal image processing real-time image processing unmanned aerial vehicle (UAV) detection

来源：评论

学校读者我要写书评

暂无评论

Online monitoring method for chip pin with minor deformation defects based on depth-histogram modalities and target-oriented multimodal self-attention mechanism

引用

JOURNAL OF MANUFACTURING PROCESSES 2024年 131卷 1158-1167页

作者： Du, Changdu Xu, Lei Chen, Jun He, Nachuan Sichuan Univ Sch Mech Engn Chengdu 610065 Peoples R China

In the process of chip SMT (surface mounting technology), the quality of the chip pins determines the success rate of the mounting process. However, existing target detection algorithms present poor performance when dealing with deformations in the pins, which is insufficient to meet the industrial demands for accuracy and speed of online monitoring. To solve this problem, a real-time detection method based on D-H (Depth-Histogram) Modalities and TMSM (Target-oriented multimodal Self-attention Mechanism) is proposed. There are three parts in this method, including feature extraction, feature fusion, and decision module. Firstly, a lightweight network for feature extraction and fusion is employed to extract geometric information from the depth images. Subsequently, the Decision Module is used to determine whether there are defects in the pins. Within this framework, the HIEF (Histogram-Integrated Embedding Function) is utilized to extract a one-dimensional vector with height information from the histogram, which is then aligned with the flattened depth image to form D-H Modalities. To validate the effectiveness of the proposed algorithm, two datasets are constructed. Experimental results demonstrate that the proposed method has a good performance to meet the speed and accuracy requirements of online monitoring.

关键词： Defect detection multimodal object detection Self-attention mechanism Online monitoring

来源：评论

学校读者我要写书评

暂无评论

Coarse to fine-based image-point cloud fusion network for 3D object detection

引用

INFORMATION FUSION 2024年 112卷

作者： Hao, Meilan Zhang, Zhongkang Li, Lei Dong, Kejian Cheng, Long Tiwari, Prayag Ning, Xin Hebei Univ Engn Sch Informat & Elect Engn Handan 056038 Peoples R China Chinese Acad Sci Inst Semicond Beijing 100083 Peoples R China North China Elect Power Univ Sch Control & Comp Engn Beijing 102206 Peoples R China Halmstad Univ Sch Informat Technol SE-30118 Halmstad Sweden Beijing Ratu Technol Co Ltd Beijing 100096 Peoples R China

Enhancing original LiDAR point cloud features with virtual points has gained widespread attention in multimodal information fusion. However, existing methods struggle to leverage image depth information due to the sparse nature of point clouds, hindering proper alignment with camera-derived features. We propose a novel 3D object detection method that refines virtual point clouds using a coarse-to-fine approach, incorporating a dynamic 2D Gaussian distribution for better matching and a dynamic posterior density-aware RoI network for refined feature extraction. Our method achieves an average precision (AP) of 90.02% for moderate car detection on the KITTI validation set, outperforming state-of-the-art methods. Additionally, our approach yields AP scores of 86.58% and 82.16% for moderate and hard car detection categories on the KITTI test set, respectively. These results underscore the effectiveness of our method in addressing point cloud sparsity and enhancing 3D object detection performance. The code is available at https://***/ZhongkangZ/LidarIG.

关键词： Coarse to fine Image-point cloud fusion multimodal object detection Dynamic 2D Gaussian distribution Quantized perception strategy

来源：评论

学校读者我要写书评

暂无评论

Domain Adaptation for Infrared-Radar Cross-Scene multimodal detection 11

Domain Adaptation for Infrared-Radar Cross-Scene Multimodal ...

引用

11th IEEE International Conference on Cybernetics and Intelligent Systems (CIS) / 11th IEEE International Conference on Robotics, Automation and Mechatronics (RAM)

作者： Chain, Bosong Zheng, Qian Nie, Xuan Jia, Jianchao Pan, Gang Zhejiang Univ Hangzhou Peoples R China Northwestern Polytech Univ Xian Peoples R China

ISBN: (纸本)9798350364200;9798350364194

Utilizing the advantages of both single-mode detections in infrared-radar dual-mode guidance systems is a crucial research direction for precise weapon guidance. Infrared images and radar high resolution range profile (HRRP) have high-dimensional feature spaces, leading to a certain degree of data heterogeneity. The difficulty of capturing targets varies across different scenes, leading to an imbalance in data distribution and consequently, a poor generalization ability of the model. We aim to improve the performance in the target domain by leveraging the knowledge from the source domain and adjusting the model to adapt to the data distribution in the target domain. We propose a dual-mode object detection algorithm, CenterNet-PK, which effectively extracts radar features and establishes temporal correlations using one-dimensional convolution and Bidirectional Gated Recurrent Units (Bi-GRU). Additionally, it incorporates Cross-modal Attention to fully leverage radar features for infrared detection tasks. Building upon this, we develop a domain adaptation (DA) multimodal detection framework to enhance the generalization of the model in various complex scenarios, employing adversarial training to narrow the domain shift. Experimental results demonstrate that the proposed algorithm achieves high detection accuracy and robustness in infrared-radar dual-mode detection tasks. Extensive experiments on DA-detection task from urban to outdoor scenes show that the proposed framework outperforms the baseline. The code is available at https://***/chaibosong/CIS-RAM.

关键词： Domain adaptation multimodal object detection Cross-modal attention

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：