检索结果-内蒙古大学图书馆

MRCap: Multi-modal and Multi-level Relationship-based Dense Video Captioning

学校读者我要写书评

暂无评论

MRCap: Multi-modal and Multi-level Relationship-based Dense ...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Wei Chen Jianwei Niu Xuefeng Liu State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing China Zhongguancun Laboratory Beijing China Zhengzhou University Research Institute of Industrial Technology School of Information Engineering Zhengzhou University Zhengzhou China

Dense video captioning, with the objective of describing a sequence of events in a video, has received much attention recently. As events in a video are highly correlated, leveraging relationships among events helps generate coherent captions. To utilize relationships among events, existing methods mainly enrich event representations with their context, either in the form of vision (i.e., video segments) or combining vision and language (i.e., captions). However, these methods do not explicitly exploit the correspondence between these two modalities. Moreover, the video-level context spanning multiple events is not fully exploited. In this paper, we propose MRCap, a novel relationship-based model for dense video captioning. The key of MRCap is a multi-modal and multi-level event relationship module (MMERM). MMERM exploits the correspondence between vision and language at both the event level and the video level via contrastive learning. Experiments on ActivityNet Captions and YouCook2 datasets demonstrate that MRCap achieves state-of-the-art performance.

关键词：

A New Co-simulation Method of Radar Detection Range under Suppression Jamming

学校读者我要写书评

暂无评论

A New Co-simulation Method of Radar Detection Range under Su...

IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Jiani An Jin Xiao Xiaoguang Hu Qing Zhou State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China China National Aeronautical Radio Electronics Research Institute Shanghai China

ISBN: (纸本)9781665409858

Military radar is often used to collect information of battlefield situation, detect targets and guide weapon systems to lock them. Electronic warfare is a counter-measure against enemy radar. In this paper, the co-simulation method of systems Tool Kit (STK) and MATLAB is proposed to simulate the detection range of enemy radar suppressed by the jammer in three-dimensional electronic warfare environment. Firstly, Radar detection range models on 2D and 3D are established respectively by combining horizontal and vertical direction functions with basic radar equations. Secondly, radar detection range variation under selfscreening jamming (SSJ) of one aircraft is studied. Then, the leader-follower algorithm is used to realize escort jamming (ESJ) model under multi-aircraft formation. Finally, the radar detection range of multiple aircraft jamming is compared with the range of a single aircraft jamming. The 3D radar detection range simulation results in STK vividly show that adopting multiaircraft jamming can greatly reduce the detection range of enemy radar and successfully strike the enemy.

关键词： Solid modeling Three-dimensional displays Simulation Radar detection Radar Aerospace electronics Mathematical models

FSD-BEV: Foreground Self-Distillation for Multi-view 3D Object Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiang, Zheng Zhang, Jinqing Zhang, Yanan Liu, Qingjie Hu, Zhenghui Wang, Baohui Wang, Yunhong Hangzhou Innovation Institute Beihang University Hangzhou China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China School of Software Beihang University Beijing China Shanghai ZEEKR Blue New Energy Technology Co. Ltd. China Zhongguancun Laboratory Beijing China

Although multi-view 3D object detection based on the Bird's-Eye-View (BEV) paradigm has garnered widespread attention as an economical and deployment-friendly perception solution for autonomous driving, there is still a performance gap compared to LiDAR-based methods. In recent years, several cross-modal distillation methods have been proposed to transfer beneficial information from teacher models to student models, with the aim of enhancing performance. However, these methods face challenges due to discrepancies in feature distribution originating from different data modalities and network structures, making knowledge transfer exceptionally challenging. In this paper, we propose a Foreground Self-Distillation (FSD) scheme that effectively avoids the issue of distribution discrepancies, maintaining remarkable distillation effects without the need for pre-trained teacher models or cumbersome distillation strategies. Additionally, we design two Point Cloud Intensification (PCI) strategies to compensate for the sparsity of point clouds by frame combination and pseudo point assignment. Finally, we develop a Multi-Scale Foreground Enhancement (MSFE) module to extract and fuse multi-scale foreground features by predicted elliptical Gaussian heatmap, further improving the model's performance. We integrate all the above innovations into a unified framework named FSD-BEV. Extensive experiments on the nuScenes dataset exhibit that FSD-BEV achieves state-of-the-art performance, highlighting its effectiveness. The code and models are available at: https://***/CocoBoom/fsd-bev. Copyright © 2024, The Authors. All rights reserved.

关键词： Object detection

Context-Enhanced Detector For Building Detection From Remote Sensing Images

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Ziyue Zhang, Mingming Liu, Qingjie Wang, Wei Dong, Zhe Wang, Yunhong The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Hangzhou Innovation Institute Beihang University Hangzhou310051 China The National Disaster Reduction Center of China Beijing100124 China

The field of building detection from remote sensing images has made significant progress, but faces challenges in achieving high-accuracy detection due to the diversity in building appearances and the complexity of vast scenes. To address these challenges, we propose a novel approach called Context-Enhanced Detector (CEDet). Our approach utilizes a three-stage cascade structure to enhance the extraction of contextual information and improve building detection accuracy. Specifically, we introduce two modules: the Semantic Guided Contextual Mining (SGCM) module, which aggregates multi-scale contexts and incorporates an attention mechanism to capture long-range interactions, and the Instance Context Mining Module (ICMM), which captures instance-level relationship context by constructing a spatial relationship graph and aggregating instance features. Additionally, we introduce a semantic segmentation loss based on pseudo-masks to guide contextual information extraction. Our method achieves state-of-the-art performance on three building detection benchmarks, including CNBuilding-9P, CNBuilding-23P, and SpaceNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantic Segmentation

LFNAT 2023 Challenge on Light Field Depth Estimation: Methods and Results

学校读者我要写书评

暂无评论

LFNAT 2023 Challenge on Light Field Depth Estimation: Method...

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023

作者： Sheng, Hao Liu, Yebin Yu, Jingyi Wu, Gaochang Xiong, Wei Guo, Longzhao Xie, Yanlin Zhang, Shuo Chang, Song Lin, Youfang Chao, Wentao Wang, Xuechun Wang, Guanghui Duan, Fuqing Wang, Tun Yang, Da Cui, Zhenglong Wang, Sizhe Zhao, Mingyuan Wang, Qiong Chen, Qianyu Liang, Zhengyu Wang, Yingqian Yang, Jungang Yang, Xueting Deng, Junli Cong, Ruixuan Chen, Rongshan State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University China Beihang Hangzhou Innovation Institute Yuhang China Faculty of Applied Sciences Macao Polytechnic University China Tsinghua University China Shanghaitech University China State Key Laboratory of Synthetical Automation for Process Industries Northeastern University China Beijing Meet Yuan Co. Ltd China Beijing Key Laboratory of Traffic Data Analysis and Mining School of Computer and Information Technology Beijing Jiaotong University China Beijing Normal University China Toronto Metropolitan University Canada College of Computer Science and Technology Zhejiang University of Technology China National University of Defense Technology China School of Information and Communication Engineering Communication University of China China

ISBN: (纸本)9798350302493

This paper reviews the 1st LFNAT challenge on light field depth estimation, which aims at predicting disparity information of central view image in a light field (i.e., pixel offset between central view image and adjacent view image). Compared to multi-view stereo matching, light field depth estimation emphasizes efficient utilization of the 2D angular information from multiple regularly varying views. This challenge specifies UrbanLF [20] light field dataset as the sole data source. There are two phases in total: submission phase and final evaluation phase, in which 75 registered participants successfully submit their predicted results in the first phase and 7 eligible teams compete in the second phase. The performance of all submissions is carefully reviewed and shown in this paper as a new standard for the current state-of-the-art in light field depth estimation. Moreover, the implementation details of these methods are also provided to stimulate related advanced research. © 2023 IEEE.

关键词： Stereo image processing

Extrinsic Parameter Calibration and Dynamic Objects Coping Strategy for Multi-camera SLAM

学校读者我要写书评

暂无评论

Extrinsic Parameter Calibration and Dynamic Objects Coping S...

Computer Science and Management technology (ICCSMT), International Conference on

作者： Yunlong Dong Wei Guo Fusheng Zha Pengfei Wang State Key Laboratory of Robotics and Systems Harbin Institute of Technology Harbin China

Perceiving the surrounding environment timely and accurately is very essential to improve the safety of mobile robots. To obtain rich information and increase the visual field of mobile robots, the development of multi-camera simultaneous localization and mapping (SLAM) system has become a hot research field. In the current research, multi-camera extrinsic parameter calibration and multi-camera dynamic SLAM are still challenging. Therefore, this paper proposes a dynamic environment-oriented multi-camera SLAM system. This system not only presents an accurate multi-camera extrinsic parameter calibration method but also can achieve accurate localization in the environment with dynamic objects. The effectiveness of the algorithm is verified through comprehensive experiments.

关键词：

SCARF: Scalable Continual Learning Framework for Memory-efficient Multiple Neural Radiance Fields

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Yuze Wang, Junyi Wang, Chen Duan, Wantong Bao, Yongtang Qi, Yue State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University China School of Computer Science and Technology Shandong University China School of Computer and Artificial Intelligence Beijing Technology and Business University China Jingdezhen Research Institute Beihang University China Shandong University of Science and Technology China

This paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of the upcoming new scene. We build on Neural Radiance Fields (NeRF), which uses multi-layer perceptron to model the density and radiance field of a scene as the implicit function. While NeRF and its extensions have shown a powerful capability of rendering photo-realistic novel views in a single 3D scene, managing these growing 3D NeRF assets efficiently is a new scientific problem. Very few works focus on the efficient representation or continuous learning capability of multiple scenes, which is crucial for the practical applications of NeRF. To achieve these goals, our key idea is to represent multiple scenes as the linear combination of a cross-scene weight matrix and a set of scene-specific weight matrices generated from a global parameter generator. Furthermore, we propose an uncertain surface knowledge distillation strategy to transfer the radiance field knowledge of previous scenes to the new model. Representing multiple 3D scenes with such weight matrices significantly reduces memory requirements. At the same time, the uncertain surface distillation strategy greatly overcomes the catastrophic forgetting problem and maintains the photo-realistic rendering quality of previous scenes. Experiments show that the proposed approach achieves state-of-the-art rendering quality of continual learning NeRF on NeRF-Synthetic, LLFF, and TanksAndTemples datasets while preserving extra low storage cost. Copyright © 2024, The Authors. All rights reserved.

关键词： Rendering (computer graphics)

Personalized Federated Learning with Collaborative Aggregation Networks for Multi-Site Brain Disorder Diagnosis

学校读者我要写书评

暂无评论

Personalized Federated Learning with Collaborative Aggregati...

Industrial Automation, robotics and Control Engineering (IARCE), International Conference on

作者： Qian Si Yang Li School of Cyber Science and Technology Beihang University Beijing China Department of Automation Science and Electrical Engineering Beijing Advanced Innovation Center for Big Data and Brain Computing State Key Laboratory of Virtual Reality Technology and Systems Advanced Institute of Information Technology Peking University Beihang University Beijing China

ISBN: (数字)9798350380323

ISBN: (纸本)9798350380330

In multi-site brain disease diagnosis studies, traditional centralized training methods necessitate sharing medical data, posing significant privacy risks. Federated learning (FL) offers a privacy-preserving solution by enabling global model training through aggregating locally trained models from multiple data centers without sharing raw data. However, current FL approaches rely on a server-based network topology, where central server failure disrupts training. Additionally, data heterogeneity across sites often slows convergence and reduces accuracy. To overcome these issues, we introduce a decentralized personalized federated learning collaborative aggregation network (pFedCAN). This framework has two core components: (1) separating local models into shared and personalized layers, and (2) forming a collaborative aggregation network via similarity detection in the shared layers. Specifically, each center trains its local model, then separates it into shared and personalized layers. The shared layer is exchanged with other centers, while the personalized layer remains local. Data centers analyze similarities in received shared layers to build a collaborative network, where shared layers from similar centers are aggregated to refine the model. This approach flexibly adapts to varying levels of data heterogeneity, enhancing model training efficiency. Validation on public datasets, ABIDE I and ADHD, shows that the proposed method outperforms current leading techniques.

关键词： Training Adaptation models Data centers Federated learning Service robots Collaboration Brain modeling Data models Servers Medical diagnostic imaging

Dynamic Arthroscopic Navigation System for Anterior Cruciate Ligament Reconstruction Based on Multi-level Memory Architecture

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Shuo Shi, Weili Yang, Shuai Cui, Jiahao Guo, Qinwei Department of Engineering Physics Key Laboratory of Particle and Radiation Imaging Ministry of Education Tsinghua University Beijing100084 China Department of Sports Medicine Peking University Third Hospital Institute of Sports Medicine of Peking University Beijing Key Laboratory of Sports Injuries Haidian District Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

This paper presents a dynamic arthroscopic navigation system based on multi-level memory architecture for anterior cruciate ligament (ACL) reconstruction surgery. The system extends our previously proposed markerless navigation method from static image matching to dynamic video sequence tracking. ACL reconstruction surgery typically lasts 60-90 minutes, generating lengthy video sequences that require continuous interpretation and navigation, while the arthroscopic environment presents unique challenges including narrow and constantly changing field of view, specular reflections from moist tissue surfaces, image blurring caused by irrigation fluid, and frequent occlusion of target structures by surgical instruments. By integrating the Atkinson-Shiffrin memory model's three-level memory architecture (sensory memory, working memory, and long-term memory), our system maintains continuous tracking of the femoral condyle throughout the surgical procedure, providing stable navigation support even in complex situations involving viewpoint changes, instrument occlusion, and tissue deformation. Unlike existing methods, our system operates in real-time on standard arthroscopic equipment without requiring additional tracking hardware, achieving 25.3 FPS with a latency of only 39.5 ms, representing a 3.5-fold improvement over our previous static system. For extended sequences (1000 frames), the dynamic system maintained an error of 5.3±1.5 pixels, compared to the static system's 12.6± 3.7 pixels—an improvement of approximately 45%. For medium-length sequences (500 frames) and short sequences (100 frames), the system achieved approximately 35% and 19% accuracy improvements, respectively. Experimental results demonstrate that the system overcomes the limitations of traditional static matching methods, providing new technical support for improving surgical precision and reducing complications in ACL reconstruction. © 2025, CC BY.

关键词： Surgical equipment