检索结果-内蒙古大学图书馆

PanoVpr: Towards Unified Perspective-to-Equirectangular visual Place Recognition via Sliding Windows Across the Panoramic View

学校读者我要写书评

暂无评论

PanoVpr: Towards Unified Perspective-to-Equirectangular Visu...

International Conference on Intelligent Transportation

作者： Ze Shi Hao Shi Kailun Yang Zhe Yin Yining Lin Kaiwei Wang State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou China Shanghai SUPREMIND Technology Co. Ltd Shanghai China National Engineering Research Center of Robot Visual Perception and Control Technology School of Robotics Hunan University Changsha China

visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose PanoVPR, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at PanoVPR.

关键词：

Towards Consistent Object Detection via LiDAR-Camera Synergy

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Luo, Kai Wu, Hao Yi, Kefu Yang, Kailun Hao, Wei Hu, Rongdong School of Traffic and Transportation Engineering Changsha University of Science and Technology China College of Automotive and Mechanical Engineering Changsha University of Science and Technology China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China Changsha Intelligent Driving Institute China

As human-machine interaction continues to evolve, the capacity for environmental perception is becoming increasingly crucial. Integrating the two most common types of sensory data, images, and point clouds, can enhance detection accuracy. Currently, there is no existing model capable of detecting an object’s position in both point clouds and images while also determining their corresponding relationship. This information is invaluable for human-machine interactions, offering new possibilities for their enhancement. In light of this, this paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework that requires only a single forward inference to simultaneously obtain an object’s position in both point clouds and images and establish their correlation. Furthermore, to assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision (CP). To verify the effectiveness of the proposed framework, an extensive set of experiments has been conducted on the KITTI and DAIR-V2X datasets. The study also explored how the proposed consistency detection method performs on images when the calibration parameters between images and point clouds are disturbed, compared to existing post-processing methods. The experimental results demonstrate that the proposed method exhibits excellent detection performance and robustness, achieving end-to-end consistency detection. The source code will be made publicly available at https://***/xifen523/COD. Copyright © 2024, The Authors. All rights reserved.

关键词： Image enhancement

Towards Consistent Object Detection via LiDAR-Camera Synergy

学校读者我要写书评

暂无评论

Towards Consistent Object Detection via LiDAR-Camera Synergy

IEEE International Conference on Systems, Man and Cybernetics

作者： Kai Luo Hao Wu Kefu Yi Kailun Yang Wei Hao Rongdong Hu College of Automotive and Mechanical Engineering Changsha University of Science and Technology China School of Traffic and Transportation Engineering Changsha University of Science and Technology China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China Changsha Intelligent Driving Institute China

ISBN: (数字)9781665410205

ISBN: (纸本)9781665410212

As human-machine interaction continues to evolve, the capacity for environmental perception is becoming increasingly crucial. Integrating the two most common types of sensory data, images, and point clouds, can enhance detection accuracy. Currently, there is no existing model capable of detecting an object's position in both point clouds and images while also determining their corresponding relationship. This information is invaluable for human-machine interactions, offering new possibilities for their enhancement. In light of this, this paper introduces an end-to-end Consistency Object Detection (COD) algorithm framework that requires only a single forward inference to simultaneously obtain an object's position in both point clouds and images and establish their correlation. Furthermore, to assess the accuracy of the object correlation between point clouds and images, this paper proposes a new evaluation metric, Consistency Precision (CP). To verify the effectiveness of the proposed framework, an extensive set of experiments has been conducted on the KITTI and DAIR-V2X datasets. The study also explored how the proposed consistency detection method performs on images when the calibration parameters between images and point clouds are disturbed, compared to existing post-processing methods. The experimental results demonstrate that the proposed method exhibits ex-cellent detection performance and robustness, achieving end-to-end consistency detection. The source code will be made publicly available at https://***/xifen523/COD.

关键词： Point cloud compression Human computer interaction Measurement Accuracy Correlation Source coding Instruments Object detection Robustness Inference algorithms

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiao, Jianbin Cheng, Xina Chen, Weijie Yin, Xiaoting Shi, Hao Yang, Kailun School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as human-computer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable self-attention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multi-perspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves state-of-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose. Copyright © 2024, The Authors. All rights reserved.

关键词： Human computer interaction

Towards Single-Lens controllable Depth-of-Field Imaging via Depth-Aware Point Spread Functions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qian, Xiaolong Jiang, Qi Gao, Yao Gao, Shaohua Yi, Zhonghua Sun, Lei Wei, Kai Li, Haifeng Yang, Kailun Wang, Kaiwei Bai, Jian State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. To the best of our knowledge, we are the first to explore the single-lens controllable DoF imaging solution. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at https://***/XiaolongQian/DCDI. Copyright © 2024, The A

关键词： Optical transfer function

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

Towards Precise 3D Human Pose Estimation with Multi-Perspect...

International Joint Conference on Neural Networks (IJCNN)

作者： Jianbin Jiao Xina Cheng Weijie Chen Xiaoting Yin Hao Shi Kailun Yang School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as humancomputer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable selfattention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multiperspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves stateof-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose.

关键词： Training Three-dimensional displays Correlation Computational modeling Source coding Pose estimation Video sequences

SF-TIM: A Simple Framework for Enhancing Quadrupedal robot Jumping Agility by Combining Terrain Imagination and Measurement

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Ze Li, Yang Xu, Long Shi, Hao Ma, Zunwang Chu, Zhen Li, Chao Gao, Fei Yang, Kailun Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University China School of Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China DeepRobotics Co. Ltd. China

Dynamic jumping on high platforms and over gaps differentiates legged robots from wheeled counterparts. Compared to walking on rough terrains, dynamic locomotion on abrupt surfaces requires fusing proprioceptive and exteroceptive perception for explosive movements. In this paper, we propose SF-TIM (Simple Framework combining Terrain Imagination and Measurement), a single-policy method that enhances quadrupedal robot jumping agility, while preserving their fundamental blind walking capabilities. In addition, we introduce a terrain-guided reward design specifically to assist quadrupedal robots in high jumping, improving their performance in this task. To narrow the simulation-to-reality gap in quadrupedal robot learning, we introduce a stable and high-speed elevation map generation framework, enabling zero-shot simulation-to-reality transfer of locomotion ability. Our algorithm has been deployed and validated on both the small-/large-size quadrupedal robots, demonstrating its effectiveness in real-world applications: the robot has successfully traversed various high platforms and gaps, showing the robustness of our proposed approach. A demo video has been made available at https://***/SF-TIM. Copyright © 2024, The Authors. All rights reserved.

关键词： Multipurpose robots

Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gao, Yao Jiang, Qi Gao, Shaohua Sun, Lei Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

Recently, joint design approaches that simultaneously optimize optical systems and downstream algorithms through data-driven learning have demonstrated superior performance over traditional separate design approaches. However, current joint design approaches heavily rely on the manual identification of initial lenses, posing challenges and limitations, particularly for compound lens systems with multiple potential starting points. In this work, we present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications;and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution in all search results. Extensive experimental results illustrate that QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability, which automatically provides compound lens based computational imaging systems with higher imaging quality compared to existing paradigms. The source code will be made publicly available at https://***/LiGpy/QGSO. Copyright © 2024, The Authors. All rights reserved.

关键词： Imaging systems

Towards Source-Free Domain Adaptive Semantic Segmentation Via Importance-Aware and Prototype-Contrast Learning

学校读者我要写书评

暂无评论

IEEE Transactions on Intelligent Vehicles

IEEE Transactions on Intelligent Vehicles 2024年 1-13页

作者： Cao, Yihong Zhang, Hui Lu, Xiao Xiao, Zheng Yang, Kailun Wang, Yaonan College of Computer Science and Electronic Engineering Hunan University Changsha China National Engineering Research Center of Robot Vision Perception and Control Technology School of Robotics Hunan University Changsha China College of Engineering and Design Hunan Normal University Changsha China

Domain adaptive semantic segmentation enables robust pixel- wise understanding in real-world driving scenes. Source-free domain adaptation, as a more practical technique, addresses the concerns of data privacy and storage limitations in typical unsupervised domain adaptation methods, making it especially relevant in the context of intelligent vehicles. It utilizes a well-trained source model and unlabeled target data to achieve adaptation in the target domain. However, in the absence of source data and target labels, current solutions cannot sufficiently reduce the impact of domain shift and fully leverage the information from the target data. In this paper, we propose an end-to-end source-free domain adaptation semantic segmentation method via Importance-Aware and Prototype-Contrast (IAPC) learning. The proposed IAPC framework effectively extracts domain-invariant knowledge from the well-trained source model and learns domain-specific knowledge from the unlabeled target domain. Specifically, considering the problem of domain shift in the prediction of the target domain by the source model, we put forward an importance-aware mechanism for the biased target prediction probability distribution to extract domain-invariant knowledge from the source model. We further introduce a prototype-contrast strategy, which includes a prototype-symmetric cross-entropy loss and a prototype-enhanced cross-entropy loss, to learn target intra-domain knowledge without relying on labels. A comprehensive variety of experiments on two domain adaptive semantic segmentation benchmarks demonstrates that the proposed end-to-end IAPC solution outperforms existing state-of-the-art methods. The source code is publicly available at https://***/yihong-97/Source-free-IAPC. IEEE

关键词： Semantics