检索结果-内蒙古大学图书馆

LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Ze Yang, Kailun Shi, Hao Li, Peng Gao, Fei Bai, Jian Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China Huzhou Institute of Zhejiang University Zhejiang University China

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using (u, v, 1)T become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a Visual Inertial SLAM framework for cameras with extremely Large FoV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method Copyright © 2022, The Authors. All rights reserved.

关键词： Cameras

GenMapping: Unleashing the Potential of Inverse Perspective Mapping for Robust Online HD Map Construction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Siyu Yang, Kailun Shi, Hao Wang, Song Yao, You Li, Zhiyong The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China The College of Computer Science Zhejiang University Hangzhou310027 China Shanghai Supremind Technology Company Ltd. Shanghai201210 China The USC Viterbi School of Engineering The University of Southern California Los AngelesCA90089 United States

Online High-Definition (HD) maps have emerged as the preferred option for autonomous driving, overshadowing the counterpart offline HD maps due to flexible update capability and lower maintenance costs. However, contemporary online HD map models embed parameters of visual sensors into training, resulting in a significant decrease in generalization performance when applied to visual sensors with different parameters. Inspired by the inherent potential of Inverse Perspective Mapping (IPM), where camera parameters are decoupled from the training process, we have designed a universal map generation framework, GenMapping. The framework is established with a triadic synergy architecture, including principal and dual auxiliary branches. When faced with a coarse road image with local distortion translated via IPM, the principal branch learns robust global features under the state space models. The two auxiliary branches are a dense perspective branch and a sparse prior branch. The former exploits the correlation information between static and moving objects, whereas the latter introduces the prior knowledge of OpenStreetMap (OSM). The triple-enhanced merging module is crafted to synergistically integrate the unique spatial features from all three branches. To further improve generalization capabilities, a Cross-View Map Learning (CVML) scheme is leveraged to realize joint learning within the common space. Additionally, a Bidirectional Data Augmentation (BiDA) module is introduced to mitigate reliance on datasets concurrently. A thorough array of experimental results shows that the proposed model surpasses current state-of-the-art methods in both semantic mapping and vectorized mapping, while also maintaining a rapid inference speed. Moreover, in cross-dataset experiments, the generalization of semantic mapping is improved by 17.3% in mIoU, while vectorized mapping is improved by 12.1% in mAP. The source code will be publicly available at https://***/lynn-yu/GenMappin

关键词： Photomapping

FlowDriveNet: An End-to-End Network for Learning Driving Policies from Image Optical Flow and LiDAR Point Flow

学校读者我要写书评

暂无评论

FlowDriveNet: An End-to-End Network for Learning Driving Pol...

IEEE International Conference on robotics and Automation (ICRA)

作者： Shuai Wang Jiahu Qin Menglin Li Yaonan Wang University of Science and Technology of China Hefei China College of Electrical and Information Engineering Hunan University Changsha China National Engineering Laboratory for Robot Visual Perception and Control Technology Changsha China

Learning driving policies using an end-to-end network has been proved a promising solution for autonomous driving. Due to the lack of a benchmark driver behavior dataset that contains both the visual and the LiDAR data, existing works solely focus on learning driving from visual sensors. Besides, most works are limited to predict steering angle yet neglect the more challenging vehicle speed control problem. In this paper, we propose a novel end-to-end network, FlowDriveNet, which takes advantages of sequential visual data and LiDAR data jointly to predict steering angle and vehicle speed. The main challenges of this problem are how to efficiently extract driving-related information from images and point clouds, and how to fuse them effectively. To tackle these challenges, we propose a concept of point flow and declare that image optical flow and LiDAR point flow are significant motion cues for driving policy learning. Specifically, we first create an enhanced dataset that consists of images, point clouds and corresponding human driver behaviors. Then, in FlowDriveNet, a deep but efficient visual feature extraction module and a point feature extraction module are utilized to extract spatial features from optical flow and point flow, respectively. Additionally, a novel temporal fusion and prediction module is designed to fuse temporal information from the extracted spatial feature sequences and predict vehicle driving commands. Finally, a series of ablation experiments verify the importance of optical flow and point flow and comparison experiments show that our flow-based method outperforms the existing image-based approaches on the task of driving policy learning.

关键词： Image motion analysis Computer vision Visualization Laser radar Fuses Predictive models Feature extraction

Domain-invariant Prototypes for Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Zhengeng Yu, Hongshan Sun, Wei Li-Cheng Mian, Ajmal The National Engineering Laboratory for Robot Visual Perception and Control Technology College of Electrical and Information Engineering Hunan University Lushan South Rd. Yuelu410082 China The Department of Electrical and Computer Engineering University of Alberta EdmontonAB Canada The Department of Computer Science The University of Western Australia PerthWA6009 Australia

Deep Learning has greatly advanced the performance of semantic segmentation, however, its success relies on the availability of large amounts of annotated data for training. Hence, many efforts have been devoted to domain adaptive semantic segmentation that focuses on transferring semantic knowledge from a labeled source domain to an unlabeled target domain. Existing self-training methods typically require multiple rounds of training, while another popular framework based on adversarial training is known to be sensitive to hyper-parameters. In this paper, we present an easy-to-train framework that learns domain-invariant prototypes for domain adaptive semantic segmentation. In particular, we show that domain adaptation shares a common character with few-shot learning in that both aim to recognize some types of unseen data with knowledge learned from large amounts of seen data. Thus, we propose a unified framework for domain adaptation and few-shot learning. The core idea is to use the class prototypes extracted from few-shot annotated target images to classify pixels of both source images and target images. Our method involves only one-stage training and does not need to be trained on large-scale un-annotated target images. Moreover, our method can be extended to variants of both domain adaptation and few-shot learning. Experiments on adapting GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes show that our method achieves competitive performance to state-of-the-art. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantic Segmentation

Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shi, Hao Wang, Song Zhang, Jiaming Yin, Xiaoting Wang, Guangming Zhu, Jianke Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China School of Robotics National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China College of Computer Science and Technology Zhejiang University Hangzhou310027 China Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany Department of Engineering University of Cambridge CambridgeCB2 1PZ United Kingdom

Vision-based occupancy prediction, also known as 3D Semantic Scene Completion (SSC), presents a significant challenge in computer vision. Previous methods, confined to onboard processing, struggle with simultaneous geometric and semantic estimation, continuity across varying viewpoints, and single-view occlusion. Our paper introduces OccFiner, a novel offboard framework designed to enhance the accuracy of vision-based occupancy predictions. OccFiner operates in two hybrid phases: 1) a multi-to-multi local propagation network that implicitly aligns and processes multiple local frames for correcting onboard model errors and consistently enhancing occupancy accuracy across all distances. 2) the region-centric global propagation, focuses on refining labels using explicit multi-view geometry and integrating sensor bias, particularly for increasing the accuracy of distant occupied voxels. Extensive experiments demonstrate that OccFiner improves both geometric and semantic accuracy across various types of coarse occupancy, setting a new state-of-the-art performance on the SemanticKITTI dataset. Notably, OccFiner significantly boosts the performance of vision-based SSC models, achieving accuracy levels competitive with established LiDAR-based onboard SSC methods. Furthermore, OccFiner is the first to achieve automatic annotation of SSC in a purely vision-based approach. Quantitative experiments prove that OccFiner successfully facilitates occupancy data loop-closure in autonomous driving. Additionally, we quantitatively and qualitatively validate the superiority of the offboard approach on city-level SSC static maps. The source code will be made publicly available at https://***/MasterHow/OccFiner. Copyright © 2024, The Authors. All rights reserved.

关键词： Forecasting

Diff9D: Diffusion-Based Domain-Generalized Category-Level 9-DoF Object Pose Estimation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Liu, Jian Sun, Wei Yang, Hui Deng, Pengchao Liu, Chongpei Sebe, Nicu Rahmani, Hossein Mian, Ajmal National Engineering Research Center for Robot Visual Perception and Control Technology College of Electrical and Information Engineering School of Robotics State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body Hunan University Changsha410082 China Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University Xi’an710049 China Department of Information Engineering and Computer Science University of Trento Trento38123 Italy School of Computing and Communications Lancaster University LA1 4YW United Kingdom Department of Computer Science The University of Western Australia WA6009 Australia

Nine-degrees-of-freedom (9-DoF) object pose and size estimation is crucial for enabling augmented reality and robotic manipulation. Category-level methods have received extensive research attention due to their potential for generalization to intra-class unknown objects. However, these methods require manual collection and labeling of large-scale real-world training data. To address this problem, we introduce a diffusion-based paradigm for domain-generalized category-level 9-DoF object pose estimation. Our motivation is to leverage the latent generalization ability of the diffusion model to address the domain generalization challenge in object pose estimation. This entails training the model exclusively on rendered synthetic data to achieve generalization to real-world scenes. We propose an effective diffusion model to redefine 9-DoF object pose estimation from a generative perspective. Our model does not require any 3D shape priors during training or inference. By employing the Denoising Diffusion Implicit Model, we demonstrate that the reverse diffusion process can be executed in as few as 3 steps, achieving near real-time performance. Finally, we design a robotic grasping system comprising both hardware and software components. Through comprehensive experiments on two benchmark datasets and the real-world robotic system, we show that our method achieves state-of-the-art domain generalization performance. Our code will be made public at https://***/CNJianLiu/Diff9D. Copyright © 2025, The Authors. All rights reserved.

关键词： Augmented reality

Towards Anytime Optical Flow Estimation with Event Cameras

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ye, Yaozu Shi, Hao Yang, Kailun Wang, Ze Yin, Xiaoting Lin, Yining Liu, Mao Wang, Yaonan Wang, Kaiwei The State Key Laboratory of Modern Optical Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Co. Ltd Shanghai201210 China Dongguan Yutong Optical Technology Co. Ltd Dongguan523866 China

Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at https://***/Yaozhuwa/EVA-Flow. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

Double PWM coordinated control based on model predictive algorithm and power compensation

学校读者我要写书评

暂无评论

International Journal of Reasoning-based Intelligent Systems 2019年第1期11卷 38-46页

作者： Fan, Bo Song, Lu Song, Shuzhong Pu, Jiexin Information Engineering College Henan University of Science and Technology Luoyang China Henan Provincial Laboratory on Intelligent Manufactory and Control Engineering Luoyang471003 China Henan Key Laboratory of Robot and Intelligent Systems Luoyang471003 China

With analysis on double PWM structure though system's energy flow theory, active power and reactive power of rectifier are controlled by model predictive algorithm. System adopts the method of combine dynamic power's compensation with static power's compensation for system power in order to reduce system power error. A new power compensation algorithm is proposed to designs a new controller to replace PI controller of system's voltage loop, which can restrain fluctuation of DC bus voltage when load power suddenly varies and reduces DC-link's capacity of capacitance. As for inverter side, the system uses the method of rotor flux linkage oriented to control three phase asynchronous motor. Results of simulation can show that, system can realise the smallest tracking error of active power and reactive power by model predictive control, and restrain fluctuation of DC bus when load power suddenly changes. As the same time, the gird current's waveform is well. © 2019 Inderscience Enterprises Ltd.

关键词： Reactive power

Beyond the Field-of-View: Enhancing Scene Visibility and perception with Clip-Recurrent Transformer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shi, Hao Jiang, Qi Yang, Kailun Yin, Xiaoting Wang, Ze Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation the National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd Shanghai201210 China

Vision sensors are widely applied in vehicles, robots, and roadside infrastructure. However, due to limitations in hardware cost and system size, camera Field-of-View (FoV) is often restricted and may not provide sufficient coverage. Nevertheless, from a spatiotemporal perspective, it is possible to obtain information beyond the camera’s physical FoV from past video streams. In this paper, we propose the concept of online video inpainting for autonomous vehicles to expand the field of view, thereby enhancing scene visibility, perception, and system safety. To achieve this, we introduce the FlowLens architecture, which explicitly employs optical flow and implicitly incorporates a novel clip-recurrent transformer for feature propagation. FlowLens offers two key features: 1) FlowLens includes a newly designed Clip-Recurrent Hub with 3D-Decoupled Cross Attention (DDCA) to progressively process global information accumulated over time. 2) It integrates a multi-branch Mix Fusion Feed Forward Network (MixF3N) to enhance the precise spatial flow of local features. To facilitate training and evaluation, we derive the KITTI360 dataset with various FoV mask, which covers both outer- and inner FoV expansion scenarios. We also conduct both quantitative assessments and qualitative comparisons of beyond-FoV semantics and beyond-FoV object detection across different models. We illustrate that employing FlowLens to reconstruct unseen scenes even enhances perception within the field of view by providing reliable semantic context. Extensive experiments and user studies involving offline and online video inpainting, as well as beyond-FoV perception tasks, demonstrate that FlowLens achieves state-of-the-art performance. The source code and dataset are made publicly available at https://github. com/MasterHow/FlowLens. Copyright © 2022, The Authors. All rights reserved.

关键词： Optical flows