检索结果-内蒙古大学图书馆

arXiv 2022年

作者： Jiang, Qi Shi, Hao Gao, Shaohua Zhang, Jiaming Yang, Kailun Sun, Lei Ni, Huajian Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd Shanghai201210 China

Semantic scene understanding with Minimalist Optical Systems (MOS) in mobile and wearable applications remains a challenge due to the corrupted imaging quality induced by optical aberrations. However, previous works only focus on improving the subjective imaging quality through the Computational Imaging (CI) technique, ignoring the feasibility of advancing semantic segmentation. In this paper, we pioneer the investigation of Semantic Segmentation under Optical Aberrations (SSOA) with MOS. To benchmark SSOA, we construct Virtual Prototype Lens (VPL) groups through optical simulation, generating Cityscapes-ab and KITTI-360-ab datasets under different behaviors and levels of aberrations. We look into SSOA via an unsupervised domain adaptation perspective to address the scarcity of labeled aberration data in real-world scenarios. Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA. Based on our benchmark, we conduct experiments on the robustness of classical segmenters against aberrations. In addition, extensive evaluations of possible solutions to SSOA reveal that CIADA achieves superior performance under all aberration distributions, bridging the gap between computational imaging and downstream applications for MOS. The project page is at https://***/zju-jiangqi/CIADA. Copyright © 2022, The Authors. All rights reserved.

关键词： Aberrations

来源：评论

学校读者我要写书评

暂无评论

LF-PGVIO: A visual-Inertial-Odometry Framework for Large Field-of-View Cameras using Points and Geodesic Segments

arXiv

引用

arXiv 2023年

作者： Wang, Ze Yang, Kailun Shi, Hao Zhang, Yufan Xu, Zhijie Gao, Fei Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computing and Engineering University of Huddersfield HuddersfieldHD1 3DH United Kingdom The State Key Laboratory of Industrial Control Technology Zhejiang University Hangzhou310027 China

In this paper, we propose LF-PGVIO, a visual-Inertial-Odometry (VIO) framework for large Field-of-View (FoV) cameras with a negative plane using points and geodesic segments. The purpose of our research is to unleash the potential of point-line odometry with large-FoV omnidirectional cameras, even for cameras with negative-plane FoV. To achieve this, we propose an Omnidirectional Curve Segment Detection (OCSD) method combined with a camera model which is applicable to images with large distortions, such as panoramic annular images, fisheye images, and various panoramic images. The geodesic segment is sliced into multiple straight-line segments based on the radian and descriptors are extracted and recombined. Descriptor matching establishes the constraint relationship between 3D line segments in multiple frames. In our VIO system, line feature residual is also extended to support large-FoV cameras. Extensive evaluations on public datasets demonstrate the superior accuracy and robustness of LF-PGVIO compared to state-of-the-art methods. The source code will be made publicly available at https://***/flysoaryun/LF-PGVIO. Copyright © 2023, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

arXiv

引用

arXiv 2023年

作者： Yi, Zhonghua Shi, Hao Yang, Kailun Jiang, Qi Ye, Yaozu Wang, Ze Ni, Huajian Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics Hunan University Changsha410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd. Shanghai201210 China

Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point control Loss (CPCL) function for diverse point-wise supervision;2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5% precision improvement on various key points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at https://***/ZhonghuaYi/FocusFlow_official. Copyright © 2023, The Authors. All rights reserved.

关键词： Autonomous vehicles

来源：评论

学校读者我要写书评

暂无评论

Exploring Event-based Human Pose Estimation with 3D Event Representations

arXiv

引用

arXiv 2023年

作者： Yin, Xiaoting Shi, Hao Chen, Jiaan Wang, Ze Ye, Yaozu Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd. Shanghai201210 China

Human pose estimation is a fundamental and appealing task in computer vision. Although traditional cameras are commonly applied, their reliability decreases in scenarios under high dynamic range or heavy motion blur, where event cameras offer a robust solution. Predominant event-based methods accumulate events into frames, ignoring the asynchronous and high temporal resolution that is crucial for distinguishing distinct actions. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC aggregates events within concise temporal slices at identical positions, preserving their 3D attributes along with statistical information, thereby significantly reducing memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. Our methods are tested on the DHP19 public dataset, MMHPSD dataset, and our EV-3DPW dataset, with further qualitative validation via a derived driving scene dataset EV–JAAD and an outdoor collection vehicle. Our code and dataset have been made publicly available at https://***/MasterHow/EventPointPose. Copyright © 2023, The Authors. All rights reserved.

关键词： 3D modeling

来源：评论

学校读者我要写书评

暂无评论

Towards Anytime Optical Flow Estimation with Event Cameras

arXiv

引用

arXiv 2023年

作者： Ye, Yaozu Shi, Hao Yang, Kailun Wang, Ze Yin, Xiaoting Lin, Yining Liu, Mao Wang, Yaonan Wang, Kaiwei The State Key Laboratory of Modern Optical Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Co. Ltd Shanghai201210 China Dongguan Yutong Optical Technology Co. Ltd Dongguan523866 China

Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at https://***/Yaozhuwa/EVA-Flow. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

来源：评论

学校读者我要写书评

暂无评论

E-3DGS: Gaussian Splatting with Exposure and Motion Events

arXiv

引用

arXiv 2024年

作者： Yin, Xiaoting Shi, Hao Bao, Yuhan Bing, Zhenshan Liao, Yiyi Yang, Kailun Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University China School of Robotics National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China College of Information Science and Electronic Engineering Zhejiang University China The Chair of Robotics AI and Real-Time Systems Technical University of Munich Germany

Estimating Neural Radiance Fields (NeRFs) from images captured under optimal conditions has been extensively explored in the vision community. However, robotic applications often face challenges such as motion blur, insufficient illumination, and high computational overhead, which adversely affect downstream tasks like navigation, inspection, and scene visualization. To address these challenges, we propose E-3DGS, a novel event-based approach that partitions events into motion (from camera or object movement) and exposure (from camera exposure), using the former to handle fast-motion scenes and using the latter to reconstruct grayscale images for high-quality training and optimization of event-based 3D Gaussian Splatting (3DGS). We introduce a novel integration of 3DGS with exposure events for high-quality reconstruction of explicit scene representations. Our versatile framework can operate on motion events alone for 3D reconstruction, enhance quality using exposure events, or adopt a hybrid mode that balances quality and effectiveness by optimizing with initial exposure events followed by high-speed motion events. We also introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds. Our method is faster and delivers better reconstruction quality than event-based NeRF while being more cost-effective than NeRF methods that combine event and RGB data by using a single event sensor. By combining motion and exposure events, E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions and lower hardware demands. The source code and dataset will be available at https://***/MasterHow/E-3DGS. Copyright © 2024, The Authors. All rights reserved.

关键词： Gaussian distribution

来源：评论

学校读者我要写书评

暂无评论

Decentralized Multi-robot Navigation Coupled with Spatial-Temporal RetNet Based on Deep Reinforcement Learning

Decentralized Multi-Robot Navigation Coupled with Spatial-Te...

引用

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Lin Chen Yaonan Wang Zhiqiang Miao Mingtao Feng Yuanzhe Wang Yang Mo Zhen Zhou Hesheng Wang Danwei Wang School of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Artificial Intelligence Xidian University Xian China School of Electrical and Electrical Engineering Nanyang Technological University Nanyang Avenue Singapore Department of Automation Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Navigating robots through dynamic multi-robot environments, avoiding collisions with both other robots and obstacles, has emerged as a central challenge in robotics. The existing approaches fall short in allowing the policy network to effectively capture spatial-temporal reciprocal collision avoidance in multi-robot environments, comprising both static and dynamic obstacles, resulting in inadequate safety and efficiency in directing robot movement. In this study, we introduce a novel policy neural network called Spatial-Temporal RetNet (STR), designed to encode reciprocal collision avoidance states between robots in spatial and temporal dimensions. The goal is to improve the safety and efficacy of the policy neural network in directing robots to complete assigned tasks. The spatial state encoder module is built upon a parallel RetNet structure, which strengthens the neural network's capacity in extracting reciprocal collision avoidance states between robots in spatial dimensions. This module addresses the limitations of position encoding in transformer-based multi-robot navigation policy neural networks. We design a temporal state encoder utilizing a recurrent RetNet structure. This innovation bolsters the multi-robot navigation policy neural network's capability to capture features in the temporal dimension of multi-robot movements. It addresses the limitations of transformer-based multi-robot navigation policy neural networks, particularly in recurrently inferring information across time dimensions. Simulation experiments were conducted to showcase the superior safety and effectiveness of our proposed method compared to previous state-of-the-art approaches in guiding robots to accomplish tasks.

关键词： Training Technological innovation Navigation Neural networks Transformers Feature extraction Encoding Safety Collision avoidance robots

来源：评论

学校读者我要写书评

暂无评论

High-Speed Trajectory Tracking control for Quadrotors via Deep Reinforcement Learning

High-Speed Trajectory Tracking Control for Quadrotors via De...

引用

Youth Academic Annual Conference of Chinese Association of Automation (YAC)

作者： Hean Hua Hang Zhong Hui Zhang Yongchun Fang Yaonan Wang College of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Robotics Hunan University Changsha China Institute of Robotics and Automatic Information System College of Artificial Intelligence Nankai University Tianjin Key Laboratory of Intelligent Robotics Nankai University Tianjin China

ISBN: (数字)9798350379228

ISBN: (纸本)9798350390780

This paper presents a learning-based high-speed trajectory tracking control strategy for quadrotors, which achieves efficient learning and strong reliability by the collaboration of deep reinforcement learning (RL) and self-tuning mechanism. Different from existing methods, the proposed strategy is designed to explore optimal control performance by taking advantage of model-based self-tuning mechanism and deep reinforcement learning. Specifically, the self-tuning guided deep RL scheme is put forward for quadrotors, with superior learning efficiency and strong adaptability. Firstly, a novel self-tuning mechanism is constructed and some auxiliary variables are introduced to enhance the tracking performance. Then, based on the model-driven self-tuning design, the deep RL is proposed to achieve model-guided learning, where the tuning actions are adopted in the evaluation process during training, aiming at removing the bad explorations by the carefully designed parallel evaluation. Finally, the convergence is analyzed based on the proposed learning framework, which indicates the efficient cooperation of exploration and self-tuning mechanism. To verify the effectiveness of the proposed controller, the guided training and hardware experiments are implemented to show efficient cooperation and satisfactory high-speed trajectory tracking control of the proposed method.

关键词： Training Adaptation models Trajectory tracking Heuristic algorithms Optimal control Deep reinforcement learning Hardware

来源：评论

学校读者我要写书评

暂无评论

Decentralized Trajectory Planning for Formation Flight in Unknown and Dense Environments

Decentralized Trajectory Planning for Formation Flight in Un...

引用

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Jianxin Zeng Yaonan Wang Zhiqiang Miao Wei He Hesheng Wang Department of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Changsha China School of Intelligence Science and Technology University of Science and Technology Beiing Beijing China Key Laboratory of Intelligent Bionic Unmanned Systems Ministry of Education University of Science and Technology Beijing Beijing China Department of Automation Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

For aerial swarms, formation flight has been applied in various scenes. However, most existing works do not consider balancing the conflicting requirements among keeping formation, keeping the smoothness of trajectories, and obstacle avoidance within the limited time. To address this issue, we propose a decentralized trajectory planning framework for formation flight in unknown and dense environments. To ensure that feasible trajectories can be found within the limited time, the formation optimization problem is decoupled into formation affine transformation and iterative trajectory generation. Firstly, the optimization problem based on affine transformation is designed to obtain the optimal affine transformation sequence, which provides the formation reference of trajectory optimization. Secondly, the iterative optimization framework of trajectory planning is designed, which balances the conflicting requirements of formation, smooth flight, and obstacle avoidance. Besides, to escape the local minima caused by non-convex dense environments, the method of topological path planning is designed to provide distinctive initial solutions for trajectory optimization. Finally, the proposed methods are proven to be effective through the simulations and real-world experiments.

关键词： Trajectory planning Planning Iterative methods Collision avoidance Trajectory optimization Intelligent robots

来源：评论

学校读者我要写书评

暂无评论

Representing Domain-Mixing Optical Degradation for Real-World Computational Aberration Correction via Vector Quantization

arXiv

引用

arXiv 2024年

作者： Jiang, Qi Yi, Zhonghua Gao, Shaohua Gao, Yao Qian, Xiaolong Shi, Hao Sun, Lei Niu, JinXing Wang, Kaiwei Yang, Kailun Bai, Jian State Key Laboratory of Extreme Photonics and Instrumentation College of Optical Science and Engineering Zhejiang University Hangzhou310027 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Intelligent Optics & Photonics Research Center Jiaxing Research Institute Zhejiang University Jiaxing314031 China School of Mechanical Engineering North China University of Water Resources and Electric Power Zhenzhou450045 China

Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervised Domain Adaptation (UDA). By incorporating readily accessible unpaired real-world data into training, we formalize the Domain Adaptive CAC (DACAC) task, and then introduce a comprehensive Real-world aberrated images (Realab) dataset to benchmark it. The setup task presents a formidable challenge due to the intricacy of understanding the target optical degradation domain. To this intent, we propose a novel Quantized Domain-Mixing Representation (QDMR) framework as a potent solution to the issue. centering around representing and quantizing the optical degradation which is consistent across different images, QDMR adapts the CAC model to the target domain from three key aspects: (1) reconstructing aberrated images of both domains by a VQGAN to learn a Domain-Mixing Codebook (DMC) characterizing the optical degradation;(2) modulating the deep features in CAC model with DMC to transfer the target domain knowledge;and (3) leveraging the trained VQGAN to generate pseudo target aberrated images from the source ones for convincing target domain supervision. Extensive experiments on both synthetic and real-world benchmarks reveal that the models with QDMR consistently surpass the competitive methods in mitigating the synthetic-to-real gap, which produces visually pleasant real-world CAC results with fewer artifacts. Codes and datasets are made publicly available at https://***/zju-jiangqi/QDMR. Copyright © 2024, The Authors. All rights reserved.

关键词： Vector quantization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：