检索结果-内蒙古大学图书馆

arXiv 2023年

作者： Yin, Xiaoting Shi, Hao Chen, Jiaan Wang, Ze Ye, Yaozu Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd. Shanghai201210 China

Human pose estimation is a fundamental and appealing task in computer vision. Although traditional cameras are commonly applied, their reliability decreases in scenarios under high dynamic range or heavy motion blur, where event cameras offer a robust solution. Predominant event-based methods accumulate events into frames, ignoring the asynchronous and high temporal resolution that is crucial for distinguishing distinct actions. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC aggregates events within concise temporal slices at identical positions, preserving their 3D attributes along with statistical information, thereby significantly reducing memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. Our methods are tested on the DHP19 public dataset, MMHPSD dataset, and our EV-3DPW dataset, with further qualitative validation via a derived driving scene dataset EV–JAAD and an outdoor collection vehicle. Our code and dataset have been made publicly available at https://***/MasterHow/EventPointPose. Copyright © 2023, The Authors. All rights reserved.

关键词： 3D modeling

来源：评论

学校读者我要写书评

暂无评论

E-3DGS: Gaussian Splatting with Exposure and Motion Events

arXiv

引用

arXiv 2024年

作者： Yin, Xiaoting Shi, Hao Bao, Yuhan Bing, Zhenshan Liao, Yiyi Yang, Kailun Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University China School of Robotics National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China College of Information Science and Electronic Engineering Zhejiang University China The Chair of Robotics AI and Real-Time Systems Technical University of Munich Germany

Estimating Neural Radiance Fields (NeRFs) from images captured under optimal conditions has been extensively explored in the vision community. However, robotic applications often face challenges such as motion blur, insufficient illumination, and high computational overhead, which adversely affect downstream tasks like navigation, inspection, and scene visualization. To address these challenges, we propose E-3DGS, a novel event-based approach that partitions events into motion (from camera or object movement) and exposure (from camera exposure), using the former to handle fast-motion scenes and using the latter to reconstruct grayscale images for high-quality training and optimization of event-based 3D Gaussian Splatting (3DGS). We introduce a novel integration of 3DGS with exposure events for high-quality reconstruction of explicit scene representations. Our versatile framework can operate on motion events alone for 3D reconstruction, enhance quality using exposure events, or adopt a hybrid mode that balances quality and effectiveness by optimizing with initial exposure events followed by high-speed motion events. We also introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds. Our method is faster and delivers better reconstruction quality than event-based NeRF while being more cost-effective than NeRF methods that combine event and RGB data by using a single event sensor. By combining motion and exposure events, E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions and lower hardware demands. The source code and dataset will be available at https://***/MasterHow/E-3DGS. Copyright © 2024, The Authors. All rights reserved.

关键词： Gaussian distribution

来源：评论

学校读者我要写书评

暂无评论

Computational Imaging for Machine perception: Transferring Semantic Segmentation beyond Aberrations

arXiv

引用

arXiv 2022年

作者： Jiang, Qi Shi, Hao Gao, Shaohua Zhang, Jiaming Yang, Kailun Sun, Lei Ni, Huajian Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd Shanghai201210 China

Semantic scene understanding with Minimalist Optical Systems (MOS) in mobile and wearable applications remains a challenge due to the corrupted imaging quality induced by optical aberrations. However, previous works only focus on improving the subjective imaging quality through the Computational Imaging (CI) technique, ignoring the feasibility of advancing semantic segmentation. In this paper, we pioneer the investigation of Semantic Segmentation under Optical Aberrations (SSOA) with MOS. To benchmark SSOA, we construct Virtual Prototype Lens (VPL) groups through optical simulation, generating Cityscapes-ab and KITTI-360-ab datasets under different behaviors and levels of aberrations. We look into SSOA via an unsupervised domain adaptation perspective to address the scarcity of labeled aberration data in real-world scenarios. Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA. Based on our benchmark, we conduct experiments on the robustness of classical segmenters against aberrations. In addition, extensive evaluations of possible solutions to SSOA reveal that CIADA achieves superior performance under all aberration distributions, bridging the gap between computational imaging and downstream applications for MOS. The project page is at https://***/zju-jiangqi/CIADA. Copyright © 2022, The Authors. All rights reserved.

关键词： Aberrations

来源：评论

学校读者我要写书评

暂无评论

Large Scale Unsupervised Brain MRI Image Registration Solution for Learn2Reg 2024

arXiv

引用

arXiv 2024年

作者： Zhang, Yuxi Chen, Xiang Wang, Jiazheng Liu, Min Wang, Yaonan Liu, Dongdong Hu, Renjiu Zhang, Hang College of Electrical and Information Engineering Hunan University Hunan Changsha China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Hunan Changsha China New York University New York United States Cornell University New York United States

In this paper, we summarize the methods and experimental results we proposed for Task 2 in the learn2reg 2024 Challenge. This task focuses on unsupervised registration of anatomical structures in brain MRI images between different patients. The difficulty lies in: (1) without segmentation labels, and (2) a large amount of data. To address these challenges, we built an efficient backbone network and explored several schemes to further enhance registration accuracy. Under the guidance of the NCC loss function and smoothness regularization loss function, we obtained a smooth and reasonable deformation field. According to the leaderboard, our method achieved a Dice coefficient of 77.34%, which is 1.4% higher than the TransMorph. Overall, we won second place on the leaderboard for Task 2. Copyright © 2024, The Authors. All rights reserved.

关键词： Image registration

来源：评论

学校读者我要写书评

暂无评论

MATErobot: Material Recognition in Wearable robotics for People with visual Impairments

arXiv

引用

arXiv 2023年

作者： Zheng, Junwei Zhang, Jiaming Yang, Kailun Peng, Kunyu Stiefelhagen, Rainer The Institute for Robotics and Anthropomatics Karlsruhe Institute of Technology Karlsruhe76131 Germany The Department of Engineering Science University of Oxford OxfordOX1 3PJ United Kingdom The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

People with visual Impairments (PVI) typically recognize objects through haptic perception. Knowing objects and materials before touching is desired by the target users but under-explored in the field of human-centered robotics. To fill this gap, in this work, a wearable vision-based robotic system, MATErobot, is established for PVI to recognize materials and object categories beforehand. To address the computational constraints of mobile platforms, we propose a lightweight yet accurate model MATEViT to perform pixel-wise semantic segmentation, simultaneously recognizing both objects and materials. Our methods achieve respective 40.2% and 51.1% of mIoU on COCOStuff-10K and DMS datasets, surpassing the previous method with +5.7% and +7.0% gains. Moreover, on the field test with participants, our wearable system reaches a score of 28 in the NASA-Task Load Index, indicating low cognitive demands and ease of use. Our MATErobot demonstrates the feasibility of recognizing material property through visual cues and offers a promising step towards improving the functionality of wearable robots for PVI. The source code has been made publicly available at MATErobot. Copyright © 2023, The Authors. All rights reserved.

关键词： NASA

来源：评论

学校读者我要写书评

暂无评论

EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving

arXiv

引用

arXiv 2024年

作者： Lin, Jiacheng Chen, Jiajun Peng, Kunyu He, Xuan Li, Zhiyong Stiefelhagen, Rainer Yang, Kailun The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University Changsha410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The Institute for Robotics and An-Thropomatics Karlsruhe Institute of Technology Karlsruhe76131 Germany

This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving. Due to the lack of semantic modeling capacity in audio and video, existing works have mainly focused on text-based multiobject tracking, which often comes at the cost of tracking quality, interaction efficiency, and even the safety of assistance systems, limiting the application of such methods in autonomous driving. In this paper, we delve into the problem of AR-MOT from the perspective of audio-video fusion and audio-video tracking. We put forward EchoTrack, an end-to-end AR-MOT framework with dual-stream vision transformers. The dual streams are intertwined with our Bidirectional Frequency-domain Cross-attention Fusion Module (Bi-FCFM), which bidirectionally fuses audio and video features from both frequency- and spatiotemporal domains. Moreover, we propose the Audio-visual Contrastive Tracking Learning (ACTL) regime to extract homogeneous semantic features between expressions and visual objects by learning homogeneous features between different audio and video objects effectively. Aside from the architectural design, we establish the first set of large-scale AR-MOT benchmarks, including Echo-KITTI, Echo-KITTI+, and Echo-BDD. Extensive experiments on the established benchmarks demonstrate the effectiveness of the proposed EchoTrack and its components. The source code and datasets are available at https://***/lab206/EchoTrack. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection

arXiv

引用

arXiv 2023年

作者： He, Xuan Yang, Fan Yang, Kailun Lin, Jiacheng Fu, Haolong Wang, Meng Yuan, Jin Li, Zhiyong The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University China410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computer Science Hefei University of Technology Hefei230009 China

Transformer-based methods have demonstrated superior performance for monocular 3D object detection recently, which aims at predicting 3D attributes from a single 2D image. Most existing transformer-based methods leverage both visual and depth representations to explore valuable query points on objects, and the quality of the learned query points has a great impact on detection accuracy. Unfortunately, existing unsupervised attention mechanisms in transformers are prone to generate low-quality query features due to inaccurate receptive fields, especially on hard objects. To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection. Specifically, SSDA presets several masks with different scales and utilizes depth and visual features to adaptively learn a scale-aware filter for object query augmentation. Imposing the scale awareness, SSDA could well predict the accurate receptive field of an object query to support robust query feature generation. Aside from this, SSDA is assigned with a Weighted Scale Matching (WSM) loss to supervise scale prediction, which presents more confident results as compared to the unsupervised attention mechanisms. Extensive experiments on the KITTI and Waymo Open datasets demonstrate that SSDA significantly improves the detection accuracy, especially on moderate and hard objects, yielding state-of-the-art performance as compared to the existing approaches. Our code will be made publicly available at https://***/mikasa3lili/SSD-MonoDETR. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Towards Anytime Optical Flow Estimation with Event Cameras

arXiv

引用

arXiv 2023年

作者： Ye, Yaozu Shi, Hao Yang, Kailun Wang, Ze Yin, Xiaoting Lin, Yining Liu, Mao Wang, Yaonan Wang, Kaiwei The State Key Laboratory of Modern Optical Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Co. Ltd Shanghai201210 China Dongguan Yutong Optical Technology Co. Ltd Dongguan523866 China

Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at https://***/Yaozhuwa/EVA-Flow. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

来源：评论

学校读者我要写书评

暂无评论

High-Speed Trajectory Tracking control for Quadrotors via Deep Reinforcement Learning

High-Speed Trajectory Tracking Control for Quadrotors via De...

引用

Youth Academic Annual Conference of Chinese Association of Automation (YAC)

作者： Hean Hua Hang Zhong Hui Zhang Yongchun Fang Yaonan Wang College of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Robotics Hunan University Changsha China Institute of Robotics and Automatic Information System College of Artificial Intelligence Nankai University Tianjin Key Laboratory of Intelligent Robotics Nankai University Tianjin China

ISBN: (数字)9798350379228

ISBN: (纸本)9798350390780

This paper presents a learning-based high-speed trajectory tracking control strategy for quadrotors, which achieves efficient learning and strong reliability by the collaboration of deep reinforcement learning (RL) and self-tuning mechanism. Different from existing methods, the proposed strategy is designed to explore optimal control performance by taking advantage of model-based self-tuning mechanism and deep reinforcement learning. Specifically, the self-tuning guided deep RL scheme is put forward for quadrotors, with superior learning efficiency and strong adaptability. Firstly, a novel self-tuning mechanism is constructed and some auxiliary variables are introduced to enhance the tracking performance. Then, based on the model-driven self-tuning design, the deep RL is proposed to achieve model-guided learning, where the tuning actions are adopted in the evaluation process during training, aiming at removing the bad explorations by the carefully designed parallel evaluation. Finally, the convergence is analyzed based on the proposed learning framework, which indicates the efficient cooperation of exploration and self-tuning mechanism. To verify the effectiveness of the proposed controller, the guided training and hardware experiments are implemented to show efficient cooperation and satisfactory high-speed trajectory tracking control of the proposed method.

关键词： Training Adaptation models Trajectory tracking Heuristic algorithms Optimal control Deep reinforcement learning Hardware

来源：评论

学校读者我要写书评

暂无评论

Decentralized Trajectory Planning for Formation Flight in Unknown and Dense Environments

Decentralized Trajectory Planning for Formation Flight in Un...

引用

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Jianxin Zeng Yaonan Wang Zhiqiang Miao Wei He Hesheng Wang Department of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Changsha China School of Intelligence Science and Technology University of Science and Technology Beiing Beijing China Key Laboratory of Intelligent Bionic Unmanned Systems Ministry of Education University of Science and Technology Beijing Beijing China Department of Automation Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

For aerial swarms, formation flight has been applied in various scenes. However, most existing works do not consider balancing the conflicting requirements among keeping formation, keeping the smoothness of trajectories, and obstacle avoidance within the limited time. To address this issue, we propose a decentralized trajectory planning framework for formation flight in unknown and dense environments. To ensure that feasible trajectories can be found within the limited time, the formation optimization problem is decoupled into formation affine transformation and iterative trajectory generation. Firstly, the optimization problem based on affine transformation is designed to obtain the optimal affine transformation sequence, which provides the formation reference of trajectory optimization. Secondly, the iterative optimization framework of trajectory planning is designed, which balances the conflicting requirements of formation, smooth flight, and obstacle avoidance. Besides, to escape the local minima caused by non-convex dense environments, the method of topological path planning is designed to provide distinctive initial solutions for trajectory optimization. Finally, the proposed methods are proven to be effective through the simulations and real-world experiments.

关键词： Trajectory planning Planning Iterative methods Collision avoidance Trajectory optimization Intelligent robots

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：