检索结果-内蒙古大学图书馆

EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Jiacheng Chen, Jiajun Peng, Kunyu He, Xuan Li, Zhiyong Stiefelhagen, Rainer Yang, Kailun The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University Changsha410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The Institute for Robotics and An-Thropomatics Karlsruhe Institute of Technology Karlsruhe76131 Germany

This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving. Due to the lack of semantic modeling capacity in audio and video, existing works have mainly focused on text-based multiobject tracking, which often comes at the cost of tracking quality, interaction efficiency, and even the safety of assistance systems, limiting the application of such methods in autonomous driving. In this paper, we delve into the problem of AR-MOT from the perspective of audio-video fusion and audio-video tracking. We put forward EchoTrack, an end-to-end AR-MOT framework with dual-stream vision transformers. The dual streams are intertwined with our Bidirectional Frequency-domain Cross-attention Fusion Module (Bi-FCFM), which bidirectionally fuses audio and video features from both frequency- and spatiotemporal domains. Moreover, we propose the Audio-visual Contrastive Tracking Learning (ACTL) regime to extract homogeneous semantic features between expressions and visual objects by learning homogeneous features between different audio and video objects effectively. Aside from the architectural design, we establish the first set of large-scale AR-MOT benchmarks, including Echo-KITTI, Echo-KITTI+, and Echo-BDD. Extensive experiments on the established benchmarks demonstrate the effectiveness of the proposed EchoTrack and its components. The source code and datasets are available at https://***/lab206/EchoTrack. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： He, Xuan Yang, Fan Yang, Kailun Lin, Jiacheng Fu, Haolong Wang, Meng Yuan, Jin Li, Zhiyong The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University China410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computer Science Hefei University of Technology Hefei230009 China

Transformer-based methods have demonstrated superior performance for monocular 3D object detection recently, which aims at predicting 3D attributes from a single 2D image. Most existing transformer-based methods leverage both visual and depth representations to explore valuable query points on objects, and the quality of the learned query points has a great impact on detection accuracy. Unfortunately, existing unsupervised attention mechanisms in transformers are prone to generate low-quality query features due to inaccurate receptive fields, especially on hard objects. To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection. Specifically, SSDA presets several masks with different scales and utilizes depth and visual features to adaptively learn a scale-aware filter for object query augmentation. Imposing the scale awareness, SSDA could well predict the accurate receptive field of an object query to support robust query feature generation. Aside from this, SSDA is assigned with a Weighted Scale Matching (WSM) loss to supervise scale prediction, which presents more confident results as compared to the unsupervised attention mechanisms. Extensive experiments on the KITTI and Waymo Open datasets demonstrate that SSDA significantly improves the detection accuracy, especially on moderate and hard objects, yielding state-of-the-art performance as compared to the existing approaches. Our code will be made publicly available at https://***/mikasa3lili/SSD-MonoDETR. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

Towards Anytime Optical Flow Estimation with Event Cameras

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ye, Yaozu Shi, Hao Yang, Kailun Wang, Ze Yin, Xiaoting Lin, Yining Liu, Mao Wang, Yaonan Wang, Kaiwei The State Key Laboratory of Modern Optical Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Co. Ltd Shanghai201210 China Dongguan Yutong Optical Technology Co. Ltd Dongguan523866 China

Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at https://***/Yaozhuwa/EVA-Flow. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

High-Speed Trajectory Tracking control for Quadrotors via Deep Reinforcement Learning

学校读者我要写书评

暂无评论

High-Speed Trajectory Tracking Control for Quadrotors via De...

Youth Academic Annual Conference of Chinese Association of Automation (YAC)

作者： Hean Hua Hang Zhong Hui Zhang Yongchun Fang Yaonan Wang College of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Robotics Hunan University Changsha China Institute of Robotics and Automatic Information System College of Artificial Intelligence Nankai University Tianjin Key Laboratory of Intelligent Robotics Nankai University Tianjin China

ISBN: (数字)9798350379228

ISBN: (纸本)9798350390780

This paper presents a learning-based high-speed trajectory tracking control strategy for quadrotors, which achieves efficient learning and strong reliability by the collaboration of deep reinforcement learning (RL) and self-tuning mechanism. Different from existing methods, the proposed strategy is designed to explore optimal control performance by taking advantage of model-based self-tuning mechanism and deep reinforcement learning. Specifically, the self-tuning guided deep RL scheme is put forward for quadrotors, with superior learning efficiency and strong adaptability. Firstly, a novel self-tuning mechanism is constructed and some auxiliary variables are introduced to enhance the tracking performance. Then, based on the model-driven self-tuning design, the deep RL is proposed to achieve model-guided learning, where the tuning actions are adopted in the evaluation process during training, aiming at removing the bad explorations by the carefully designed parallel evaluation. Finally, the convergence is analyzed based on the proposed learning framework, which indicates the efficient cooperation of exploration and self-tuning mechanism. To verify the effectiveness of the proposed controller, the guided training and hardware experiments are implemented to show efficient cooperation and satisfactory high-speed trajectory tracking control of the proposed method.

关键词： Training Adaptation models Trajectory tracking Heuristic algorithms Optimal control Deep reinforcement learning Hardware

Decentralized Trajectory Planning for Formation Flight in Unknown and Dense Environments

学校读者我要写书评

暂无评论

Decentralized Trajectory Planning for Formation Flight in Un...

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Jianxin Zeng Yaonan Wang Zhiqiang Miao Wei He Hesheng Wang Department of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Changsha China School of Intelligence Science and Technology University of Science and Technology Beiing Beijing China Key Laboratory of Intelligent Bionic Unmanned Systems Ministry of Education University of Science and Technology Beijing Beijing China Department of Automation Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

For aerial swarms, formation flight has been applied in various scenes. However, most existing works do not consider balancing the conflicting requirements among keeping formation, keeping the smoothness of trajectories, and obstacle avoidance within the limited time. To address this issue, we propose a decentralized trajectory planning framework for formation flight in unknown and dense environments. To ensure that feasible trajectories can be found within the limited time, the formation optimization problem is decoupled into formation affine transformation and iterative trajectory generation. Firstly, the optimization problem based on affine transformation is designed to obtain the optimal affine transformation sequence, which provides the formation reference of trajectory optimization. Secondly, the iterative optimization framework of trajectory planning is designed, which balances the conflicting requirements of formation, smooth flight, and obstacle avoidance. Besides, to escape the local minima caused by non-convex dense environments, the method of topological path planning is designed to provide distinctive initial solutions for trajectory optimization. Finally, the proposed methods are proven to be effective through the simulations and real-world experiments.

关键词： Trajectory planning Planning Iterative methods Collision avoidance Trajectory optimization Intelligent robots

Hierarchical Task Impedance control of a Serial Manipulator for Minimally Invasive Surgery

学校读者我要写书评

暂无评论

Hierarchical Task Impedance Control of a Serial Manipulator ...

IEEE International Conference on Human-Machine Systems

作者： Hang Su Chenguang Yang Jiehao Li Yiming Jiang Giancarlo Ferrigno Elena De Momi Dipartimento di Elettronica Informazione e Bioingegneria Politecnico di Milano Milan Italy Bristol Robotics Laboratory University of the West of England Bristol United Kingdom State Key Laboratory of Intelligent Control and Decision of Complex Systems Beijing Institute of Technology Beijing China National Engineering Laboratory for Robot Visual Perception and Control Hunan University Changsha China Dipartimento di Elettronica Informazione e Bioingegneria Politecnico di Milano Milano Italy

ISBN: (数字)9781728158716

ISBN: (纸本)9781728158723

Flexibility and robustness have become key points in the development of surgical robot controller for physical interactions. However, the conventional impedance control schemes unaware of the actual surgical scenario, including complex physical interaction on the robot arm, lead to the loss of accuracy. In this paper, a hierarchical task impedance control scheme is proposed for Minimally Invasive Surgery (MIS) based on an operational space formulation of a 7 DoFs redundant robot. Its redundancy is exploited to guarantee a remote center of motion (RCM) constraint and to provide a flexible workspace for the medical staff to assist physicians. In addition to the achievement of the classical whole-body impedance control, the issue of uncertain disturbances will be addressed by a decoupled adaptive approximation based on a radial basis function neural network (RBFNN) within the control framework. Task performances under the hierarchical task impedance controller were validated and compared with previous work in the literature. Experimental results showed its improved performance in terms of positional error and RCM constraint, regardless of the existing uncertain physical interaction.

关键词： Task analysis Impedance Manipulators Tools Surgery Jacobian matrices

Towards to robust and generalized medical image segmentation framework

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Yurong Zhang, Hui Wang, Yaonan Liu, Lizhu Wu, Q.M. Jonathan Yang, Yimin National Engineering Laboratory of Robot Visual Perception and Control Technology School of Robotics Hunan University Hunan Changsha410082 China Department of Electrical and Computer Engineering University of Windsor WindsorONN9B3P4 Canada Department of Computer Science Lakehead University Vector Institute TorontoM5G 1M1 Canada

—To mitigate the radiologist’s workload, computer-aided diagnosis with the capability to review and analyze medical images is gradually deployed. Deep learning-based region of interest segmentation is among the most exciting use cases. However, this paradigm is restricted in real-world clinical applications due to the poor robustness and generalization. The issue is more sinister with a lack of training data. In this paper, we address the challenge from the representation learning point of view. We investigate that the collapsed representations, as one of the main reasons which caused poor robustness and generalization, could be avoided through transfer learning. Therefore, we propose a novel two-stage framework for robust generalized segmentation. In particular, an unsupervised Tile-wise AutoEncoder (T-AE) pretraining architecture is coined to learn meaningful representation. Furthermore, this learned prior knowledge is transferred to the segmentation benchmark. Coupled with an image reconstruction network, the representation keeps to be decoded, encouraging the model to capture more semantic features. Experiments of lung segmentation on multi chest X-ray datasets are conducted. Empirically, the comprehensive results demonstrate the superior generalization capability of the proposed framework on unseen domains in terms of high performance and robustness to corruption, especially under the scenario of the limited training data. © 2021, CC BY.

关键词： Computer aided diagnosis

Decentralized Multi-robot Navigation Coupled with Spatial-Temporal RetNet Based on Deep Reinforcement Learning

学校读者我要写书评

暂无评论

Decentralized Multi-Robot Navigation Coupled with Spatial-Te...

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Lin Chen Yaonan Wang Zhiqiang Miao Mingtao Feng Yuanzhe Wang Yang Mo Zhen Zhou Hesheng Wang Danwei Wang School of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Artificial Intelligence Xidian University Xian China School of Electrical and Electrical Engineering Nanyang Technological University Nanyang Avenue Singapore Department of Automation Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Navigating robots through dynamic multi-robot environments, avoiding collisions with both other robots and obstacles, has emerged as a central challenge in robotics. The existing approaches fall short in allowing the policy network to effectively capture spatial-temporal reciprocal collision avoidance in multi-robot environments, comprising both static and dynamic obstacles, resulting in inadequate safety and efficiency in directing robot movement. In this study, we introduce a novel policy neural network called Spatial-Temporal RetNet (STR), designed to encode reciprocal collision avoidance states between robots in spatial and temporal dimensions. The goal is to improve the safety and efficacy of the policy neural network in directing robots to complete assigned tasks. The spatial state encoder module is built upon a parallel RetNet structure, which strengthens the neural network's capacity in extracting reciprocal collision avoidance states between robots in spatial dimensions. This module addresses the limitations of position encoding in transformer-based multi-robot navigation policy neural networks. We design a temporal state encoder utilizing a recurrent RetNet structure. This innovation bolsters the multi-robot navigation policy neural network's capability to capture features in the temporal dimension of multi-robot movements. It addresses the limitations of transformer-based multi-robot navigation policy neural networks, particularly in recurrently inferring information across time dimensions. Simulation experiments were conducted to showcase the superior safety and effectiveness of our proposed method compared to previous state-of-the-art approaches in guiding robots to accomplish tasks.

关键词： Training Technological innovation Navigation Neural networks Transformers Feature extraction Encoding Safety Collision avoidance robots

Fast ORB-SLAM without Keypoint Descriptors

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Fu, Qiang Yu, Hongshan Wang, Xiaolong Yang, Zhengeng He, Yong Zhang, Hong Mian, Ajmal The National Engineering Laboratory for Robot Visual Perception and Control Technology College of Electrical and Information Engineering Hunan University China The Department of Computing Science University of Alberta Canada Quanzhou Institute of Industrial Design and Machine Intelligence Innovation Hunan University China The Southern University of Science and Technology Shenzhen China The School of Mathematics and Information Science Shaanxi Normal University China The Department of Computer Science The University of Western Australia WA6009 Australia

Indirect methods for visual SLAM are gaining popularity due to their robustness to environmental variations. ORB-SLAM2 [1] is a benchmark method in this domain, however, it consumes significant time for computing descriptors that never get reused unless a frame is selected as a keyframe. To overcome these problems, we present FastORB-SLAM which is light-weight and efficient as it tracks keypoints between adjacent frames without computing descriptors. To achieve this, a two stage descriptor-independent keypoint matching method is proposed based on sparse optical flow. In the first stage, we predict initial keypoint correspondences via a simple but effective motion model and then robustly establish the correspondences via pyramid-based sparse optical flow tracking. In the second stage, we leverage the constraints of the motion smoothness and epipolar geometry to refine the correspondences. In particular, our method computes descriptors only for keyframes. We test FastORB-SLAM on TUM and ICL-NUIM RGB-D datasets and compare its accuracy and efficiency to nine existing RGB-D SLAM methods. Qualitative and quantitative results show that our method achieves state-of-the-art accuracy and is about twice as fast as the ORB-SLAM2. Copyright © 2020, The Authors. All rights reserved.

关键词： Optical flows