检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Lin, Jiacheng Chen, Jiajun Peng, Kunyu He, Xuan Li, Zhiyong Stiefelhagen, Rainer Yang, Kailun The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University Changsha410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The Institute for Robotics and An-Thropomatics Karlsruhe Institute of Technology Karlsruhe76131 Germany

This paper introduces the task of Auditory Referring Multi-Object Tracking (AR-MOT), which dynamically tracks specific objects in a video sequence based on audio expressions and appears as a challenging problem in autonomous driving. Due to the lack of semantic modeling capacity in audio and video, existing works have mainly focused on text-based multiobject tracking, which often comes at the cost of tracking quality, interaction efficiency, and even the safety of assistance systems, limiting the application of such methods in autonomous driving. In this paper, we delve into the problem of AR-MOT from the perspective of audio-video fusion and audio-video tracking. We put forward EchoTrack, an end-to-end AR-MOT framework with dual-stream vision transformers. The dual streams are intertwined with our Bidirectional Frequency-domain Cross-attention Fusion Module (Bi-FCFM), which bidirectionally fuses audio and video features from both frequency- and spatiotemporal domains. Moreover, we propose the Audio-visual Contrastive Tracking Learning (ACTL) regime to extract homogeneous semantic features between expressions and visual objects by learning homogeneous features between different audio and video objects effectively. Aside from the architectural design, we establish the first set of large-scale AR-MOT benchmarks, including Echo-KITTI, Echo-KITTI+, and Echo-BDD. Extensive experiments on the established benchmarks demonstrate the effectiveness of the proposed EchoTrack and its components. The source code and datasets are available at https://***/lab206/EchoTrack. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection

arXiv

引用

arXiv 2023年

作者： He, Xuan Yang, Fan Yang, Kailun Lin, Jiacheng Fu, Haolong Wang, Meng Yuan, Jin Li, Zhiyong The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The School of Robotics Hunan University China410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computer Science Hefei University of Technology Hefei230009 China

Transformer-based methods have demonstrated superior performance for monocular 3D object detection recently, which aims at predicting 3D attributes from a single 2D image. Most existing transformer-based methods leverage both visual and depth representations to explore valuable query points on objects, and the quality of the learned query points has a great impact on detection accuracy. Unfortunately, existing unsupervised attention mechanisms in transformers are prone to generate low-quality query features due to inaccurate receptive fields, especially on hard objects. To tackle this problem, this paper proposes a novel "Supervised Scale-aware Deformable Attention" (SSDA) for monocular 3D object detection. Specifically, SSDA presets several masks with different scales and utilizes depth and visual features to adaptively learn a scale-aware filter for object query augmentation. Imposing the scale awareness, SSDA could well predict the accurate receptive field of an object query to support robust query feature generation. Aside from this, SSDA is assigned with a Weighted Scale Matching (WSM) loss to supervise scale prediction, which presents more confident results as compared to the unsupervised attention mechanisms. Extensive experiments on the KITTI and Waymo Open datasets demonstrate that SSDA significantly improves the detection accuracy, especially on moderate and hard objects, yielding state-of-the-art performance as compared to the existing approaches. Our code will be made publicly available at https://***/mikasa3lili/SSD-MonoDETR. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents

arXiv

引用

arXiv 2022年

作者： Wang, Ze Yang, Kailun Shi, Hao Li, Peng Gao, Fei Bai, Jian Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China Huzhou Institute of Zhejiang University Zhejiang University China

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using (u, v, 1)T become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a visual Inertial SLAM framework for cameras with extremely Large FoV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method Copyright © 2022, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

LF-PGVIO: A visual-Inertial-Odometry Framework for Large Field-of-View Cameras using Points and Geodesic Segments

arXiv

引用

arXiv 2023年

作者： Wang, Ze Yang, Kailun Shi, Hao Zhang, Yufan Xu, Zhijie Gao, Fei Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The School of Computing and Engineering University of Huddersfield HuddersfieldHD1 3DH United Kingdom The State Key Laboratory of Industrial Control Technology Zhejiang University Hangzhou310027 China

In this paper, we propose LF-PGVIO, a visual-Inertial-Odometry (VIO) framework for large Field-of-View (FoV) cameras with a negative plane using points and geodesic segments. The purpose of our research is to unleash the potential of point-line odometry with large-FoV omnidirectional cameras, even for cameras with negative-plane FoV. To achieve this, we propose an Omnidirectional Curve Segment Detection (OCSD) method combined with a camera model which is applicable to images with large distortions, such as panoramic annular images, fisheye images, and various panoramic images. The geodesic segment is sliced into multiple straight-line segments based on the radian and descriptors are extracted and recombined. Descriptor matching establishes the constraint relationship between 3D line segments in multiple frames. In our VIO system, line feature residual is also extended to support large-FoV cameras. Extensive evaluations on public datasets demonstrate the superior accuracy and robustness of LF-PGVIO compared to state-of-the-art methods. The source code will be made publicly available at https://***/flysoaryun/LF-PGVIO. Copyright © 2023, The Authors. All rights reserved.

关键词： Cameras

来源：评论

学校读者我要写书评

暂无评论

Towards Anytime Optical Flow Estimation with Event Cameras

arXiv

引用

arXiv 2023年

作者： Ye, Yaozu Shi, Hao Yang, Kailun Wang, Ze Yin, Xiaoting Lin, Yining Liu, Mao Wang, Yaonan Wang, Kaiwei The State Key Laboratory of Modern Optical Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Co. Ltd Shanghai201210 China Dongguan Yutong Optical Technology Co. Ltd Dongguan523866 China

Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at https://***/Yaozhuwa/EVA-Flow. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical flows

来源：评论

学校读者我要写书评

暂无评论

Binocular Vision System for Measuring the Height of control Rod Drive Rod in Nuclear Reactor

Binocular Vision System for Measuring the Height of Control ...

引用

Chinese control Conference (CCC)

作者： Xianen Zhou Yingjian Wen Qing Zhu Yaonan Wang Feixiang Sun Shaonan Chen National Engineering Research Center for Robot Vision Perception and Control Technology Hunan University Jiangxi Communication Terminal Industry Technology Research Institute Co. Ltd. University of Electronic Science and Technology of China China Nuclear Power Technology Research Institute Co. Ltd.

Nuclear energy is widely recognized as an important source to meet the increasing energy demand in the future. However, since the occurrence of the Fukushima accident, its potential safety hazards have been strongly concerned by the public, and has become one of the most significant problems to be solved at this stage. In this paper, a vision inspection system with support detection scheme based on RGB-D fusion is proposed to achieve accurate detection of the control rod position for the potential danger in the decoupling process of the control rod drive mechanism. In order to separate the depth region containing the relevant information of the driving rod head, the ellipse region of the top ring of the driving rod is detected from the color image by the method based on line segment extraction and region of interest detection. Due to the smoothness and reflection of the surface of the control rod, part of the depth of the head of the control rod is lost. In order to predict the reference distance between the top value of the control rod and the camera, a center point filling method based on multi-direction linear regression in gradient stage is designed. In order to evaluate the effectiveness of the proposed method, a series of experiments are carried out on the head location and depth calculation of the drive rod. The results show that the vision inspection system in this paper can achieve high-precision positioning of the control rod.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Novel Pose Estimation Method of Object in robotic Manipulation Using Vision-Based Tactile Sensor 5th

A Novel Pose Estimation Method of Object in Robotic Manipula...

引用

5th International Conference on Cognitive Systems and Signal Processing, ICCSIP 2020

作者： Zhao, Dan Sun, Fuchun Zhou, Quan Wang, Zongtao Beijing National Research Center for Information Science and Technology Department of Computer Science and Technology Tsinghua University Beijing100083 China AnHui Province Key Laboratory of Special Heavy Load Robot Anhui University of Technology Ma Anshan China Key Lab of Industrial Computer Control Engineering of Hebei Province Yanshan University Qinhuangdao066000 China

ISBN: (纸本)9789811623356

The computer vision techniques have been widely used in the robotic manipulation for perception and positioning. However, duo to the inaccuracy of camera calibration and measurement, there is a greatly need for more accurate method to estimate the object pose especially for robotic precise manipulation. In this paper, we propose a novel pose estimation method of object in robotic manipulation using a vision-based tactile sensor. The structure of the tactile sensor and the proposed model are introduced in detail, and verification experiments have been performed. The results show the advantages and better performance of the proposed method. © 2021, Springer Nature Singapore Pte Ltd.

关键词： robot vision

来源：评论

学校读者我要写书评

暂无评论

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

arXiv

引用

arXiv 2023年

作者： Yi, Zhonghua Shi, Hao Yang, Kailun Jiang, Qi Ye, Yaozu Wang, Ze Ni, Huajian Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics Hunan University Changsha410012 China The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd. Shanghai201210 China

Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point control Loss (CPCL) function for diverse point-wise supervision;2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5% precision improvement on various key points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at https://***/ZhonghuaYi/FocusFlow_official. Copyright © 2023, The Authors. All rights reserved.

关键词： Autonomous vehicles

来源：评论

学校读者我要写书评

暂无评论

Online Parameter Estimation For Uncertain robot Manipulators With Fixed-time Convergence

Online Parameter Estimation For Uncertain Robot Manipulators...

引用

IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Chengzhi Zhu Yiming Jiang Chenguang Yang Key Laboratory of Autonomous Systems and Networked Control College of Automation Science and Engineering South China University of Technology Guangzhou China National Engineering Laboratory for Robot Visual Perception and Control Hunan University Changsha China Bristol Robotics Laboratory University of the West of England Bristol UK

ISBN: (数字)9781728151694

ISBN: (纸本)9781728151700

For traditional parameter estimation schemes of uncertain robot, most of them were proposed to identify unknown parameter with desired precision, but few of them focused on the convergence time. Recently finite-time estimation techniques have been proposed by scholars to achieve estimation in finite time. In this paper, we proposed a novel estimation scheme for uncertain robot systems with fixed time instead of finite time. In order to avoid using acceleration signals during the estimation, a kind of auxiliary filtering technique was employed. Besides, a continuous and recursive update law was employed for the parameter estimation such that the computational burdens of real-time inversion of square matrices could be avoided. Finally the effectiveness of the identification algorithm is verified based on a 2-DOF uncertain robot model.

关键词： Parameter estimation Filtering Heuristic algorithms Estimation Real-time systems Manipulator dynamics Convergence

来源：评论

学校读者我要写书评

暂无评论

Stiffness Estimation and Intention Detection for Human-robot Collaboration

Stiffness Estimation and Intention Detection for Human-Robot...

引用

IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Xiongjun Chen Yiming Jiang Chenguang Yang Key Laboratory of Autonomous Systems and Networked Control College of Automation Science and Engineering South China University of Technology Guangzhou China National Engineering Laboratory for Robot Visual Perception and Control Hunan University Changsha China Bristol Robotics Laboratory University of the West of England Bristol UK

ISBN: (数字)9781728151694

ISBN: (纸本)9781728151700

In this paper, we propose a stiffness estimation and intention detection method for human-robot collaboration. The human arm endpoint stiffness can be obtained according to the muscle activation levels of the upper arm and the human arm configurations. The estimated endpoint stiffness of human arm is matching to the robot arm joint stiffness through an appropriate mapping. The motion intention of human arm is detected based on the wrist configuration which is recognized by a Myo armband attached at the forearm of the operator. In order to reduce the time of feature engineering to ensure the performance of real-time collaboration, the wrist configuration recognition is realised based on the neural learning algorithm. The sEMG of the human forearm is directly fed into the neural network after processing by filters and sliding windows. The force sensor at the end of the robot arm is embedded in the feedback loop to make the robot arm better adapted to the operator's movement. The results of experiments performed on Baxter robot platform illustrate a good performance and verifies the proposed method.

关键词： Wrist Neural networks Collaboration Estimation robot sensing systems Manipulators robots

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：