检索结果-内蒙古大学图书馆

Multi-Keypoint Affordance Representation for Functional Dexterous Grasping

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yang, Fan Luo, Dongsheng Chen, Wenrui Lin, Jiacheng Cai, Junjie Yang, Kailun Li, Zhiyong Wang, Yaonan School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China College of Computer Science and Electronic Engineering Hunan University Changsha China

Functional dexterous grasping requires precise hand-object interaction, going beyond simple gripping. Existing affordance-based methods primarily predict coarse interaction regions and cannot directly constrain the grasping posture, leading to a disconnection between visual perception and manipulation. To address this issue, we propose a multi-keypoint affordance representation for functional dexterous grasping, which directly encodes task-driven grasp configurations by localizing functional contact points. Our method introduces Contact-guided Multi-Keypoint Affordance (CMKA), leveraging human grasping experience images for weak supervision combined with Large Vision Models for fine affordance feature extraction, achieving generalization while avoiding manual keypoint annotations. Additionally, we present a Keypoint-based Grasp matrix Transformation (KGT) method, ensuring spatial consistency between hand keypoints and object contact points, thus providing a direct link between visual perception and dexterous grasping actions. Experiments on public real-world FAH datasets, IsaacGym simulation, and challenging robotic tasks demonstrate that our method significantly improves affordance localization accuracy, grasp consistency, and generalization to unseen tools and tasks, bridging the gap between visual affordance learning and dexterous robotic manipulation. The source code and demo videos will be publicly available at https://***/PopeyePxx/MKA. Copyright © 2025, The Authors. All rights reserved.

关键词： Vision

Neural Bayes estimators for censored inference with peaks-over-threshold models

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2024年第1期25卷 19070-19118页

作者： Jordan Richards Matthew Sainsbury-Dale Andrew Zammit-Mangion Raphaël Huser School of Mathematics and Maxwell Institute for Mathematical Sciences University of Edinburgh Edinburgh UK and Statistics Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) Thuwal Saudi Arabia Statistics Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) Thuwal Saudi Arabia and School of Mathematics and Applied Statistics University of Wollongong Wollongong Australia School of Mathematics and Applied Statistics University of Wollongong Wollongong Australia Statistics Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology (KAUST) Thuwal Saudi Arabia

Making inference with spatial extremal dependence models can be computationally burdensome since they involve intractable and/or censored likelihoods. Building on recent advances in likelihood-free inference with neural Bayes estimators, that is, neural networks that approximate Bayes estimators, we develop highly efficient estimators for censored peaks-over-threshold models that use augmented data to encode censoring information in the neural network input. Our new method provides a paradigm shift that challenges traditional censored likelihood-based inference methods for spatial extremal dependence models. Our simulation studies highlight significant gains in both computational and statistical efficiency, relative to competing likelihood-based approaches, when applying our novel estimators to make inference with popular extremal dependence models, such as max-stable, r-Pareto, and random scale mixture process models. We also illustrate that it is possible to train a single neural Bayes estimator for a general censoring level, precluding the need to retrain the network when the censoring level is changed. We illustrate the efficacy of our estimators by making fast inference on hundreds-of-thousands of high-dimensional spatial extremal dependence models to assess extreme particulate matter 2.5 microns or less in diameter (PM2:5) concentration over the whole of Saudi Arabia.

关键词： censored data convolutional neural network likelihood-free inference neural estimation spatial extremes threshold exceedance

AirShot: Efficient Few-Shot Detection for Autonomous Exploration

学校读者我要写书评

暂无评论

AirShot: Efficient Few-Shot Detection for Autonomous Explora...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Zihan Wang Bowen Li Chen Wang Sebastian Scherer Institute for Imaging Data and Communications School of Engineering The University of Edinburgh UK AirLab Robotics Institute Carnegie Mellon University Pittsburgh PA USA Department of Computer Science and Engineering Spatial AI & Robotics (SAIR) Lab Institute for Artificial Intelligence and Data Science University at Buffalo NY USA

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Few-shot object detection has drawn increasing attention in the field of robotic exploration, where robots are required to find unseen objects with a few online provided examples. Despite recent efforts have been made to yield online processing capabilities, slow inference speeds of low-powered robots fail to meet the demands of real-time detection-making them impractical for autonomous exploration. Existing methods still face performance and efficiency challenges, mainly due to unreliable features and exhaustive class loops. In this work, we propose a new paradigm AirShot, and discover that, by fully exploiting the valuable correlation map, AirShot can result in a more robust and faster few-shot object detection system, which is more applicable to robotics community. The core module Top Prediction Filter (TPF) can operate on multi-scale correlation maps in both the training and inference stages. During training, TPF supervises the generation of a more representative correlation map, while during inference, it reduces looping iterations by selecting top-ranked classes, thus cutting down on computational costs with better performance. Surprisingly, this dual functionality exhibits general effectiveness and efficiency on various off-the-shelf models. Exhaustive experiments on COCO2017, VOC2014, and SubT datasets demonstrate that TPF can significantly boost the efficacy and efficiency of most off-the-shelf models, achieving up to 36.4% precision improvements along with 56.3% faster inference speed. Code and Data are at: https://***/ImNotPrepared/AirShot.

关键词： Training Correlation Codes Computational modeling Atmospheric modeling Object detection Real-time systems Computational efficiency Intelligent robots Faces

VARIQuery: VAE Segment-Based Active Learning for Query Selection in Preference-Based Reinforcement Learning

学校读者我要写书评

暂无评论

VARIQuery: VAE Segment-Based Active Learning for Query Selec...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Daniel Marta Simon Holk Christian Pek Jana Tumova Iolanda Leite Division of Robotics Perception and Learning School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm Sweden Digital Futures

Human-in-the-loop reinforcement learning (RL) methods actively integrate human knowledge to create reward functions for various robotic tasks. Learning from preferences shows promise as alleviates the requirement of demonstrations by querying humans on state-action sequences. However, the limited granularity of sequence-based approaches complicates temporal credit assignment. The amount of human querying is contingent on query quality, as redundant queries result in excessive human involvement. This paper addresses the often-overlooked aspect of query selection, which is closely related to active learning (AL). We propose a novel query selection approach that leverages variational autoencoder (VAE) representations of state sequences. In this manner, we formulate queries that are diverse in nature while simultaneously taking into account reward model estimations. We compare our approach to the current state-of-the-art query selection methods in preference-based RL, and find ours to be either on-par or more sample efficient through extensive benchmarking on simulated environments relevant to robotics. Lastly, we conduct an online study to verify the effectiveness of our query selection approach with real human feedback and examine several metrics related to human effort.

关键词：

On Motion Blur and Deblurring in Visual Place Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ismagilov, Timur Ferrarini, Bruno Milford, Michael Nguyen, Tan Viet Tuyen Ramchurn, S.D. Ehsan, Shoaib The School of Electronics and Computer Science University of Southampton SouthamptonSO17 1BJ United Kingdom MyWay srl Via Osti 6 PC Vernasca29010 Italy The QUT Centre for Robotics School of Electrical Engineering and Robotics BrisbaneQLD4000 Australia The School of Computer Science and Electronic Engineering University of Essex ColchesterCO4 3SQ United Kingdom

Visual Place Recognition (VPR) in mobile robotics enables robots to localize themselves by recognizing previously visited locations using visual data. While the reliability of VPR methods has been extensively studied under conditions such as changes in illumination, season, weather and viewpoint, the impact of motion blur is relatively unexplored despite its relevance not only in rapid motion scenarios but also in low-light conditions where longer exposure times are necessary. Similarly, the role of image deblurring in enhancing VPR performance under motion blur has received limited attention so far. This paper bridges these gaps by introducing a new benchmark designed to evaluate VPR performance under the influence of motion blur and image deblurring. The benchmark includes three datasets that encompass a wide range of motion blur intensities, providing a comprehensive platform for analysis. Experimental results with several well-established VPR and image deblurring methods provide new insights into the effects of motion blur and the potential improvements achieved through deblurring. Building on these findings, the paper proposes adaptive deblurring strategies for VPR, designed to effectively manage motion blur in dynamic, real-world scenarios. © 2024, CC BY.

关键词： Mobile robots

Enhancing Environmental Robustness in Few-shot Learning via Conditional Representation Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Guo, Qianyu Wu, Jingrong Wu, Tianxing Wang, Haofen Ge, Weifeng Zhang, Wenqiang School of Computer Science Fudan University China Shanghai Institute of Virology Shanghai Jiao Tong University School of Medicine China School of Computer Science and Engineering Southeast University Nanjing210096 China College of Design and Innovation Tongji University China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai20043 China

Few-shot learning (FSL) has recently been extensively utilized to overcome the scarcity of training data in domain-specific visual recognition. In real-world scenarios, environmental factors such as complex backgrounds, varying lighting conditions, long-distance shooting, and moving targets often cause test images to exhibit numerous incomplete targets or noise disruptions. However, current research on evaluation datasets and methodologies has largely ignored the concept of "environmental robustness", which refers to maintaining consistent performance in complex and diverse physical environments. This neglect has led to a notable decline in the performance of FSL models during practical testing compared to their training performance. To bridge this gap, we introduce a new real-world multi-domain few-shot learning (RD-FSL) benchmark, which includes four domains and six evaluation datasets. The test images in this benchmark feature various challenging elements, such as camouflaged objects, small targets, and blurriness. Our evaluation experiments reveal that existing methods struggle to utilize training images effectively to generate accurate feature representations for challenging test images. To address this problem, we propose a novel conditional representation learning network (CRLNet) that integrates the interactions between training and testing images as conditional information in their respective representation processes. The main goal is to reduce intra-class variance or enhance inter-class variance at the feature representation level. Finally, comparative experiments reveal that CRLNet surpasses the current state-of-the-art methods, achieving performance improvements ranging from 6.83% to 16.98% across diverse settings and backbones. The source code and dataset are available at https://***/guoqianyualberta/Conditional-Representation-Learning. Copyright © 2025, The Authors. All rights reserved.

关键词： Zero-shot learning

Hi-EF: Benchmarking Emotion Forecasting in Human-interaction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Haoran Mai, Xinji Tao, Zeng Wang, Yan Yu, Jiawen Zhou, Ziheng Tong, Xuan Yan, Shaoqi Zhao, Qing Gao, Shuyong Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China School of Information Science and Technology Fudan University Shanghai China

Affective Forecasting, a research direction in psychology that predicts individuals' future emotions, is often constrained by numerous external factors like social influence and temporal distance. To address this, we transform Affective Forecasting into a Deep Learning problem by designing an Emotion Forecasting paradigm based on two-party interactions. We propose a novel Emotion Forecasting (EF) task grounded in the theory that an individual's emotions are easily influenced by the emotions or other information conveyed during interactions with another person. To tackle this task, we have developed a specialized dataset, Humaninteraction- based Emotion Forecasting (Hi-EF), which contains 3069 two-party Multilayered-Contextual Interaction Samples (MCIS) with abundant affectiverelevant labels and three modalities. Hi-EF not only demonstrates the feasibility of the EF task but also highlights its potential. Additionally, we propose a methodology that establishes a foundational and referential baseline model for the EF task and extensive experiments are provided. The dataset and code is available HERE. Copyright © 2024, The Authors. All rights reserved.

关键词： Prediction models

Aligning Human Preferences with Baseline Objectives in Reinforcement Learning

学校读者我要写书评

暂无评论

Aligning Human Preferences with Baseline Objectives in Reinf...

IEEE International Conference on robotics and Automation (ICRA)

Practical implementations of deep reinforcement learning (deep RL) have been challenging due to an amplitude of factors, such as designing reward functions that cover every possible interaction. To address the heavy burden of robot reward engineering, we aim to leverage subjective human preferences gathered in the context of human-robot interaction, while taking advantage of a baseline reward function when available. By considering baseline objectives to be designed beforehand, we are able to narrow down the policy space, solely requesting human attention when their input matters the most. To allow for control over the optimization of different objectives, our approach contemplates a multi-objective setting. We achieve human-compliant policies by sequentially training an optimal policy from a baseline specification and collecting queries on pairs of trajectories. These policies are obtained by training a reward estimator to generate Pareto optimal policies that include human preferred behaviours. Our approach ensures sample efficiency and we conducted a user study to collect real human preferences, which we utilized to obtain a policy on a social navigation environment.

关键词：

Non-smooth Control Barrier Functions for Stochastic Dynamical Systems

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Vahs, Matti Tumova, Jana The Division of Robotics Perception and Learning School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm Sweden Digital Futures

Uncertainties arising in various control systems, such as robots that are subject to unknown disturbances or environmental variations, pose significant challenges for ensuring system safety, such as collision avoidance. At the same time, safety specifications are getting more and more complex, e.g., by composing multiple safety objectives through Boolean operators resulting in non-smooth descriptions of safe sets. Control Barrier Functions (CBFs) have emerged as a control technique to provably guarantee system safety. In most settings, they rely on an assumption of having deterministic dynamics and smooth safe sets. This paper relaxes these two assumptions by extending CBFs to encompass control systems with stochastic dynamics and safe sets defined by non-smooth functions. By explicitly considering the stochastic nature of system dynamics and accommodating complex safety specifications, our method enables the design of safe control strategies in uncertain and complex systems. We provide formal guarantees on the safety of the system by leveraging the theoretical foundations of stochastic CBFs and non-smooth safe sets. Numerical simulations demonstrate the effectiveness of the approach in various scenarios. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems