检索结果-内蒙古大学图书馆

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Xiao Ma Sumit Patidar Iain Haughton Stephen James Dyson Robot Learning Lab

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent which predicts a distant next-best end-effector pose (NBP), and a low-level goal-conditioned diffusion policy which generates optimal motion trajectories. The factorised policy representation allows HDP to tackle both long-horizon task planning while generating fine-grained low-level actions. To generate context-aware motion trajectories while satisfying robot kinematics constraints, we present a novel kinematics-aware goal-conditioned control agent, robot Kinematics Diffuser (RK-Diffuser). Specifically, RK-Diffuser learns to generate both the end-effector pose and joint position trajectories, and distill the accurate but kinematics-unaware end-effector pose diffuser to the kinematics-aware but less accurate joint position diffuser via differentiable kinematics. Empirically, we show that HDP achieves a significantly higher success rate than the state-of-the-art methods in both simulation and real-world. 1 1 Code and videos are available in our project page.

关键词： Accuracy robot kinematics Kinematics Multitasking End effectors Robustness Trajectory

来源：评论

学校读者我要写书评

暂无评论

Practical Considerations for Discrete-Time Implementations of Continuous-Time Control Barrier Function-Based Safety Filters

Practical Considerations for Discrete-Time Implementations o...

引用

American Control Conference (ACC)

作者： Brunke, Lukas Zhou, Siqi Che, Mingxuan Schoellig, Angela P. Tech Univ Munich Learning Syst & Robot Lab Munich Germany Univ Toronto Toronto ON Canada Munich Inst Robot & Machine Intelligence MIRMI Munich Germany Univ Toronto Inst Robot Toronto ON Canada Vector Inst Artificial Intelligence Toronto ON Canada

ISBN: (纸本)9798350382662;9798350382655

Safety filters based on control barrier functions (CBFs) have become a popular method to guarantee safety for uncertified control policies, e.g., as resulting from reinforcement learning. Here, safety is defined as staying in a pre-defined set, the safe set, that adheres to the system's state constraints, e.g., as given by lane boundaries for a self-driving vehicle. In this paper, we examine one commonly overlooked problem that arises in practical implementations of continuous-time CBF-based safety filters. In particular, we look at the issues caused by discrete-time implementations of the continuous-time CBF-based safety filter, especially for cases where the magnitude of the Lie derivative of the CBF with respect to the control input is zero or close to zero. When overlooked, this filter can result in undesirable chattering effects or constraint violations. In this work, we propose three mitigation strategies that allow us to use a continuous-time safety filter in a discrete-time implementation with a local relative degree. Using these strategies in augmented CBF-based safety filters, we achieve safety for all states in the safe set by either using an additional penalty term in the safety filtering objective or modifying the CBF such that those undesired states are not encountered during closed-loop operation. We demonstrate the presented issue and validate our three proposed mitigation strategies in simulation and on a real-world quadrotor.

关键词： Continuous time systems

来源：评论

学校读者我要写书评

暂无评论

Is Data All That Matters? The Role of Control Frequency for learning-Based Sampled-Data Control of Uncertain Systems

Is Data All That Matters? The Role of Control Frequency for ...

引用

American Control Conference (ACC)

作者： Roemer, Ralf Brunke, Lukas Zhou, Siqi Schoellig, Angela P. Tech Univ Munich Sch Computat Informat & Technol Learning Syst & Robot Lab Learn Syslab Org Munich Germany Tech Univ Munich Munich Inst Robot & Machine Intelligence MIRMI Munich Germany

ISBN: (纸本)9798350382662;9798350382655

learning models or control policies from data has become a powerful tool to improve the performance of uncertain systems. While a strong focus has been placed on increasing the amount and quality of data to improve performance, data can never fully eliminate uncertainty, making feedback necessary to ensure stability and performance. We show that the control frequency at which the input is recalculated is a crucial design parameter, yet it has hardly been considered before. We address this gap by combining probabilistic model learning and sampled-data control. We use Gaussian processes (GPs) to learn a continuous-time model and compute a corresponding discrete-time controller. The result is an uncertain sampled-data control system, for which we derive robust stability conditions. We formulate semidefinite programs to compute the minimum control frequency required for stability and to optimize performance. As a result, our approach enables us to study the effect of both control frequency and data on stability and closed-loop performance. We show in numerical simulations of a quadrotor that performance can be improved by increasing either the amount of data or the control frequency, and that we can trade off one for the other. For example, by increasing the control frequency by 33%, we can reduce the number of data points by half while still achieving similar performance.

关键词： Continuous time systems

来源：评论

学校读者我要写书评

暂无评论

Multiplanar Self-Calibration for Mobile Cobot 3D Object Manipulation using 2D Detectors and Depth Estimation

Multiplanar Self-Calibration for Mobile Cobot 3D Object Mani...

引用

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Dang, Tuan Nguyen, Khang Huber, Manfred Univ Texas Arlington Dept Comp Sci & Engn Learning & Adapt Robot Lab Arlington TX 76013 USA

ISBN: (纸本)9781665491907

Calibration is the first and foremost step in dealing with sensor displacement errors that can appear during extended operation and off-time periods to enable robot object manipulation with precision. In this paper, we present a novel multiplanar self-calibration between the camera system and the robot's end-effector for 3D object manipulation. Our approach first takes the robot end-effector as ground truth to calibrate the camera's position and orientation while the robot arm moves the object in multiple planes in 3D space, and a 2D state-of-the-art vision detector identifies the object's center in the image coordinates system. The transformation between world coordinates and image coordinates is then computed using 2D pixels from the detector and 3D known points obtained by robot kinematics. Next, an integrated stereo-vision system estimates the distance between the camera and the object, resulting in 3D object localization. We test our proposed method on the Baxter robot with two 7-DOF arms and a 2D detector that can run in real time on an onboard GPU. After self-calibrating, our robot can localize objects in 3D using an RGB camera and depth image. The source code is available at https://***/tuantdang/calib_cobot.

关键词： Calibration

来源：评论

学校读者我要写书评

暂无评论

Vision Foundation Model Enables Generalizable Object Pose Estimation 38

Vision Foundation Model Enables Generalizable Object Pose Es...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Chen, Kai Ma, Yiyao Lin, Xingyu James, Stephen Zhou, Jianshu Liu, Yun-Hui Abbeel, Pieter Dou, Qi The Chinese University of Hong Kong Hong Kong UC Berkeley United States Dyson Robot Learning Lab United Kingdom

Object pose estimation plays a crucial role in robotic manipulation, however, its practical applicability still suffers from limited generalizability. This paper addresses the challenge of generalizable object pose estimation, particularly focusing on category-level object pose estimation for unseen object categories. Current methods either require impractical instance-level training or are confined to predefined categories, limiting their applicability. We propose VFM-6D, a novel framework that explores harnessing existing vision and language models, to elaborate object pose estimation into two stages: category-level object viewpoint estimation and object coordinate map estimation. Based on the two-stage framework, we introduce a 2D-to-3D feature lifting module and a shape-matching module, both of which leverage pre-trained vision foundation models to improve object representation and matching accuracy. VFM-6D is trained on cost-effective synthetic data and exhibits superior generalization capabilities. It can be applied to both instance-level unseen object pose estimation and category-level object pose estimation for novel categories. Evaluations on benchmark datasets demonstrate the effectiveness and versatility of VFM-6D in various real-world scenarios. Project website: ***/. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Rapid learning with phase-change memory-based in-memory computing through learning-to-learn

引用

NATURE COMMUNICATIONS 2025年第1期16卷 1-16页

作者： Ortner, Thomas Petschenig, Horst Vasilopoulos, Athanasios Renner, Roland Brglez, Spela Limbacher, Thomas Pinero, Enrique Linares-Barranco, Alejandro Pantazi, Angeliki Legenstein, Robert IBM Res Europe Zurich Ruschlikon Switzerland Graz Univ Technol Inst Machine Learning & Neural Computat Graz Austria Univ Seville SCORE Lab EPS Robot & Technol Comp ETSII Seville Spain

There is a growing demand for low-power, autonomously learning artificial intelligence (AI) systems that can be applied at the edge and rapidly adapt to the specific situation at deployment site. However, current AI models struggle in such scenarios, often requiring extensive fine-tuning, computational resources, and data. In contrast, humans can effortlessly adjust to new tasks by transferring knowledge from related ones. The concept of learning-to-learn (L2L) mimics this process and enables AI models to rapidly adapt with only little computational effort and data. In-memory computing neuromorphic hardware (NMHW) is inspired by the brain's operating principles and mimics its physical co-location of memory and compute. In this work, we pair L2L with in-memory computing NMHW based on phase-change memory devices to build efficient AI models that can rapidly adapt to new tasks. We demonstrate the versatility of our approach in two scenarios: a convolutional neural network performing image classification and a biologically-inspired spiking neural network generating motor commands for a real robotic arm. Both models rapidly learn with few parameter updates. Deployed on the NMHW, they perform on-par with their software equivalents. Moreover, meta-training of these models can be performed in software with high-precision, alleviating the need for accurate hardware models.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Fine Manipulation Using a Tactile Skin: learning in Simulation and Sim-to-Real Transfer

Fine Manipulation Using a Tactile Skin: Learning in Simulati...

引用

2024 International Conference on Intelligent robots and Systems

作者： Kasolowsky, Ulf Baeuml, Berthold Tech Univ Munich Learning AI Dextrous Robots Lab Munich Germany German Aerosp Ctr DLR Inst Robot & Mechatron Cologne Germany

ISBN: (纸本)9798350377712;9798350377705

We want to enable fine manipulation with a multi-fingered robotic hand by using modern deep reinforcement learning methods. Key for fine manipulation is a spatially resolved tactile sensor. Here, we present a novel model of a tactile skin that can be used together with rigid-body (hence fast) physics simulators. The model considers the softness of the real fingertips such that a contact can spread across multiple taxels of the sensor depending on the contact geometry. We calibrate the model parameters to allow for an accurate simulation of the real-world sensor. For this, we present a self-contained calibration method without external tools or sensors. To demonstrate the validity of our approach, we learn two challenging fine manipulation tasks: Rolling a marble and a bolt between two fingers. We show in simulation experiments that tactile feedback is crucial for precise manipulation and reaching sub-taxel resolution of <1 mm (despite a taxel spacing of 4 mm). Moreover, we demonstrate that all policies successfully transfer from the simulation to the real robotic hand. Website: ***/skin/iros24

关键词： robotic arms

来源：评论

学校读者我要写书评

暂无评论

SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo 22

SimpleMapping: Real-Time Visual-Inertial Dense Mapping with ...

引用

22nd IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

作者： Xin, Yingye Zuo, Xingxing Lu, Dongyue Leutenegger, Stefan Tech Univ Munich Smart Robot Lab Munich Germany Munich Ctr Machine Learning MCML Munich Germany

ISBN: (纸本)9798350328387

We present a real-time visual-inertial dense mapping method capable of performing incremental 3D mesh reconstruction with high quality using only sequential monocular images and inertial measurement unit (IMU) readings. 6-DoF camera poses are estimated by a robust feature-based visual-inertial odometry (VIO), which also generates noisy sparse 3D map points as a by-product. We propose a sparse point aided multi-view stereo neural network (SPA-MVSNet) that can effectively leverage the informative but noisy sparse points from the VIO system. The sparse depth from VIO is firstly completed by a single-view depth completion network. This dense depth map, although naturally limited in accuracy, is then used as a prior to guide our MVS network in the cost volume generation and regularization for accurate dense depth prediction. Predicted depth maps of keyframe images by the MVS network are incrementally fused into a global map using TSDF-Fusion. We extensively evaluate both the proposed SPA-MVSNet and the entire dense mapping system on several public datasets as well as our own dataset, demonstrating the system's impressive generalization capabilities and its ability to deliver high-quality 3D reconstruction online. Our proposed dense mapping system achieves a 39.7% improvement in F-score over existing systems when evaluated on the challenging scenarios of the EuRoC dataset.

关键词： Real-Time System Dense Mapping Depth Completion Multi-view Stereo

来源：评论

学校读者我要写书评

暂无评论

Emotion Trajectory and Student Performance in Engineering Education: A Preliminary Study 18th

Emotion Trajectory and Student Performance in Engineering Ed...

引用

18th International Conference on Research Challenges in Information Sciences (RCIS)

作者： Nadaud, Edouard Yaacoub, Antoun Haidar, Siba Le Grand, Benedicte Prevost, Lionel ESIEA Learning Data & Robot LDR Lab Paris France Univ Paris 1 Pantheon Sorbonne Ctr Rech Informat CRI Paris France

ISBN: (纸本)9783031594649;9783031594656

In this study, we aim to establish the connection between the emotional trajectory of students during a pedagogical sequence and their performances. The project aims to develop an affective and intelligent tutoring system for detecting students facing difficulties and helping them. We designed this experimentation during the 2022-2023 academic year with students in a French engineering school. We collected and analyzed two primary data sources: student results from the learning Management System (LMS) and images captured by students' webcams during their learning activities. It is known that basic (primary) emotions (like fear or disgust) do not reflect student affective states when facing pedagogical issues (like misunderstanding or proudness). Since such "academic emotions" are not easy to define and detect, we changed the paradigm and used a 2D dimensional model that describes better the wide spectrum of emotion encountered. Moreover, it allows to build a temporal emotion trajectory reflecting the student's emotional trajectory. Firstly, we observed a correlation between these trajectories and academic results. Secondly, we found that high-performing student trajectories are significantly different from the others. These preliminary results, support the idea that emotions are pivotal in distinguishing highly performing students from their less successful counterparts. This is the first step to assess students' profiles and proactively identify those at risk of failure in a human learning context.

关键词： Emotion Trajectories Affective Computing Early Difficulties Detection Machine-learning Applications Ethical AI

来源：评论

学校读者我要写书评

暂无评论

InsActor: Instruction-driven Physics-based Characters 37

InsActor: Instruction-driven Physics-based Characters

引用

37th Conference on Neural Information Processing Systems (NeurIPS)

作者： Ren, Jiawei Zhang, Mingyuan Yu, Cunjun Ma, Xiao Pan, Liang Liu, Ziwei Nanyang Technol Univ S Lab Singapore Singapore Natl Univ Singapore Singapore Singapore Dyson Robot Learning Lab London England

ISBN: (纸本)9781713899921

Generating animation of physics-based characters with intuitive control has long been a desirable task with numerous applications. However, generating physically simulated animations that reflect high-level human instructions remains a difficult problem due to the complexity of physical environments and the richness of human language. In this paper, we present InsActor, a principled generative framework that leverages recent advancements in diffusion-based human motion models to produce instruction-driven animations of physics-based characters. Our framework empowers InsActor to capture complex relationships between high-level human instructions and character motions by employing diffusion policies for flexibly conditioned motion planning. To overcome invalid states and infeasible state transitions in planned motions, InsActor discovers low-level skills and maps plans to latent skill sequences in a compact latent space. Extensive experiments demonstrate that InsActor achieves state-of-the-art results on various tasks, including instruction-driven motion generation and instruction-driven waypoint heading. Notably, the ability of InsActor to generate physically simulated animations using high-level human instructions makes it a valuable tool, particularly in executing long-horizon tasks with a rich set of instructions. Our project page is available at ***/projects/insactor/***

关键词： High level languages

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：