检索结果-内蒙古大学图书馆

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Ma, Xiao Patidar, Sumit Haughton, Iain James, Stephen Dyson Robot Learning Lab London England

ISBN: (纸本)9798350353006

This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent which predicts a distant next-best end-effector pose (NBP), and a low-level goal-conditioned diffusion policy which generates optimal motion trajectories. The factorised policy representation allows HDP to tackle both long-horizon task planning while generating fine-grained low-level actions. To generate context-aware motion trajectories while satisfying robot kinematics constraints, we present a novel kinematics-aware goal-conditioned control agent, robot Kinematics Diffuser (RK-Diffuser). Specifically, RK-Diffuser learns to generate both the end-effector pose and joint position trajectories, and distill the accurate but kinematics-unaware end-effector pose diffuser to the kinematics-aware but less accurate joint position diffuser via differentiable kinematics. Empirically, we show that HDP achieves a significantly higher success rate than the state-of-the-art methods in both simulation and real-world.1

关键词： Diffusion Policy robotic Manipulation

来源：评论

学校读者我要写书评

暂无评论

Continuous Control with Coarse-to-fine Reinforcement learning 8

Continuous Control with Coarse-to-fine Reinforcement Learnin...

引用

8th Conference on robot learning, CoRL 2024

作者： Seo, Younggyo Uruç, Jafar James, Stephen Dyson Robot Learning Lab United Kingdom

Despite recent advances in improving the sample-efficiency of reinforcement learning (RL) algorithms, designing an RL algorithm that can be practically deployed in real-world environments remains a challenge. In this paper, we present Coarse-to-fine Reinforcement learning (CRL), a framework that trains RL agents to zoom-into a continuous action space in a coarse-to-fine manner, enabling the use of stable, sample-efficient value-based RL algorithms for fine-grained continuous control tasks. Our key idea is to train agents that output actions by iterating the procedure of (i) discretizing the continuous action space into multiple intervals and (ii) selecting the interval with the highest Q-value to further discretize at the next level. We then introduce a concrete, value-based algorithm within the CRL framework called Coarse-to-fine Q-Network (CQN). Our experiments demonstrate that CQN significantly outperforms RL and behavior cloning baselines on 20 sparsely-rewarded RLBench manipulation tasks with a modest number of environment interactions and expert demonstrations. We also show that CQN robustly learns to solve real-world manipulation tasks within a few minutes of online training. Project website: ***/cqn. © 2024 Proceedings of Machine learning Research.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark 8

BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark

引用

8th Conference on robot learning, CoRL 2024

作者： Chernyadev, Nikita Backshall, Nicholas Ma, Xiao Lu, Yunfan Seo, Younggyo James, Stephen Dyson Robot Learning Lab United Kingdom

We introduce BiGym, a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching to complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports a variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views. To validate the usability of BiGym, we thoroughly benchmark the state-of-the-art imitation learning algorithms and demo-driven reinforcement learning algorithms within the environment and discuss the future opportunities. Project website: https://***/bigym/. © 2024 Proceedings of Machine learning Research.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

Generative Image as Action Models 8

Generative Image as Action Models

引用

8th Conference on robot learning, CoRL 2024

作者： Shridhar, Mohit Lo, Yat Long James, Stephen Dyson Robot Learning Lab United Kingdom

Image-generation diffusion models have been fine-tuned to unlock new capabilities such as image-editing and novel view synthesis. Can we similarly unlock image-generation models for visuomotor control? We present GENIMA, a behavior-cloning agent that fine-tunes Stable Diffusion to "draw joint-actions" as targets on RGB images. These images are fed into a controller that maps the visual targets into a sequence of joint-positions. We study GENIMA on 25 RLBench and 9 real-world manipulation tasks. We find that, by lifting actions into image-space, internet pre-trained diffusion models can generate policies that outperform state-of-the-art visuomotor approaches, especially in robustness to scene perturbations and generalizing to novel objects. Our method is also competitive with 3D agents, despite lacking priors such as depth, keypoints, or motion-planners. © 2024 Proceedings of Machine learning Research.

关键词： Clone cells

来源：评论

学校读者我要写书评

暂无评论

Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models

Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-La...

引用

IEEE International Conference on robotics and Automation (ICRA)

作者： Kapelyukh, Ivan Ren, Yifei Alzugaray, Ignacio Johns, Edward Imperial Coll London Robot Learning Lab London England Dyson Robot Lab London England

ISBN: (纸本)9798350384581;9798350384574

We introduce Dream2Real, a robotics framework which integrates vision-language models (VLMs) trained on 2D data into a 3D object rearrangement pipeline. This is achieved by the robot autonomously constructing a 3D representation of the scene, where objects can be rearranged virtually and an image of the resulting arrangement rendered. These renders are evaluated by a VLM, so that the arrangement which best satisfies the user instruction is selected and recreated in the real world with pick-and-place. This enables language-conditioned rearrangement to be performed zero-shot, without needing to collect a training dataset of example arrangements. Results on a series of real-world tasks show that this framework is robust to distractors, controllable by language, capable of understanding complex multi-object relations, and readily applicable to both tabletop and 6-DoF rearrangement tasks. Videos are available on our webpage at: https://***/dream2real.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

Real-time Mapping of Physical Scene Properties with an Autonomous robot Experimenter 6

Real-time Mapping of Physical Scene Properties with an Auton...

引用

6th Conference on robot learning (CoRL)

作者： Haughton, Iain Sucar, Edgar Mouton, Andre Johns, Edward Davison, Andrew J. Dyson Technol Ltd Malmesbury England Imperial Coll Dyson Robot Lab London England Imperial Coll Robot Learning Lab London England

Neural fields can be trained from scratch to represent the shape and appearance of 3D scenes efficiently. It has also been shown that they can densely map correlated properties such as semantics, via sparse interactions from a human labeller. In this work, we show that a robot can densely annotate a scene with arbitrary discrete or continuous physical properties via its own fully-autonomous experimental interactions, as it simultaneously scans and maps it with an RGB-D camera. A variety of scene interactions are possible, including poking with force sensing to determine rigidity, measuring local material type with single-pixel spectroscopy or predicting force distributions by pushing. Sparse experimental interactions are guided by entropy to enable high efficiency, with tabletop scene properties densely mapped from scratch in a few minutes from a few tens of interactions.

关键词： neural field robot experimentation physical properties

来源：评论

学校读者我要写书评

暂无评论

Redundancy-Aware Action Spaces for robot learning

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第8期9卷 6912-6919页

作者： Mazzaglia, Pietro Backshall, Nicholas Ma, Xiao James, Stephen Univ Ghent DLab B-9052 Ghent Belgium Dyson Robot Learning Lab London SW7 2AZ England

Joint space and task space control are the two dominant action modes for controlling robot arms within the robot learning literature. Actions in joint space provide precise control over the robot's pose, but tend to suffer from inefficient training;actions in task space boast data-efficient training but sacrifice the ability to perform tasks in confined spaces due to limited control over the full joint configuration. This work analyses the criteria for designing action spaces for robot manipulation and introduces ER (End-effector Redundancy), a novel action space formulation that, by addressing the redundancies present in the manipulator, aims to combine the advantages of both joint and task spaces, offering fine-grained comprehensive control with overactuated robot arms whilst achieving highly efficient robot learning. We present two implementations of ER, ERAngle (ERA) and ERJoint (ERJ), and we show that ERJ in particular demonstrates superior performance across multiple settings, especially when precise control over the robot configuration is required. We validate our results both in simulated and real robotic environments.

关键词： Reinforcement learning imitation learning machine learning for robot control

来源：评论

学校读者我要写书评

暂无评论

DALL-E-Bot: Introducing Web-Scale Diffusion Models to robotics

引用

IEEE robotICS AND AUTOMATION LETTERS 2023年第7期8卷 3956-3963页

作者： Kapelyukh, Ivan Vosylius, Vitalis Johns, Edward Imperial Coll London Robot Learning Lab London SW7 2AZ England Imperial Coll London Dyson Robot Lab London SW7 2AZ England

We introduce the first work to explore web-scale diffusion models for robotics. DALL-E-Bot enables a robot to rearrange objects in a scene, by first inferring a text description of those objects, then generating an image representing a natural, human-like arrangement of those objects, and finally physically arranging the objects according to that goal image. We show that this is possible zero-shot using DALL-E, without needing any further example arrangements, data collection, or training. DALL-E-Bot is fully autonomous and is not restricted to a pre-defined set of objects or scenes, thanks to DALL-E's web-scale pre-training. Encouraging real-world results, with both human studies and objective metrics, show that integrating web-scale diffusion models into robotics pipelines is a promising direction for scalable, unsupervised robot learning.

关键词： AI-based methods Big Data in robotics and automation deep learning in grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

Language-Conditioned Path Planning 7

Language-Conditioned Path Planning

引用

7th Conference on robot learning (CoRL)

作者： Xie, Amber Lee, Youngwoon Abbeel, Pieter James, Stephen Univ Calif Berkeley Berkeley CA 94720 USA Dyson Robot Learning Lab London England

Contact is at the core of robotic manipulation. At times, it is desired (e.g. manipulation and grasping), and at times, it is harmful (e.g. when avoiding obstacles). However, traditional path planning algorithms focus solely on collision-free paths, limiting their applicability in contact-rich tasks. To address this limitation, we propose the domain of Language-Conditioned Path Planning, where contact-awareness is incorporated into the path planning problem. As a first step in this domain, we propose Language-Conditioned Collision Functions (LACO), a novel approach that learns a collision function using only a single-view image, language prompt, and robot configuration. LACO predicts collisions between the robot and the environment, enabling flexible, conditional path planning without the need for manual object annotations, point cloud data, or ground-truth object meshes. In both simulation and the real world, we demonstrate that LACO can facilitate complex, nuanced path plans that allow for interaction with objects that are safe to collide, rather than prohibiting any collision.

关键词： robotic Manipulation Path Planning Collision Avoidance Learned Collision Function

来源：评论

学校读者我要写书评

暂无评论

Multi-View Masked World Models for Visual robotic Manipulation 40

Multi-View Masked World Models for Visual Robotic Manipulati...

引用

40th International Conference on Machine learning, ICML 2023

作者： Seo, Younggyo Kim, Junsu James, Stephen Lee, Kimin Shin, Jinwoo Abbeel, Pieter KAIST Korea Republic of Dyson Robot Learning Lab Google Research UC Berkeley United States

Visual robotic manipulation research and applications often use multiple cameras, or views, to better perceive the world. How else can we utilize the richness of multi-view data? In this paper, we investigate how to learn good representations with multi-view data and utilize them for visual robotic manipulation. Specifically, we train a multi-view masked autoencoder which reconstructs pixels of randomly masked viewpoints and then learn a world model operating on the representations from the autoencoder. We demonstrate the effectiveness of our method in a range of scenarios, including multi-view control and single-view control with auxiliary cameras for representation learning. We also show that the multi-view masked autoencoder trained with multiple randomized viewpoints enables training a policy with strong viewpoint randomization and transferring the policy to solve real-robot tasks without camera calibration and an adaptation procedure. Video demonstrations are available at: https://***/view/mv-mwm. © 2023 Proceedings of Machine learning Research. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：