检索结果-内蒙古大学图书馆

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Ma, Xiao Patidar, Sumit Haughton, Iain James, Stephen Dyson Robot Learning Lab London England

ISBN: (纸本)9798350353006

This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent which predicts a distant next-best end-effector pose (NBP), and a low-level goal-conditioned diffusion policy which generates optimal motion trajectories. The factorised policy representation allows HDP to tackle both long-horizon task planning while generating fine-grained low-level actions. To generate context-aware motion trajectories while satisfying robot kinematics constraints, we present a novel kinematics-aware goal-conditioned control agent, robot Kinematics Diffuser (RK-Diffuser). Specifically, RK-Diffuser learns to generate both the end-effector pose and joint position trajectories, and distill the accurate but kinematics-unaware end-effector pose diffuser to the kinematics-aware but less accurate joint position diffuser via differentiable kinematics. Empirically, we show that HDP achieves a significantly higher success rate than the state-of-the-art methods in both simulation and real-world.1

关键词： Diffusion Policy robotic Manipulation

来源：评论

学校读者我要写书评

暂无评论

learning to Search in Task and Motion Planning With Streams

引用

IEEE robotICS AND AUTOMATION LETTERS 2023年第4期8卷 1983-1990页

作者： Khodeir, Mohamed Agro, Ben Shkurti, Florian Univ Toronto Robot Vis & Learning Lab Robot Inst Toronto ON M5S Canada

Task and motion planning problems in robotics combine symbolic planning over discrete task variables with motion optimization over continuous state and action variables. Recent works such as PDDLStream [Garrett et al. 2020)] have focused on optimistic planning with an incrementally growing set of objects until a feasible trajectory is found. However, this set is exhaustively expanded in a breadth-first manner, regardless of the logical and geometric structure of the problem at hand, which makes long-horizon reasoning with large numbers of objects prohibitively time-consuming. To address this issue, we propose a geometrically informed symbolic planner that expands the set of objects and facts in a best-first manner, prioritized by a Graph Neural Network that is learned from prior search computations. We evaluate our approach on a diverse set of problems and demonstrate an improved ability to plan in difficult scenarios. We also apply our algorithm on a 7DOF robotic arm in block-stacking manipulation tasks.

关键词： Planning Task analysis Optimized production technology Generators Cognition Stacking Search problems Task and motion planning integrated planning and learning manipulation planning

来源：评论

学校读者我要写书评

暂无评论

Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models

Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-La...

引用

IEEE International Conference on robotics and Automation (ICRA)

作者： Kapelyukh, Ivan Ren, Yifei Alzugaray, Ignacio Johns, Edward Imperial Coll London Robot Learning Lab London England Dyson Robot Lab London England

ISBN: (纸本)9798350384581;9798350384574

We introduce Dream2Real, a robotics framework which integrates vision-language models (VLMs) trained on 2D data into a 3D object rearrangement pipeline. This is achieved by the robot autonomously constructing a 3D representation of the scene, where objects can be rearranged virtually and an image of the resulting arrangement rendered. These renders are evaluated by a VLM, so that the arrangement which best satisfies the user instruction is selected and recreated in the real world with pick-and-place. This enables language-conditioned rearrangement to be performed zero-shot, without needing to collect a training dataset of example arrangements. Results on a series of real-world tasks show that this framework is robust to distractors, controllable by language, capable of understanding complex multi-object relations, and readily applicable to both tabletop and 6-DoF rearrangement tasks. Videos are available on our webpage at: https://***/dream2real.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第3期9卷 2032-2039页

作者： Di Palo, Norman Johns, Edward Imperial Coll London Robot Learning Lab London SW7 2BX England

Imitation learning with visual observations is notoriously inefficient when addressed with end-to-end behavioural cloning methods. In this letter, we explore an alternative paradigm which decomposes reasoning into three phases. First, a retrieval phase, which informs the robot what it can do with an object. Second, an alignment phase, which informs the robot where to interact with the object. And third, a replay phase, which informs the robot how to interact with the object. Through a series of real-world experiments on everyday tasks, such as grasping, pouring, and inserting objects, we show that this decomposition brings unprecedented learning efficiency, and effective inter- and intra-class generalisation.

关键词： robots End effectors Visualization Trajectory Task analysis Training robot vision systems Deep learning in grasping and manipulation imitation learning learning from demonstration

来源：评论

学校读者我要写书评

暂无评论

Language Models as Zero-Shot Trajectory Generators

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第7期9卷 6728-6735页

作者： Kwon, Teyun Di Palo, Norman Johns, Edward Imperial Coll London Robot Learning Lab London SW7 2AZ England

Large Language Models (LLMs) have recently shown promise as high-level planners for robots when given access to a selection of low-level skills. However, it is often assumed that LLMs do not possess sufficient knowledge to be used for the low-level trajectories themselves. In this work, we address this assumption thoroughly, and investigate if an LLM (GPT-4) can directly predict a dense sequence of end-effector poses for manipulation tasks, when given access to only object detection and segmentation vision models. We designed a single, task-agnostic prompt, without any in-context examples, motion primitives, or external trajectory optimisers. Then we studied how well it can perform across 30 real-world language-based tasks, such as "open the bottle cap" and "wipe the plate with the sponge", and we investigated which design choices in this prompt are the most important. Our conclusions raise the assumed limit of LLMs for robotics, and we reveal for the first time that LLMs do indeed possess an understanding of low-level robot control sufficient for a range of common tasks, and that they can additionally detect failures and then re-plan trajectories accordingly.

关键词： Task analysis robots Trajectory End effectors Codes robot kinematics Object detection AI-based methods big data in robotics and automation deep learning in grasping and manipulation

来源：评论

学校读者我要写书评

暂无评论

Adapting Skills to Novel Grasps: A Self-Supervised Approach

Adapting Skills to Novel Grasps: A Self-Supervised Approach

引用

2024 International Conference on Intelligent robots and Systems

作者： Papagiannis, Georgios Dreczkowski, Kamil Vosylius, Vitalis Johns, Edward Imperial Coll London Robot Learning Lab London England

ISBN: (纸本)9798350377712;9798350377705

In this paper, we study the problem of adapting manipulation trajectories involving grasped objects (e.g. tools) defined for a single grasp pose to novel grasp poses. A common approach to address this is to define a new trajectory for each possible grasp explicitly, but this is highly inefficient. Instead, we propose a method to adapt such trajectories directly while only requiring a period of self-supervised data collection, during which a camera observes the robot's end-effector moving with the object rigidly grasped. Importantly, our method requires no prior knowledge of the grasped object (such as a 3D CAD model), it can work with RGB images, depth images, or both, and it requires no camera calibration. Through a series of real-world experiments involving 1360 evaluations, we find that self-supervised RGB data consistently outperforms alternatives that rely on depth images including several state-of-the-art pose estimation methods. Compared to the best-performing baseline, our method results in an average of 28.5% higher success rate when adapting manipulation trajectories to novel grasps on several everyday tasks. The appendix accompanying the paper and videos of the experiments are available on our webpage at ***/adapting-skills.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Bathymetric Reconstruction From Sidescan Sonar With Deep Neural Networks

引用

IEEE JOURNAL OF OCEANIC ENGINEERING 2023年第2期48卷 372-383页

作者： Xie, Yiping Bore, Nils Folkesson, John Royal Inst Technol Robot Percept & Learning Lab SE-10044 Stockholm Sweden

In this article, we propose a novel data-driven approach for high-resolution bathymetric reconstruction from sidescan. Sidescan sonar intensities as a function of range do contain some information about the slope of the seabed. However, that information must be inferred. In addition, the navigation system provides the estimated trajectory, and normally, the altitude along this trajectory is also available. From these, we obtain a very coarse seabed bathymetry as an input. This is then combined with the indirect but high-resolution seabed slope information from the sidescan to estimate the full bathymetry. This sparse depth could be acquired by single-beam echo sounder, Doppler velocity log, and other bottom tracking sensors or bottom tracking algorithm from sidescan itself. In our work, a fully convolutional network is used to estimate the depth contour and its aleatoric uncertainty from the sidescan images and sparse depth in an end-to-end fashion. The estimated depth is then used together with the range to calculate the point's three-dimensional location on the seafloor. A high-quality bathymetric map can be reconstructed after fusing the depth predictions and the corresponding confidence measures from the neural networks. We show the improvement of the bathymetric map gained by using sparse depths with sidescan over estimates with sidescan alone. We also show the benefit of confidence weighting when fusing multiple bathymetric estimates into a single map.

关键词： Bathymetric mapping data-driven neural network sidescan sonar (SSS)

来源：评论

学校读者我要写书评

暂无评论

Neural Network Normal Estimation and Bathymetry Reconstruction From Sidescan Sonar

引用

IEEE JOURNAL OF OCEANIC ENGINEERING 2023年第1期48卷 218-232页

作者： Xie, Yiping Bore, Nils Folkesson, John Royal Inst Technol Robot Percept & Learning Lab SE-10044 Stockholm Sweden

Sidescan sonar intensity encodes information about changes in the surface normal of the seabed. However, other factors such as seabed geometry as well as its material composition also affect the return intensity. One can model these intensity changes in a forward direction from the surface normals from a bathymetric map and physical properties to the measured intensity, or alternatively one can use an inverse model which starts from the intensities and models the surface normals. Here, we use an inverse model which leverages deep learning's ability to learn from data;a convolutional neural network is used to estimate the surface normal from the sidescan. Once this information is estimated, a bathymetric map can be reconstructed through an optimization framework that also includes altimeter readings to provide a sparse depth profile as a constraint. Implicit neural representation learning was recently proposed to represent the bathymetric map in such an optimization framework. In this article, we use a neural network to represent the map and optimize it under constraints of altimeter points and estimated surface normal from sidescan. By fusing multiple observations from different angles from several sidescan lines, the estimated results are improved through optimization. We demonstrate the efficiency and scalability of the approach by reconstructing a high-quality bathymetry using sidescan data from a large sidescan survey. We compare the proposed data-driven inverse model approach of modeling a sidescan with a forward Lambertian model. We assess the quality of each reconstruction by comparing it with data constructed from a multibeam sensor.

关键词： Convolutional neural networks Bathymetry Neural networks Sonar Image reconstruction Optimization Data models Bathymetry reconstruction implicit neural representations neural networks sidescan sonar

来源：评论

学校读者我要写书评

暂无评论

DINOBot: robot Manipulation via Retrieval and Alignment with Vision Foundation Models

DINOBot: Robot Manipulation via Retrieval and Alignment with...

引用

IEEE International Conference on robotics and Automation (ICRA)

作者： Di Palo, Norman Johns, Edward Imperial Coll London Robot Learning Lab London England

ISBN: (纸本)9798350384581;9798350384574

We propose DINOBot, a novel imitation learning framework for robot manipulation, which leverages the image-level and pixel-level capabilities of features extracted from Vision Transformers trained with DINO. When interacting with a novel object, DINOBot first uses these features to retrieve the most visually similar object experienced during human demonstrations, and then uses this object to align its end-effector with the novel object to enable effective interaction. Through a series of real-world experiments on everyday tasks, we show that exploiting both the image-level and pixel-level properties of vision foundation models enables unprecedented learning efficiency and generalisation. Videos and code are available at https://***/dinobot.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark 8

BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark

引用

8th Conference on robot learning, CoRL 2024

作者： Chernyadev, Nikita Backshall, Nicholas Ma, Xiao Lu, Yunfan Seo, Younggyo James, Stephen Dyson Robot Learning Lab United Kingdom

We introduce BiGym, a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching to complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports a variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views. To validate the usability of BiGym, we thoroughly benchmark the state-of-the-art imitation learning algorithms and demo-driven reinforcement learning algorithms within the environment and discuss the future opportunities. Project website: https://***/bigym/. © 2024 Proceedings of Machine learning Research.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：