检索结果-内蒙古大学图书馆

IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors Without Any Outdoor Experience

IEEE robotics AND AUTOMATION LETTERS 2024年第5期9卷 4798-4805页

作者： Truong, Joanne Zitkovich, April Chernova, Sonia Batra, Dhruv Zhang, Tingnan Tan, Jie Yu, Wenhao Georgia Inst Technol Atlanta GA 30318 USA Google DeepMind Mountain View CA 94043 USA Meta AI Menlo Pk CA 94025 USA

We present IndoorSim-to-OutdoorReal (I2O), an end-to-end learned visual navigation approach, trained solely in simulated short-range indoor environments, and demonstrate zero-shot sim-to-real transfer to the outdoors for long-range navigation on the Spot robot. Our method uses zero real-world experience (indoor or outdoor), and requires the simulator to model no predominantly-outdoor phenomenon (sloped grounds, sidewalks, etc). The key to I2O transfer is in providing the robot with additional context of the environment (i.e. a satellite map, a rough sketch of a map by a human, etc.) to guide the robot's navigation in the real-world. The provided context-maps do not need to be accurate or complete- real-world obstacles (e.g. trees, bushes, pedestrians, etc.) are not drawn on the map, and openings are not aligned with where they are in the real-world. Crucially, these inaccurate context-maps provide a hint to the robot about a route to take to the goal. We find that our method that leverages Context-Maps is able to successfully navigate over a hundred meters in novel environments, avoiding novel obstacles on its path, to a distant goal without a single collision or human intervention. In comparison, policies without the additional context fail completely. We additionally find that the Context-Map policy is surprisingly robust to noise. In the presence of significantly inaccurate maps in simulation (corrupted with 50% noise, or entirely blank maps), the policy gracefully regresses to the behavior of a policy with no context.

关键词： Robots Robot sensing systems Satellite navigation systems Indoor environment Collision avoidance Visualization Legged locomotion ai-enabled robotics vision-based navigation reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Make it Dense: Self-Supervised Geometric Scan Completion of Sparse 3D LiDAR Scans in Large Outdoor Environments

引用

IEEE robotics AND AUTOMATION LETTERS 2022年第3期7卷 8534-8541页

作者： Vizzo, Ignacio Vizzo Mersch, Benedikt Marcuzzi, Rodrigo Wiesmann, Louis Behley, Jens Stachniss, Cyrill Univ Bonn D-53115 Bonn Germany Univ Oxford Dept Engn Sci Oxford OX1 3PJ England

Mapping systems that turn sensor data into a model of the environment are standard components in mobile robotics. Outdoor robots are often equipped with 3D LiDAR sensors to obtain accurate range measurements at a high frame rate. The price for a robotic LiDAR sensor scales roughly linearly with the number of beams and thus the vertical resolution of the scanner. In general, the cheaper the sensors, the sparser the point cloud. In this letter, we address the problem of building dense models from sparse range data. Instead of requiring the vehicle to move slowly through the environment or to traverse the scene multiple times to cover the space densely, we investigate geometric scan completion through a learning-based approach. We revisit the traditional volumetric fusion pipeline based on truncated signed distance fields (TSDF) and propose a neural network to aid the 3D reconstruction on a frame-to-frame basis by completing each scan towards a dense TSDF volume. We propose a geometric scan completion network that is trained in a self-supervised fashion without labels. Our experiments illustrate that such frame-wise completion leads to maps that are on-par or even better compared to maps generated using a higher resolution LiDAR sensor. We additionally show that our system can be used to improve the performance of SLAM systems.

关键词： Mapping ai-enabled robotics

来源：评论

学校读者我要写书评

暂无评论

3D Detection and Tracking of a Moving Object by an Autonomous Underwater Vehicle With a Multibeam Imaging Sonar: Toward Continuous Observation of Marine Life

引用

IEEE robotics AND AUTOMATION LETTERS 2024年第4期9卷 3037-3044页

作者： Chun, Sehwa Kawamura, Chihaya Ohkuma, Kenji Maki, Toshihiro Univ Tokyo Inst Ind Sci Tokyo 1538505 Japan

Biologging has been the prevalent method for studying the ecology and behavior of aquatic life. However, it has problems of operational inconvenience and stress on organisms. Recently, research towards remote tracking using Autonomous Underwater Vehicles (AUVs) has been gaining attention, particularly those equipped with multibeam imaging sonar (MBS) less affected by underwater light and turbidity. However, existing studies have focused mainly on 2D detection and tracking due to the inherent limitations of MBS. In this letter, we propose a novel method for 3D detection and tracking of a moving target for marine life surveys, utilizing the tilt control of MBS on AUVs. YOLOv3 is employed to identify the target's position in sonar imagery. Incorporation of the particle filter algorithm enhances position an estimation accuracy and enables a estimation of the target's state, velocity and orientation as well. 3D tracking method is introduced based on the estimated states. This method has been validated through tank experiments with a hovering-type AUV and a life-sized sea turtle replica, successfully achieving 3D tracking of the target moving horizontally and vertically. The approach demonstrated a position estimation with an RMSE of 0.23 m and velocity and orientation within the error of 0.022 m/s and 16.59 degrees, respectively.

关键词： Marine robotics object detection segmentation and categorization ai-enabled robotics autonomous underwater vehicle target tracking

来源：评论

学校读者我要写书评

暂无评论

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents

引用

IEEE robotics AND AUTOMATION LETTERS 2024年第2期9卷 1059-1066页

作者： Park, Jeongeun Lim, Seungwon Lee, Joonhyung Park, Sangbeom Chang, Minsuk Yu, Youngjae Choi, Sungjoon Korea Univ Dept Artificial Intelligence Seoul 02841 South Korea Yonsei Univ Dept Artificial Intelligence Seoul 03722 South Korea Google Res Seattle WA 98103 USA

In this letter, we focus on inferring whether the given user command is clear, ambiguous, or infeasible in the context of interactive robotic agents utilizing large language models (LLMs). To tackle this problem, we first present an uncertainty estimation method for LLMs to classify whether the command is certain (i.e., clear) or not (i.e., ambiguous or infeasible). Once the command is classified as uncertain, we further distinguish it between ambiguous or infeasible commands leveraging LLMs with situational aware context prompts. For ambiguous commands, we disambiguate the command by interacting with users via question generation with LLMs. We believe that proper recognition of the given commands could lead to a decrease in malfunction and undesired actions of the robot, enhancing the reliability of interactive robot agents. We present a dataset for robotic situational awareness consisting of pairs of high-level commands, scene descriptions, and labels of command type (i.e., clear, ambiguous, or infeasible). We validate the proposed method on the collected dataset and pick-and-place tabletop simulation environment. Finally, we demonstrate the proposed approach in real-world human-robot interaction experiments.

关键词： ai-enabled robotics human-centered robotics human-centered automation

来源：评论

学校读者我要写书评

暂无评论

XBG: End-to-End Imitation Learning for Autonomous Behaviour in Human-Robot Interaction and Collaboration

引用

IEEE robotics AND AUTOMATION LETTERS 2024年第12期9卷 11617-11624页

作者： Cardenas-Perez, Carlos Romualdi, Giulio Elobaid, Mohamed Dafarra, Stefano L'Erario, Giuseppe Traversaro, Silvio Morerio, Pietro Del Bue, Alessio Pucci, Daniele Ist Italiano Tecnol Artificial & Mech Intelligence I-16163 Genoa Italy Univ Manchester Dept Comp Sci Manchester M13 9PL England Ist Italiano Tecnol Pattern Anal & Comp Vis I-16152 Genoa Italy

This letter presents XBG (eXteroceptive Behaviour Generation), a multimodal end-to-end Imitation Learning (IL) system for whole-body autonomous humanoid robots used in real-world Human-Robot Interaction (HRI) scenarios. The main contribution is an architecture for learning HRI behaviours using a data-driven approach. A diverse dataset is collected via teleoperation, covering multiple HRI scenarios, such as handshaking, handwaving, payload reception, walking, and walking with a payload. After synchronizing, filtering, and transforming the data, we show how to train the presented Deep Neural Networks (DNN), integrating exteroceptive and proprioceptive information to help the robot understand both its environment and its actions. The robot takes in sequences of images (RGB and depth) and joints state information to react accordingly. By fusing multimodal signals over time, the model enables autonomous capabilities in a robotic platform. The models are evaluated based on the success rates in the mentioned HRI scenarios and they are deployed on the ergoCub humanoid robot. XBG achieves success rates between 60% and 100% even when tested in unseen environments.

关键词： Robots Robot sensing systems Humanoid robots Human-robot interaction Feature extraction Imitation learning Transformers Legged locomotion Solid modeling Sensors ai-enabled robotics humanoid robot systems imitation learning learning from demonstration

来源：评论

学校读者我要写书评

暂无评论

Long-Term Human Trajectory Prediction Using 3D Dynamic Scene Graphs

引用

IEEE robotics AND AUTOMATION LETTERS 2024年第12期9卷 10978-10985页

作者： Gorlo, Nicolas Schmid, Lukas Carlone, Luca MIT MIT SPARK Lab Cambridge MA 02139 USA

We present a novel approach for long-term human trajectory prediction in indoor human-centric environments, which is essential for long-horizon robot planning in these environments. State-of-the-art human trajectory prediction methods are limited by their focus on collision avoidance and short-term planning, and their inability to model complex interactions of humans with the environment. In contrast, our approach overcomes these limitations by predicting sequences of human interactions with the environment and using this information to guide trajectory predictions over a horizon of up to 60s . We leverage Large Language Models (LLMs) to predict interactions with the environment by conditioning the LLM prediction on rich contextual information about the scene. This information is given as a 3D Dynamic Scene Graph that encodes the geometry, semantics, and traversability of the environment into a hierarchical representation. We then ground these interaction sequences into multi-modal spatio-temporal distributions over human positions using a probabilistic approach based on continuous-time Markov Chains. To evaluate our approach, we introduce a new semi-synthetic dataset of long-term human trajectories in complex indoor environments, which also includes annotations of human-object interactions. We show in thorough experimental evaluations that our approach achieves a 54% lower average negative log-likelihood and a 26.5% lower Best-of-20 displacement error compared to the best non-privileged (i.e., evaluated in a zero-shot fashion on the dataset) baselines for a time horizon of 60 s .

关键词： Trajectory Probabilistic logic Three-dimensional displays Predictive models Indoor environment Planning Cognition Annotations Service robots Legged locomotion ai-enabled robotics human-centered robotics service robotics datasets for human motion modeling and simulating humans

来源：评论

学校读者我要写书评

暂无评论

Can I Pour Into It? Robot Imagining Open Containability Affordance of Previously Unseen Objects via Physical Simulations

引用

IEEE robotics AND AUTOMATION LETTERS 2021年第1期6卷 271-278页

作者： Wu, Hongtao Chirikjian, Gregory S. Johns Hopkins Univ Lab Computat Sensing & Robot Baltimore MD 21218 USA Natl Univ Singapore Dept Mech Engn Singapore 117575 Singapore

Open containers, i.e., containers without covers, are an important and ubiquitous class of objects in human life. In this letter, we propose a novel method for robots to "imagine" the open containability affordance of a previously unseen object via physical simulations. The robot autonomously scans the object with an RGB-D camera. The scanned 3D model is used for open containability imagination which quantifies the open containability affordance by physically simulating dropping particles onto the object and counting how many particles are retained in it. This quantification is used for open-container vs. non-open-container binary classification (hereafter referred to as open container classification). If the object is classified as an open container, the robot further imagines pouring into the object, again using physical simulations, to obtain the pouring position and orientation for real robot autonomous pouring. We evaluate our method on open container classification and autonomous pouring of granular material on a dataset containing 130 previously unseen objects with 57 object categories. Although our proposed method uses only 11 objects for simulation calibration, its open container classification aligns well with human judgements. In addition, our method endows the robot with the capability to autonomously pour into the 55 containers in the dataset with a very high success rate. We also compare to a deep learning method. Results show that our method achieves the same performance as the deep learning method on open container classification and outperforms it on autonomous pouring. Moreover, our method is fully explainable.

关键词： Robots Containers Cognition Solid modeling Robot sensing systems Three-dimensional displays Physics ai-based methods ai-enabled robotics object affordances robot imagination simulation and animation

来源：评论

学校读者我要写书评

暂无评论

Safe Planning in Dynamic Environments Using Conformal Prediction

引用

IEEE robotics AND AUTOMATION LETTERS 2023年第8期8卷 5116-5123页

作者： Lindemann, Lars Cleaveland, Matthew Shim, Gihyun Pappas, George J. Univ Southern Calif Thomas Lord Dept Comp Sci Los Angeles CA 90089 USA Univ Penn Dept Elect & Syst Engn Philadelphia PA 19104 USA

We propose a framework for planning in unknown dynamic environments with probabilistic safety guarantees using conformal prediction. Particularly, we design a model predictive controller (MPC) that uses i) trajectory predictions of the dynamic environment, and ii) prediction regions quantifying the uncertainty of the predictions. To obtain prediction regions, we use conformal prediction, a statistical tool for uncertainty quantification, that requires availability of offline trajectory data - a reasonable assumption in many applications such as autonomous driving. The prediction regions are valid, i.e., they hold with a user-defined probability, so that the MPC is provably safe. We illustrate the results in the self-driving car simulator CARLA at a pedestrian-filled intersection. The strength of our approach is compatibility with state of the art trajectory predictors, e.g., RNNs and LSTMs, while making no assumptions on the underlying trajectory-generating distribution. To the best of our knowledge, these are the first results that provide valid safety guarantees in such a setting.

关键词： Planning under uncertainty robot safety motion planning autonomous agents ai-enabled robotics

来源：评论

学校读者我要写书评

暂无评论

Safe POMDP Online Planning Among Dynamic Agents via Adaptive Conformal Prediction

引用

IEEE robotics AND AUTOMATION LETTERS 2024年第11期9卷 9946-9953页

作者： Sheng, Shili Yu, Pian Parker, David Kwiatkowska, Marta Feng, Lu Univ Virginia Sch Engn & Appl Sci Charlottesville VA 22904 USA Univ Oxford Dept Comp Sci Oxford OX1 2JD England

Online planning for partially observable Markov decision processes (POMDPs) provides efficient techniques for robot decision-making under uncertainty. However, existing methods fall short of preventing safety violations in dynamic environments. This letter presents a novel safe POMDP online planning approach that maximizes expected returns while providing probabilistic safety guarantees amidst environments populated by multiple dynamic agents. Our approach utilizes data-driven trajectory prediction models of dynamic agents and applies Adaptive Conformal Prediction (ACP) to quantify the uncertainties in these predictions. Leveraging the obtained ACP-based trajectory predictions, our approach constructs safety shields on-the-fly to prevent unsafe actions within POMDP online planning. Through experimental evaluation in various dynamic environments using real-world pedestrian trajectory data, the proposed approach has been shown to effectively maintain probabilistic safety guarantees while accommodating up to hundreds of dynamic agents.

关键词： Safety Planning Robots Trajectory Uncertainty Pedestrians Heuristic algorithms Formal methods in robotics and automation planning under uncertainty robot safety ai-enabled robotics

来源：评论

学校读者我要写书评

暂无评论

3D Part Assembly Generation With Instance Encoded Transformer

引用

IEEE robotics AND AUTOMATION LETTERS 2022年第4期7卷 9051-9058页

作者： Zhang, Rufeng Kong, Tao Wang, Weihao Han, Xuan You, Mingyu Tongji Univ Coll Elect & Informat Engn Frontiers Sci Ctr Intelligent Autonomous Syst Shanghai 201800 Peoples R China ByteDance AI Lab Beijing 100089 Peoples R China

It is desirable to enable robots capable of automatic assembly. Structural understanding of object parts plays a crucial role in this task yet remains relatively unexplored. In this letter, we focus on the setting of furniture assembly from a complete set of part geometries, which is essentially a 6-DoF part pose estimation problem. We propose a multi-layer transformer-based framework that involves geometric and relational reasoning between parts to update the part poses iteratively. We carefully design a unique instance encoding to solve the ambiguity between geometrically-similar parts so that all parts can be distinguished. In addition to assembling from scratch, we extend our framework to a new task called in-process part assembly. Analogous to furniture maintenance, it requires robots to continue with unfinished products and assemble the remaining parts into appropriate positions. Our method achieves far more than 10% improvements over the current state-of-the-art in multiple metrics on the public PartNet dataset. Extensive experiments and quantitative comparisons demonstrate the effectiveness of the proposed framework.

关键词： Assembly deep learning for visual perception ai-enabled robotics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：