Large Language Models (LLMs) have the potential to catalyze a paradigm shift in end-user robot programming-moving from the conventional process of user specifying programming logic to an iterative, collaborative proce...
详细信息
ISBN:
(数字)9798400703225
ISBN:
(纸本)9798400703225
Large Language Models (LLMs) have the potential to catalyze a paradigm shift in end-user robot programming-moving from the conventional process of user specifying programming logic to an iterative, collaborative process in which the user *** desired program outcomes while LLM produces detailed ***. We introduce a novel integrated development system, Alchemist, that leverages LLMs to empower end-users in creating, testing, and running robot programs using natural language inputs, aiming to reduce the required knowledge for developing robot applications. We present a detailed examination of our system design and provide an exploratory study involving true end-users to assess capabilities, usability, and limitations of our system. Through the design, development, and evaluation of our system, we derive a set of lessons learned from the use of LLMs in robot programming. We discuss how LLMs may be the next frontier for democratizing end-user development of robot applications.
The multi-agent trajectory planning problem is a difficult problem in robotics due to its computational complexity and real-world environment complexity with uncertainty, non-linearity, and real-time requirements. Man...
详细信息
ISBN:
(纸本)9798350377712;9798350377705
The multi-agent trajectory planning problem is a difficult problem in robotics due to its computational complexity and real-world environment complexity with uncertainty, non-linearity, and real-time requirements. Many existing solutions are either search-based or optimization-based approaches with simplified assumptions of environment, limited planning speed, and limited scalability in the number of agents. In this work, we first attempt to reformulate single-agent and multi-agent trajectory planning problems as query problems over an implicit neural representation of trajectories. We formulate such implicit representations as Neural Trajectory Models (NTM) which can be queried to generate nearly optimal trajectory in complex environments. We conduct experiments in simulation environments and demonstrate that NTM achieve (1) sub-millisecond planning time using GPUs, (2) almost avoiding all collisions, and (3) generating almost shortest paths. We also demonstrate that the same NTM framework can also be used for refining low-quality and conflicting multi-agent trajectories into nearly optimal solutions efficiently. (Open source code is available at https://***/laser2099/neural-trajectory-model)
Equipping autonomous robots with the ability to navigate safely and efficiently around humans is a crucial step toward achieving trusted robot autonomy. However, generating robot plans while ensuring safety in dynamic...
详细信息
ISBN:
(纸本)9798350377712;9798350377705
Equipping autonomous robots with the ability to navigate safely and efficiently around humans is a crucial step toward achieving trusted robot autonomy. However, generating robot plans while ensuring safety in dynamic multi-agent environments remains a key challenge. Building upon recent work on leveraging deep generative models for robot planning in static environments, this paper proposes CoBL-Diffusion, a novel diffusion-based safe robot planner for dynamic environments. CoBL-Diffusion uses Control Barrier and Lyapunov functions to guide the denoising process of a diffusion model, iteratively refining the robot control sequence to satisfy the safety and stability constraints. We demonstrate the effectiveness of CoBL-Diffusion using two settings: a synthetic single-agent environment and a real-world pedestrian dataset. Our results show that CoBL-Diffusion generates smooth trajectories that enable the robot to reach goal locations while maintaining a low collision rate with dynamic obstacles.
The household rearrangement task involves spotting misplaced objects in a scene and accommodate them with proper places. It depends both on common-sense knowledge on the objective side and human user preference on the...
详细信息
ISBN:
(纸本)9798400711312
The household rearrangement task involves spotting misplaced objects in a scene and accommodate them with proper places. It depends both on common-sense knowledge on the objective side and human user preference on the subjective side. In achieving such a task, we propose to mine object functionality with user preference alignment directly from the scene itself, without relying on human intervention. To do so, we work with scene graph representation and propose LLM-enhanced scene graph learning which transforms the input scene graph into an affordance-enhanced graph (AEG) with information-enhanced nodes and newly discovered edges (relations). In AEG, the nodes corresponding to the receptacle objects are augmented with context-induced affordance which encodes what kind of carriable objects can be placed on it. New edges are discovered with newly discovered non-local relations. With AEG, we perform task planning for scene rearrangement by detecting misplaced carriables and determining a proper placement for each of them. We test our method by implementing a tiding robot in simulator and perform evaluation on a new benchmark we build. Extensive evaluations demonstrate that our method achieves state-of-the-art performance in misplacement detection and the following rearrangement planning.
Object grasping is a complex task that requires high environmental awareness. While vision generally provides highly detailed environmental information, light changes, object transparency, camera resolution, and other...
详细信息
ISBN:
(纸本)9798350371635;9798350371628
Object grasping is a complex task that requires high environmental awareness. While vision generally provides highly detailed environmental information, light changes, object transparency, camera resolution, and other factors such as occlusion and clutter affect its perception of object pose. Due to these limitations, there may be some deviation between the estimated and actual object pose in unstructured environments. The use of compliant tactile sensors relaxes the requirement of strict finger position planning while providing essential information regarding contact with the target object. Therefore, under positional uncertainty, the robotic system may use compliant tactile sensors to perform multiple attempts before a successful grasp. In the present paper, we investigate using reinforcement learning and compliant tactile sensors to provide adaptive grasping under pose uncertainty. Here, we identify a policy that models an object position estimation error while minimizing the exploratory sensor contact before obtaining a grasp. Our method was able to perform a successful grasp while reducing the number of attempts from an average of five to an average of two per episode.
A unique selling point for cyber-physical production system manufacturers becomes the easy with which machines and cells can be adapted to new products and production processes. Adaptations, however, are often done by...
详细信息
ISBN:
(纸本)9781450393171
A unique selling point for cyber-physical production system manufacturers becomes the easy with which machines and cells can be adapted to new products and production processes. Adaptations, however, are often done by domain experts without in-depth programming know-how. We investigate in this paper, the implications of using a planning-based approach for using a domain expert's knowledge to control the sequences of a robot and injection molding machine (IMM). We find that current engineering support is insufficient to address testing, understanding, and change impact assessment concerns during the evolution of a PDDL/HDDL domain specification.
Automatic visual quality inspection is pivotal in both computer vision and robotics. It plays a crucial role in manufacturing, where robotic systems are increasingly employed to enhance the speed and efficiency of vis...
详细信息
ISBN:
(纸本)9798350377712;9798350377705
Automatic visual quality inspection is pivotal in both computer vision and robotics. It plays a crucial role in manufacturing, where robotic systems are increasingly employed to enhance the speed and efficiency of visual quality assessments. Several inspection planning methodologies have been developed;however, they often address the inspection challenge from a singular perspective of robotics or computer vision. This work introduces a comprehensive approach that synergistically integrates principles from both domains. We present an innovative algorithm designed to generate optimal inspection poses by considering the interplay between the inspected object's geometry and the kinematics of the robotic setup used for inspection. This is accomplished by taking advantage of the concept of visibility. The effectiveness of our algorithm is demonstrated through simulations and experiments, revealing complete coverage for diverse geometries and materials with a small number of inspection poses. Moreover, we benchmark our framework against box constraints and workspace sampling techniques to generate feasible inspection poses. The results indicate superior performance in achieving extensive coverage and reducing the number of required optimal inspection poses, enhancing the overall inspection process.
Legged robot proved their capability to cross complex terrain in recent research, yet the autonomy of robots on discrete terrain still needs to be enhanced since it requires a full stack framework. This paper introduc...
详细信息
ISBN:
(纸本)9798350377712;9798350377705
Legged robot proved their capability to cross complex terrain in recent research, yet the autonomy of robots on discrete terrain still needs to be enhanced since it requires a full stack framework. This paper introduces a real-time motion and foothold planning framework tailored for legged robots navigating uneven terrains, such as stepping stones. Our approach addresses the critical challenges of determining feasible global paths and local footholds to enhance autonomous mobility across complex landscapes. By using a sampling-based global path planner integrated with terrain segmentation and the robot's kinematic model, our framework swiftly generates viable navigation paths. Concurrently, it utilizes a Mixed Integer programming (MIP) methodology for real-time foothold optimization, ensuring the robot's stability and safety through dynamic terrain interaction. Finally, an execution layer including Model Predictive Control (MPC) and Whole-Body Control (WBC) generates the robots' motion. Simulation and real-world experiments demonstrate that our framework improves legged robots' adaptability on discrete terrains.
Recent planning methods based on Large Language Models typically employ the In-Context Learning paradigm. Complex long-horizon planning tasks require more context(including instructions and demonstrations) to guarante...
详细信息
ISBN:
(纸本)9798350384581;9798350384574
Recent planning methods based on Large Language Models typically employ the In-Context Learning paradigm. Complex long-horizon planning tasks require more context(including instructions and demonstrations) to guarantee that the generated plan can be executed correctly. However, in such conditions, LLMs may overlook(unfaithful) the rules in the given context, resulting in the generated plans being invalid or even leading to dangerous actions. In this paper, we investigate the faithfulness of LLMs for complex long-horizon tasks. Inspired by human intelligence, we introduce a novel framework named FLTRNN. FLTRNN employs a language-based RNN structure to integrate task decomposition and memory management into LLM planning inference, which could effectively improve the faithfulness of LLMs and make the planner more reliable. We conducted experiments in VirtualHome household tasks. Results show that our model significantly improves faithfulness and success rates for complex long-horizon tasks. Website at https://***/***/
Mobile robots frequently navigate on roadmaps, i.e., graphs where edges represent safe motions, in applications such as healthcare, hospitality, and warehouse automation. Often the environment is quasi-static, i.e., i...
详细信息
ISBN:
(纸本)9798350377712;9798350377705
Mobile robots frequently navigate on roadmaps, i.e., graphs where edges represent safe motions, in applications such as healthcare, hospitality, and warehouse automation. Often the environment is quasi-static, i.e., it is sufficient to construct a roadmap once and then use it for any future planning queries. Roadmaps are typically used with graph search algorithm to find feasible paths for the robots. Therefore, the roadmap should be well-connected, and graph searches should produce near-optimal solutions with short solution paths while simultaneously be computationally efficient to execute queries quickly. We propose a new method to construct roadmaps based on the Gray-Scott reaction diffusion system and Delaunay triangulation. Our approach, GSRM, produces roadmaps with evenly distributed vertices and edges that are well-connected even in environments with challenging narrow passages. Empirically, we compare to classical roadmaps generated by 8-connected grids, probabilistic roadmaps (PRM, SPARS2), and optimized roadmap graphs (ORM). Our results show that GSRM consistently produces superior roadmaps that are well-connected, have high query efficiency, and result in short solution paths.
暂无评论