Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, rolling sushi, and making pottery). Howev...
详细信息
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, rolling sushi, and making pottery). However, due to the high degrees of freedom of elasto-plastic objects, significant challenges exist in virtually every aspect of the robotic manipulation pipeline, for example, representing the states, modeling the dynamics, and synthesizing the control signals. We propose to tackle these challenges by employing a particle-based representation for elasto-plastic objects in a model-based planning framework. Our system, RoboCraft, only assumes access to raw RGBD visual observations. It transforms the sensory data into particles and learns a particle-based dynamics model using graph neural networks (GNNs) to capture the structure of the underlying system. The learned model can then be coupled with model predictive control (MPC) algorithms to plan the robot's behavior. We show through experiments that with just 10 min of real-world robot interaction data, our robot can learn a dynamics model that can be used to synthesize control signals to deform elasto-plastic objects into various complex target shapes, including shapes that the robot has never encountered before. We perform systematic evaluations in both simulation and the real world to demonstrate the robot's manipulation capabilities.
The application of artificial intelligence (AI) and robotics in extreme environments, is crucial for addressing complex challenges and performing high risk tasks. We highlight the importance of multi-modal sensor redu...
详细信息
ISBN:
(纸本)9798350364200;9798350364194
The application of artificial intelligence (AI) and robotics in extreme environments, is crucial for addressing complex challenges and performing high risk tasks. We highlight the importance of multi-modal sensor redundancy to ensure system reliability and accuracy despite sensor failures caused by harsh environmental conditions. We propose design considerations for sensors in extreme environments, emphasizing both the hardware and software design. One method is the non-contact heart rate and temperature monitoring using RGB visible and infrared cameras. This method addresses the limitations of traditional visible light sensors under complex illumination conditions, enhancing data reliability through advanced data fusion techniques. Furthermore, we propose a panoramic sensor lens design with a 270-degree view for comprehensive environmental perception, reducing mechanical vulnerabilities. These designs demonstrate the effectiveness of combining infrared and visible light sensors for improved environmental perception and physiological monitoring.
Vision-based grasp estimation is an essential part of robotic manipulation tasks in the real world. Existing planar grasp estimation algorithms have been demonstrated to work well in relatively simple scenes. However,...
详细信息
ISBN:
(纸本)9798350364200;9798350364194
Vision-based grasp estimation is an essential part of robotic manipulation tasks in the real world. Existing planar grasp estimation algorithms have been demonstrated to work well in relatively simple scenes. However, when it comes to complex scenes, such as cluttered scenes with messy backgrounds and moving objects, the algorithms from previous works are prone to generate inaccurate and unstable grasping contact points. In this work, we first study the existing planar grasp estimation algorithms and analyze the related challenges in complex scenes. Secondly, we design a Pixel-wise Efficient Grasp Generation Network (PEGG-Net) to tackle the problem of grasping in complex scenes. PEGG-Net can achieve improved state-of-the-art performance on the Cornell dataset (98.9%) and secondbest performance on the Jacquard dataset (93.8%), outperforming other existing algorithms without the introduction of complex structures. Thirdly, PEGG-Net could operate in a closed-loop manner to add robustness in dynamic environments using position-based visual servoing (PBVS). Finally, we conduct real-world experiments on static, dynamic, and cluttered objects in different complex scenes. The results show that our proposed network achieves a high success rate in grasping irregular objects, household objects, and workshop tools. To benefit the community, our trained model and supplementary materials are available at https://***/HZWang96/PEGG-Net.
Addressing the diverse requirements of campus road condition inspection tasks in terms of map accuracy, detail, and hierarchical structure, this paper proposes a layered road scene mapping approach based on key vertic...
详细信息
ISBN:
(纸本)9798350364200;9798350364194
Addressing the diverse requirements of campus road condition inspection tasks in terms of map accuracy, detail, and hierarchical structure, this paper proposes a layered road scene mapping approach based on key vertices. Initially, by introducing prior information and elevation data, a global topological map with elevation is constructed based on a two-dimensional topological map, thereby reflecting the actual conditions of campus roads more precisely and comprehensively. Subsequently, the mapping of local regions at key vertices was further conducted. A local mapping method based on the CAUnet model was proposed, which integrates real-time semantic information, depth maps, and an efficient octree structure, and improves the real-time performance of map construction but also greatly enhances the map's ability to retain and distinguish environmental information and obstacles. The layered map provides solid and reliable fundamental data support for subsequent tasks such as robot navigation and path planning, potentially further enhancing the efficiency and accuracy of campus road condition inspections.
Sensors are routinely mounted on robots to acquire various forms of measurements in spatio-temporal fields. Locating features within these fields and reconstruction (mapping) of the dense fields can be challenging in ...
详细信息
Sensors are routinely mounted on robots to acquire various forms of measurements in spatio-temporal fields. Locating features within these fields and reconstruction (mapping) of the dense fields can be challenging in resource-constrained situations, such as when trying to locate the source of a gas leak from a small number of measurements. In such cases, a model of the underlying complex dynamics can be exploited to discover informative paths within the field. We use a fluid simulator as a model, to guide inference for the location of a gas leak. We perform localization via minimization of the discrepancy between observed measurements and gas concentrations predicted by the simulator. Our method is able to account for dynamically varying parameters of wind flow (e.g., direction and strength), and its effects on the observed distribution of gas. We develop algorithms for off-line inference as well as for on-line path discovery via active sensing. We demonstrate the efficiency, accuracy and versatility of our algorithm using experiments with a physical robot conducted in outdoor environments. We deploy an unmanned air vehicle (UAV) mounted with a CO2 sensor to automatically seek out a gas cylinder emitting CO2 via a nozzle. We evaluate the accuracy of our algorithm by measuring the error in the inferred location of the nozzle, based on which we show that our proposed approach is competitive with respect to state of the art baselines.
暂无评论