检索结果-内蒙古大学图书馆

RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service Robots Using Foundation Models for Perception and Planning

学校读者我要写书评

暂无评论

RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service...

27th RoboCup International Symposium, 2024

作者： Memmesheimer, Raphael Nogga, Jan Pätzold, Bastian Kruzhkov, Evgenii Bultmann, Simon Schreiber, Michael Bode, Jonas Karacora, Bertan Park, Juhui Savinykh, Alena Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI Lamarr Institute for Machine Learning and Artificial Intelligence and Center for Robotics University of Bonn Bonn Germany

ISBN: (纸本)9783031858581

We present the approaches and contributions of the winning team NimbRo@Home at the RoboCup@Home 2024 competition in the Open Platform League held in Eindhoven, NL. Further, we describe our hardware setup and give an overview of the results for the task stages and the final demonstration. For this year’s competition, we put a special emphasis on open-vocabulary object segmentation and grasping approaches that overcome the labeling overhead of supervised vision approaches, commonly used in RoboCup@Home. We successfully demonstrated that we can segment and grasp non-labeled objects by text descriptions. Further, we extensively employed Large Language Model (LLMs) for natural language understanding and task planning. Throughout the competition, our approaches showed robustness and generalization capabilities. A video of our performance can be found online (https://***/videos/RoboCup_2024). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Robot programming

LiLMaps: Learnable Implicit Language Maps

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kruzhkov, Evgenii Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI University of Bonn Germany Autonomous Intelligent Systems Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics and The Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

One of the current trends in robotics is to employ large language models (LLMs) to provide non-predefined command execution and natural human-robot interaction. It is useful to have an environment map together with its language representation, which can be further utilized by LLMs. Such a comprehensive scene representation enables numerous ways of interaction with the map for autonomously operating robots. In this work, we present an approach that enhances incremental implicit mapping through the integration of vision-language features. Specifically, we (i) propose a decoder optimization technique for implicit language maps which can be used when new objects appear on the scene, and (ii) address the problem of inconsistent vision-language predictions between different viewing positions. Our experiments demonstrate the effectiveness of LiLMaps and solid improvements in performance. © 2025, CC BY-NC-ND.

关键词： visual languages

Object-Centric Image to video Generation with Language Guidance

学校读者我要写书评

暂无评论

arXiv 2025年

作者： villar-Corrales, Angel Plepi, Gjergj Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Accurate and flexible world models are crucial for autonomous systems to understand their environment and predict future events. Object-centric models, with structured latent spaces, have shown promise in modeling object dynamics and interactions, but often face challenges in scaling to complex datasets and incorporating external guidance, limiting their applicability in robotics. To address these limitations, we propose TextOCVP, an object-centric model for image-to-video generation guided by textual descriptions. TextOCVP parses an observed scene into object representations, called slots, and utilizes a text-conditioned transformer predictor to forecast future object states and video frames. Our approach jointly models object dynamics and interactions while incorporating textual guidance, thus leading to accurate and controllable predictions. Our method’s structured latent space offers enhanced control over the prediction process, outperforming several image-to-video generative baselines. Additionally, we demonstrate that structured object-centric representations provide superior controllability and interpretability, facilitating the modeling of object dynamics and enabling more precise and understandable predictions. videos and code are available at https://***/TextOCVP/. © 2025, CC BY.

关键词： Digital elevation model

PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric video Prediction and Planning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： villar-Corrales, Angel Behnke, Sven Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics The Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Predicting future scene representations is a crucial task for enabling robots to understand and interact with the environment. However, most existing methods rely on video sequences and simulations with precise action annotations, limiting their ability to leverage the large amount of available unlabeled video data. To address this challenge, we propose PlaySlot, an object-centric video prediction model that infers object representations and latent actions from unlabeled video sequences. It then uses these representations to forecast future object states and video frames. PlaySlot allows to generate multiple possible futures conditioned on latent actions, which can be inferred from video dynamics, provided by a user, or generated by a learned action policy, thus enabling versatile and interpretable world modeling. Our results show that PlaySlot outperforms both stochastic and object-centric baselines for video prediction across different environments. Furthermore, we show that our inferred latent actions can be used to learn robot behaviors sample-efficiently from unlabeled video demonstrations. videos and code are available at https://***/PlaySlot/. © 2025, CC BY.

关键词： Stochastic systems

Towards Conscious Service Robots

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics the Lamarr Institute for Machine Learning and Artificial Intelligence University of Bonn Germany

Deep learning’s success in perception, natural language processing, etc. inspires hopes for advancements in autonomous robotics. However, real-world robotics face challenges like variability, high-dimensional state spaces, non-linear dependencies, and partial observability. A key issue is non-stationarity of robots, environments, and tasks, leading to performance drops with out-of-distribution data. Unlike current machine learning models, humans adapt quickly to changes and new tasks due to a cognitive architecture that enables systematic generalization and meta-cognition. Human brain’s System 1 handles routine tasks unconsciously, while System 2 manages complex tasks consciously, facilitating flexible problem-solving and self-monitoring. For robots to achieve human-like learning and reasoning, they need to integrate causal models, working memory, planning, and metacognitive processing. By incorporating human cognition insights, the next generation of service robots will handle novel situations and monitor themselves to avoid risks and mitigate errors. Copyright © 2025, The Authors. All rights reserved.

关键词： Problem solving

Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of view LiDAR Sensors

学校读者我要写书评

暂无评论

Person Segmentation and Action Classification for Multi-Chan...

IEEE/SICE International Symposium on System Integration

作者： Svetlana Seliunina Artem Otelepko Raphael Memmesheimer Sven Behnke Autonomous Intelligent Systems Group Computer Science Institute VI – Intelligent Systems and Robotics Lamarr Institute for Machine Learning and Artificial Intelligence and Center for Robotics University of Bonn

ISBN: (数字)9798331531614

ISBN: (纸本)9798331531621

关键词： Reflectivity Adaptation models Laser radar Three-dimensional displays System integration Robot sensing systems Encoding Sensor systems Sensors Safety

LIAM: Multimodal Transformer for Language Instructions, Images, Actions and Semantic Maps

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Yihao Memmesheimer, Raphael Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI University of Bonn Germany

The availability of large language models and open-vocabulary object perception methods enables more flexibility for domestic service robots. The large variability of domestic tasks can be addressed without implementing each task individually by providing the robot with a task description along with appropriate environment information. In this work, we propose LIAM — an end-to-end model that predicts action transcripts based on language, image, action, and map inputs. Language and image inputs are encoded with a CLIP backbone, for which we designed two pre-training tasks to fine-tune its weights and pre-align the latent spaces. We evaluate our method on the ALFRED dataset, a simulator-generated benchmark for domestic tasks. Our results demonstrate the importance of pre-aligning embedding spaces from different modalities and the efficacy of incorporating semantic maps. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantics

II-NVM: Enhancing Map Accuracy and Consistency With Normal Vector-Assisted Mapping

学校读者我要写书评

暂无评论

IEEE robotics and Automation Letters 2025年第6期10卷 5465-5472页

作者： Zhao, Chengwei Li, Yixuan Jian, Yina Xu, Jie Wang, Linji Ma, Yongxin Jin, Xinglai Xi'an Jiaotong University Institute of Artificial Intelligence and Robotics Xi'an710049 China Hangzhou Qisheng Intelligent Techology Company Ltd Hangzhou311217 China Columbia University in the City of New York Department of Computer Science New YorkNY10027 United States Harbin Institute of Technology State Key Laboratory of Robotics and Systems Harbin150001 China Nanyang Technological University School of Electrical and Electronic Engineering 639798 Singapore School of Electrical and Electronic Engineering Nanyang Technological University 639798 Singapore

SLAM technology plays a crucial role in indoor mapping and localization. A common challenge in indoor environments is the "double-sided mapping issue", where closely positioned walls, doors, and other surfaces are mistakenly identified as a single plane, significantly hindering map accuracy and consistency. To addressing this issue this letter introduces a SLAM approach that ensures accurate mapping using normal vector consistency. We enhance the voxel map structure to store both point cloud data and normal vector information, enabling the system to evaluate consistency during nearest neighbor searches and map updates. This process distinguishes between the front and back sides of surfaces, preventing incorrect point-to-plane constraints. Moreover, we implement an adaptive radius KD-tree search method that dynamically adjusts the search radius based on the local density of the point cloud, thereby enhancing the accuracy of normal vector calculations. To further improve real-time performance and storage efficiency, we incorporate a Least Recently Used (LRU) cache strategy, which facilitates efficient incremental updates of the voxel map. The code is released as open-source and validated in both simulated environments and real indoor scenarios. Experimental results demonstrate that this approach effectively resolves the "double-sided mapping issue" and significantly improves mapping precision. Additionally, we have developed and open-sourced the first simulation and real-world dataset specifically tailored for the "double-sided mapping issue". © 2016 IEEE.

关键词： Mapping

Enhancing Mobile Robot Navigation: A Graph Decomposition Submodule for TEA*

学校读者我要写书评

暂无评论

Enhancing Mobile Robot Navigation: A Graph Decomposition Sub...

IEEE International Conference on Autonomous Robot systems and Competitions (ICARSC)

作者： Filipe Cardoso Diogo Miguel Matos Marina Brilhante Pedro Costat Héber Sobreira Carlos Silva CRIIS - Centre of Robotics in Industry and Intelligent Systems INESC TEC - Institute for Systems and Computer Engineering Technology and Science Porto Portugal Faculty of Engineering University of Porto Porto Portugal Flowbotic Mobile Systems SA Mangualde Portugal

ISBN: (数字)9798331538606

ISBN: (纸本)9798331538613

Rising industrial complexity demands efficient mobile robots to drive automation and productivity. Effective navigation relies on perception, localization, mapping, path planning, and motion control, with path planning being key. The Time Enhanced A * (TEA *) algorithm extends A * by adding time as a dimension to resolve temporal conflicts in multi-robot coordination. However, inconsistencies in edge lengths within the graph can hinder optimal path calculation. To address this, a Graph Decomposition submodule was developed to standardize edge lengths and temporal costs. Integrated into a ROS-based fleet coordination system, this approach significantly reduces execution time and improves coordination capacity.

关键词：