检索结果-内蒙古大学图书馆

Open-World Semantic Segmentation Including Class Similarity

学校读者我要写书评

暂无评论

Open-World Semantic Segmentation Including Class Similarity

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Matteo Sodano Federico Magistri Lucas Nunes Jens Behley Cyrill Stachniss Center for Robotics University of Bonn Lamarr Institute for Machine Learning and Artificial Intelligence

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Interpreting camera data is key for autonomously acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world se-mantic segmentation, i.e., the variant of interpreting image data in which objects occur that have not been seen during training. We propose a novel approach that performs accu-rate closed-world semantic segmentation and, at the same time, can identify new categories without requiring any ad-ditional training data. Our approach 1 1 Code: https://***/PRBonn/ContMAV additionally provides a similarity measure for every newly discovered class in an image to a known category, which can be useful information in downstream tasks such as planning or mapping. Through extensive experiments, we show that our model achieves state-of-the-art results on classes known from training data as well as for anomaly segmentation and can distinguish between different unknown classes.

关键词： Training Computer vision Semantic segmentation machine vision Training data Computer architecture Data models

Int-HRL: towards intention-based hierarchical reinforcement learning

学校读者我要写书评

暂无评论

Neural Computing and Applications 2024年 1-12页

作者： Penzkofer, Anna Schaefer, Simon Strohm, Florian Bâce, Mihai Leutenegger, Stefan Bulling, Andreas Institute for Visualisation and Interactive Systems University of Stuttgart Pfaffenwaldring 5A Stuttgart70569 Germany Machine Learning for Robotics Technical University of Munich Boltzmannstrasse 3 Munich85748 Germany Department of Computer Science KU Leuven box 2600 Andreas Vesaliusstraat 13 Leuven3000 Belgium

While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e. the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma’s Revenge–one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to an HRL agent that is significantly more sample efficient than previous methods. © The Author(s) 2024.

关键词： Deep reinforcement learning

3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sodano, Matteo Magistri, Federico Marks, Elias Hosn, Fares Zurbayev, Aibek Marcuzzi, Rodrigo Malladi, Meher V.R. Behley, Jens Stachniss, Cyrill Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Crop yield estimation is a relevant problem in agriculture, because an accurate crop yield estimate can support farmers’ decisions on harvesting or precision intervention. Robots can help to automate this process. To do so, they need to be able to perceive the surrounding environment to identify target objects. In this paper, we introduce a novel approach to address the problem of hierarchical panoptic segmentation of apple orchards on 3D data from different sensors. Our approach is able to simultaneously provide semantic segmentation, instance segmentation of trunks and fruits, and instance segmentation of plants (a single trunk with its fruits). This allows us to identify relevant information such as individual plants, fruits, and trunks, and capture the relationship among them, such as precisely estimate the number of fruits associated to each tree in an orchard. Additionally, to efficiently evaluate our approach for hierarchical panoptic segmentation, we provide a dataset designed specifically for this task. Our dataset is recorded in Bonn in a real apple orchard with a variety of sensors, spanning from a terrestrial laser scanner to a RGB-D camera mounted on different robotic platforms. The experiments show that our approach surpasses state-of-the-art approaches in 3D panoptic segmentation in the agricultural domain, while also providing full hierarchical panoptic segmentation. Our dataset has been made publicly available at https://***/data/hops/. We will provide the open-source implementation of our approach and public competiton for hierarchical panoptic segmentation on the hidden test sets upon paper acceptance. Copyright © 2025, The Authors. All rights reserved.

关键词： Fruits

Towards Generating Realistic 3D Semantic Training Data for Autonomous Driving

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Nunes, Lucas Marcuzzi, Rodrigo Behley, Jens Stachniss, Cyrill Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Semantic scene understanding is crucial for robotics and computer vision applications. In autonomous driving, 3D semantic segmentation plays an important role for enabling safe navigation. Despite significant advances in the field, the complexity of collecting and annotating 3D data is a bottleneck in this developments. To overcome that data annotation limitation, synthetic simulated data has been used to generate annotated data on demand. There is still however a domain gap between real and simulated data. More recently, diffusion models have been in the spotlight, enabling close-to-real data synthesis. Those generative models have been recently applied to the 3D data domain for generating scene-scale data with semantic annotations. Still, those methods either rely on image projection or decoupled models trained with different resolutions in a coarse-to-fine manner. Such intermediary representations impact the generated data quality due to errors added in those transformations. In this work, we propose a novel approach able to generate 3D semantic scene-scale data without relying on any projection or decoupled trained multi-resolution models, achieving more realistic semantic scene data generation compared to previous state-of-the-art methods. Besides improving 3D semantic scene-scale data synthesis, we thoroughly evaluate the use of the synthetic scene samples as labeled data to train a semantic segmentation network. In our experiments, we show that using the synthetic annotated data generated by our method as training data together with the real semantic segmentation labels, leads to an improvement in the semantic segmentation model performance. Our results show the potential of generated scene-scale point clouds to generate more training data to extend existing datasets, reducing the data annotation effort. Our code is available at https://***/PRBonn/3DiSS. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantic Segmentation

学校读者我要写书评

暂无评论

Editorial

Computational Intelligence 2023年第4期39卷 530-531页

作者： Er, Meng Joo Ju, Zhaojie Ferrein, Alexander Changjiang Scholar Distinguished Director of Institute of Artificial Intelligence and Marine Robotics College of Marine Electrical Engineering Dalian Maritime University Dalian China Reader in Machine Learning and Robotics School of Computing University of Portsmouth Portsmouth United Kingdom Department of Electrical Engineering and Information Technology Mobile Autonomous Systems & Cognitive Robotics Institute Robotics and Foundations of Computer Science FH Aachen - University of Applied Sciences Aachen Germany

Object-Centric Image to Video Generation with Language Guidance

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Villar-Corrales, Angel Plepi, Gjergj Behnke, Sven Autonomous Intelligent Systems Computer Science Institute VI – Intelligent Systems and Robotics Center for Robotics Lamarr Institute for Machine Learning and Artificial Intelligence Germany

Accurate and flexible world models are crucial for autonomous systems to understand their environment and predict future events. Object-centric models, with structured latent spaces, have shown promise in modeling object dynamics and interactions, but often face challenges in scaling to complex datasets and incorporating external guidance, limiting their applicability in robotics. To address these limitations, we propose TextOCVP, an object-centric model for image-to-video generation guided by textual descriptions. TextOCVP parses an observed scene into object representations, called slots, and utilizes a text-conditioned transformer predictor to forecast future object states and video frames. Our approach jointly models object dynamics and interactions while incorporating textual guidance, thus leading to accurate and controllable predictions. Our method’s structured latent space offers enhanced control over the prediction process, outperforming several image-to-video generative baselines. Additionally, we demonstrate that structured object-centric representations provide superior controllability and interpretability, facilitating the modeling of object dynamics and enabling more precise and understandable predictions. Videos and code are available at https://***/TextOCVP/. © 2025, CC BY.

关键词： Digital elevation model

Efficient and Accurate Transformer-Based 3D Shape Completion and Reconstruction of Fruits for Agricultural Robots

学校读者我要写书评

暂无评论

Efficient and Accurate Transformer-Based 3D Shape Completion...

IEEE International Conference on robotics and Automation (ICRA)

作者： Federico Magistri Rodrigo Marcuzzi Elias Marks Matteo Sodano Jens Behley Cyrill Stachniss Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

Robots that operate in agricultural environments need a robust perception system that can deal with occlusions, which are naturally present in agricultural scenarios. In this paper, we address the problem of estimating 3D shapes of fruits when only partial observations are available. Generally speaking, such a shape completion can be realized by exploiting prior knowledge about the geometry of the fruit. This is typically done by template matching using traditional optimization algorithms, which are slow but accurate, or by encoding such knowledge into the weights of a neural network, leading to faster but often less accurate estimates. Our approach combines the best of both worlds. It exploits the benefit of having a template representing our object of interest with the advantages of using a neural network to learn how to deform a template. Our experimental evaluation demonstrates that our approach yields accurate estimation at a competitively low inference time in challenging greenhouse environments.

关键词： Knowledge engineering Geometry Accuracy Three-dimensional displays Shape Neural networks Green products

Spatio-Temporal Consistent Mapping of Growing Plants for Agricultural Robots in the Wild

学校读者我要写书评

暂无评论

Spatio-Temporal Consistent Mapping of Growing Plants for Agr...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Luca Lobefaro Meher V. R. Malladi Tiziano Guadagnino Cyrill Stachniss Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Tracking changes in growing plants is important for automating phenotyping and robots managing crops. In this paper, we propose a system that uses a 3D model of plants along crop rows to enable a robotic platform to localize itself even in the presence of heavy changes and deforming the model to adapt the scene description to the new measurements. In particular, we focus on consumer RGB-D cameras due to their cost-effectiveness and ease of deployment on real platforms. Our approach exploits modern deep-learning-based feature descriptors and geometric information to obtain matches between 3D points corresponding to temporally distant sessions. We then use the associations in a non-rigid registration pipeline to obtain the final result, an updated representation of the 3D model that reflects plant changes. Using a standard RGB-D sensor, we validate our approach on a real-world dataset recorded in a glasshouse. We obtain accurate 4D models of the plants and track the plant traits’ evolution over time. We show, through experiments, that our method is applicable to interpolate plant organs’ evolution, a helpful result for phenotypic trait measurement. We see our approach as a relevant step toward 4D reconstruction for robotic agriculture in the wild.

关键词： Deformable models Solid modeling Three-dimensional displays Shape Deformation Plants (biology) Pipelines Crops Streams Image reconstruction

Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Nunes, Lucas Marcuzzi, Rodrigo Mersch, Benedikt Behley, Jens Stachniss, Cyrill Center for Robotics University of Bonn Germany Lamarr Institute for Machine Learning and Artificial Intelligence Germany

关键词： Optical radar