检索结果-内蒙古大学图书馆

FoundationGrasp: Generalizable Task-Oriented Grasping With Foundation Models

学校读者我要写书评

暂无评论

IEEE Transactions on Automation Science and Engineering 2025年 22卷 12418-12435页

作者： Tang, Chao Huang, Dehao Dong, Wenlong Xu, Ruinian Zhang, Hong Southern University of Science and Technology Shenzhen Key Laboratory of Robotics and Computer Vision Department of Electronic and Electrical Engineering Shenzhen518055 China Institute for Robotics and Intelligent Machines Georgia Institute of Technology AtlantaGA30332 United States

Task-oriented grasping (TOG), which refers to synthesizing grasps on an object that are configurationally compatible with the downstream manipulation task, is the first milestone towards tool manipulation. Analogous to the activation of two brain regions responsible for semantic and geometric reasoning during cognitive processes, modeling the intricate relationship between objects, tasks, and grasps neces-sitates rich semantic and geometric prior knowledge about these elements. Existing methods typically restrict the prior knowledge to a closed-set scope, limiting their generalization to novel objects and tasks out of the training set. To addresssuch a limitation, we propose FoundationGrasp, a foundationmodel-based TOG framework that leverages the open-ended knowledge from foundation models to learn generalizable TOG skills. Extensive experiments are conducted on the contributed Language and vision Augmented TaskGrasp (LaViA-TaskGrasp)dataset, demonstrating the superiority of FoundationGrasp over existing methods when generalizing to novel object instances,object classes, and tasks out of the training set. Furthermore,the effectiveness of FoundationGrasp is validated in real-robot grasping and manipulation experiments on a 7-DoF robotic arm. © 2004-2012 IEEE.

关键词： Robotic arms

A Hierarchical Progressive Perception System for Autonomous Luggage Trolley Collection

学校读者我要写书评

暂无评论

IEEE Transactions on Automation Science and Engineering 2025年 22卷 14484-14497页

作者： Sun, Zhirui Zhang, Zhe Zhao, Jieting Ye, Hanjing Wang, Jiankun Southern University of Science and Technology Shenzhen Key Laboratory of Robotics Perception and Intelligence Department of Electronic and Electrical Engineering Shenzhen518055 China Southern University of Science and Technology Jiaxing Research Institute Jiaxing314000 China Southern University of Science and Technology Shenzhen Key Laboratory of Robotics and Computer Vision Department of Electronic and Electrical Engineering Shenzhen518055 China

Advancements in intelligent vehicle technologies and autonomous driving are now used for tasks like collecting luggage trolleys at airports. The robotic autonomous luggage trolley collection system employs robots to gather and transport scattered luggage trolleys. However, existing methods for detecting and locating luggage trolleys often fail when they are not fully visible. To address this, we introduce the hierarchical progressive perception system, which enhances the detection and localization of luggage trolleys under partial occlusion. The proposed system integrates the hierarchical process structure and progressive perception strategy. This innovative structure processes the luggage trolley’s position and orientation separately. It can accurately determine the luggage trolley’s position with just one well-detected keypoint and estimate its orientation when partially occluded. Once the luggage trolley’s initial pose is detected, the progressive perception strategy continuously refines this information until the robot begins grasping. The proposed system only needs RGB images for labeling and training, eliminating the need for complex data collection and annotation. The experiments on detection and localization demonstrate that the proposed system is more reliable under partial occlusion compared to existing methods. Its effectiveness and robustness have also been confirmed through practical tests in actual luggage trolley collection tasks. © 2004-2012 IEEE.

关键词： Intelligent robots

Real-Time Multi-Object Tracking using YOLOv8 and SORT on a SoC FPGA

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Danilowicz, Michal Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Krakow Poland

关键词： System-on-chip

Introspective Perception for Long-Term Aerial Telemanipulation With Virtual Reality

学校读者我要写书评

暂无评论

Field Robotics

Field robotics 2025年 3卷 323-367页

作者： Jongseok Lee Ribin Balachandran Konstantin Kondak Andre Coelho Marco De Stefano Matthias Humt Jianxiang Feng Tamim Asfour Rudolph Triebel Institute of Robotics and Mechatronics German Aerospace Center (DLR) Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology (KIT) Robotics and Mechatronics Laboratory University of Twente (UT) Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology (KIT) Chair of Computer Vision and Artificial Intelligence Technical University of Munich (TUM)

This paper presents a novel telepresence system for advancing aerial manipulation in dynamic and unstructured environments. The proposed system not only features a haptic device, but also a virtual reality (VR) interface that provides real-time 3D displays of the robot's workspace as well as a haptic guidance to its remotely located operator. To realize this, multiple sensors, namely, a LiDAR, cameras, and IMUs are utilized. For processing of the acquired sensory data, pose estimation pipelines are devised for industrial objects of both known and unknown geometries. We further propose an active learning pipeline in order to increase the sample efficiency of a pipeline component that relies on a Deep Neural Network (DNN) based object detector. All these algorithms jointly address various challenges encountered during the execution of perception tasks in industrial scenarios. In the experiments, exhaustive ablation studies are provided to validate the proposed pipelines. Methodologically, these results commonly suggest how an awareness of the algorithms' own failures and uncertainty (“introspection”) can be used to tackle the encountered problems. Moreover, outdoor experiments are conducted to evaluate the effectiveness of the overall system in enhancing aerial manipulation capabilities. In particular, with flight campaigns over days and nights, from spring to winter, and with different users and locations, we demonstrate over 70 robust executions of pick-and-place, force application and peg-in-hole tasks with the DLR cable-Suspended Aerial Manipulator (SAM). As a result, we show the viability of the proposed system in future industrial applications. 1

关键词： Robots Service robots Sensors Pipelines Active learning Pose estimation Haptic interfaces Telepresence Visualization Virtual reality

FlowPlan: Zero-Shot Task Planning with LLM Flow Engineering for Robotic Instruction Following

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Lin, Zijun Tang, Chao Ye, Hanjing Zhang, Hong Southern University of Science and Technology China Shenzhen Key Laboratory of Robotics and Computer Vision China National University of Singapore Singapore

Robotic instruction following tasks require seamless integration of visual perception, task planning, target localization, and motion execution. However, existing task planning methods for instruction following are either data-driven or underperform in zero-shot scenarios due to difficulties in grounding lengthy instructions into actionable plans under operational constraints. To address this, we propose FlowPlan, a structured multi-stage LLM workflow that elevates zero-shot pipeline and bridges the performance gap between zero-shot and data-driven in-context learning methods. By decomposing the planning process into modular stages—task information retrieval, language-level reasoning, symbolic-level planning, and logical evaluation—FlowPlan generates logically coherent action sequences while adhering to operational constraints and further extracts contextual guidance for precise instance-level target localization. Benchmarked on ALFRED and validated in real-world applications, our method achieves competitive performance relative to data-driven in-context learning methods and demonstrates adaptability across diverse environments. This work advances zero-shot task planning in robotic systems without reliance on labeled data. Project website: https://***/. Copyright © 2025, The Authors. All rights reserved.

关键词： Zero-shot learning

vision-based automatic fruit counting with UAV

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Szolc, Hubert Wasala, Mateusz Mietla, Remigiusz Iwicki, Kacper Kryjak, Tomasz Embedded Vision Systems Group Computer Vision Laboratory Department of Automatic Control and Robotics AGH University of Krakow al. Mickiewicza 30 Krakow30–059 Poland

The use of unmanned aerial vehicles (UAVs) for smart agriculture is becoming increasingly popular. This is evidenced by recent scientific works, as well as the various competitions organised on this topic. Therefore, in this work we present a system for automatic fruit counting using UAVs. To detect them, our solution uses a vision algorithm that processes streams from an RGB camera and a depth sensor using classical image operations. Our system also allows the planning and execution of flight trajectories, taking into account the minimisation of flight time and distance covered. We tested the proposed solution in simulation and obtained an average score of 87.27/100 points from a total of 500 missions. We also submitted it to the UAV Competition organised as part of the ICUAS 2024 conference, where we achieved an average score of 84.83/100 points, placing 6th in a field of 23 teams and advancing to the finals. © 2025, CC BY.

关键词：

An Aerial/Ground Robot Team for Autonomous Firefighting in Urban GNSS-Denied Scenarios

学校读者我要写书评

暂无评论

Field Robotics

Field robotics 2025年 2卷 241-273页

作者： Simón Martínez-Rozas Rafael Rey David Alejo Domingo Acedo José Antonio Cobano Alejandro Rodríguez-Ramos Pascual Campoy Luis Merino Fernando Caballero Service Robotics Laboratory Universidad Pablo de Olavide Crta. Utrera km. 1 41013 Sevilla Spain Computer Vision and Aerial Robotics Group Centre for Automation and Robotics (C.A.R.) Universidad Politécnica de Madrid (UPM-CSIC) Calle Jose Gutierrez Abascal 2 28006 Madrid Spain

This paper presents a framework for urban firefighting with a heterogeneous aerial/ground robot team. The system was developed to address Challenge 3 of the MBZIRC'20. The challenge required autonomously detecting, locating, and extinguishing fires on multiple interior floors and on exterior facade and ground surfaces. Our multi-robot system consists of a robot team of up to three Unmanned Aerial Vehicles (UAV) and one Unmanned Ground Vehicle (UGV). We describe the main hardware and software components for UAV and UGV platforms and present the system's main algorithmic components: a 3D LIDAR-based Mapping and Localization module able to work in GPS-denied scenarios, a Global Planner and a fast local re-planning system for robot navigation, Infrared-Based Perception and robot actuation control for fire extinguishing, and Mission Executive and coordination module based on Behavior Trees. Finally we describe our results from the competition, where the system worked fully autonomously and scored in all the trials performed. The presented system ended in 7 th place (out of 20 teams) in Challenge 3 and 5 th place (out of 17 teams) in the Grand Challenge's Challenge 3 portion.

关键词： Robots Robot kinematics Buildings Robot sensing systems Navigation Autonomous aerial vehicles Sensors Hardware Global Positioning System Autonomous robots

HGDiffuser: Efficient Task-Oriented Grasp Generation via Human-Guided Grasp Diffusion Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Huang, Dehao Dong, Wenlong Tang, Chao Zhang, Hong Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology Shenzhen China Department of Electronic and Electrical Engineering Southern University of Science and Technology Shenzhen China

Task-oriented grasping (TOG) is essential for robots to perform manipulation tasks, requiring grasps that are both stable and compliant with task-specific constraints. Humans naturally grasp objects in a task-oriented manner to facilitate subsequent manipulation tasks. By leveraging human grasp demonstrations, current methods can generate high-quality robotic parallel-jaw task-oriented grasps for diverse objects and tasks. However, they still encounter challenges in maintaining grasp stability and sampling efficiency. These methods typically rely on a two-stage process: first performing exhaustive task-agnostic grasp sampling in the 6-DoF space, then applying demonstration-induced constraints (e.g., contact regions and wrist orientations) to filter candidates. This leads to inefficiency and potential failure due to the vast sampling space. To address this, we propose the Human-guided Grasp Diffuser (HGDiffuser), a diffusion-based framework that integrates these constraints into a guided sampling process. Through this approach, HGDiffuser directly generates 6-DoF task-oriented grasps in a single stage, eliminating exhaustive task-agnostic sampling. Furthermore, by incorporating Diffusion Transformer (DiT) blocks as the feature backbone, HGDiffuser improves grasp generation quality compared to MLP-based methods. Experimental results demonstrate that our approach significantly improves the efficiency of task-oriented grasp generation, enabling more effective transfer of human grasping strategies to robotic systems. To access the source code and supplementary videos, visit https://***/ view/hgdiffuser. Copyright © 2025, The Authors. All rights reserved.

关键词： robotics

Toward Lung Ultrasound Automation: Fully Autonomous Robotic Longitudinal and Transverse Scans Along Intercostal Spaces

学校读者我要写书评

暂无评论

IEEE Transactions on Medical Robotics and Bionics

IEEE Transactions on Medical robotics and Bionics 2025年第2期7卷 768-781页

作者： Lei, Long Hu, Yingbai Jiang, Zixing Miao, Juzheng Luo, Xiao Zhang, Yu Wang, Qiong Wang, Shujun Li, Zheng Heng, Pheng-Ann The Chinese University of Hong Kong Department of Computer Science and Engineering Hong Kong Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China The Chinese University of Hong Kong Multi-Scale Medical Robotics Centre Ltd. Hong Kong The Chinese University of Hong Kong Department of Surgery Hong Kong Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen518055 China The Hong Kong Polytechnic University Department of Biomedical Engineering Hong Kong Chow Yuk Ho Technology Centre for Innovative Medicine Li Ka Shing Institute of Health Sciences Multiscale Medical Robotics Centre Ltd. The Chinese University of Hong Kong Department of Surgery Hong Kong

Lung ultrasound scanning is essential for diagnosing lung diseases. The scan effectiveness critically depends on both longitudinal and transverse scans through intercostal spaces to reduce rib shadowing interference, as well as maintaining the probe perpendicular to pleura for pathological artifact generation. Achieving this level of scan quality often depends heavily on the experience of doctors. Robotic ultrasound scanning shows promise, but currently lacks a direct path planning method for intercostal scanning, and probe orientation does not consider imaging differences between lungs and solid organs. In this paper, we aim to fully automate two fundamental operations in lung ultrasound scanning: longitudinal and transverse scans. We propose pioneering path planning methods along intercostal spaces and innovative solutions for adaptive probe posture adjustment using real-time pleural line feedback, specifically addressing the unique characteristics of lung ultrasound scanning. This ensures the acquisition of high-quality, diagnostically meaningful ultrasound images. In addition, we develop a robotic lung ultrasound system to validate the proposed methods. Extensive experimental results on two volunteers and a chest phantom confirm the efficacy of our methods, and demonstrate the system's feasibility in automated lung ultrasound examinations. Our work lays a solid foundation for automated robotic complete lung scanning. © 2018 IEEE.

关键词： Motion planning