检索结果-内蒙古大学图书馆

BiCap: A novel bi-modal dataset of daily living dual-arm manipulation actions

INTERNATIONAL JOURNAL OF robotICS RESEARCH 2025年第6期44卷 891-907页

作者： Carmona, David Yu, Haoyong Natl Univ Singapore Biorobot Lab Dept Biomed Engn Block E6Level 75 Engn Dr 1 Singapore 117608 Singapore

Dual-arm manipulation of daily living objects is essential for robots operating in household environments. learning from demonstration is a promising approach for teaching robots dual-arm manipulation skills. It usually requires a dataset of participants demonstrating the sequence of manipulation actions to achieve a goal. That sequence can be represented using symbolical or trajectory encoding. The symbolic-encoded sequence is known as the task plan. The chief limitations of current datasets are that most tend to disregard dual-arm manipulation skills and omit the formal grammar used to annotate the task plans. This paper introduces BiCap, a novel bi-modal dataset of dual-arm manipulation actions on daily living objects coupled with a bio-inspired action context-free grammar for fine-grained task plan annotation to train, test, and validate learning from demonstration-based algorithms for robotic dual-arm manipulation. 15 participants were recruited. The experimenter placed reflective markers on their upper limbs and pelvis. Then, the participants sat at a table where one or two objects were placed. They performed one of the following tasks: pouring, opening, and passing, using both hands. An RGB camera pointing towards the table recorded the participants' hand movements. Subsequently, an annotator reviewed the RGB videos and wrote the participants' task plans using the bio-inspired action context-free grammar. The participants' upper-limb kinematics were computed, too, which provides the trajectory-encoded action sequences. The resulting dataset, BiCap, contains 4,026 task plans, videos, and motion data of the 15 participants.

关键词： data sets for robot learning learning from demonstration robot learning applications and data papers datasets for human motion

来源：评论

学校读者我要写书评

暂无评论

Balancing Cobot Productivity and Longevity Through Pre-Runtime Developer Feedback

引用

IEEE robotICS AND AUTOMATION LETTERS 2025年第2期10卷 1617-1624页

作者： Kolvig-Raun, Emil Stubbe Hviid, Jakob Kjaergaard, Mikkel Baun Brorsen, Ralph Jacob, Peter Univ Southern Denmark SDU Software Engn DK-5230 Odense Denmark Universal Robots DK-5260 Odense Denmark

In our experience, the task of optimizing robot longevity and efficiency is challenging due to the limited understanding and awareness developers' have about how their code influences a robot's expected lifespan. Unfortunately, acquiring the necessary information for computations is a complex task, and the data needed for these calculations remains unattainable until after runtime. In software engineering, traditional Static Code Analysis (SCA) techniques are applied to address such challenges. Although effective in identifying software anomalies and inefficiencies without execution, current SCA techniques do not adequately address the unique requirements of Cyber-Physical Systems (CPSs) in robotics. In this study, we propose a novel Machine learning (ML) approach to assess robot program lines, considering the balance between speed and lifespan. Our solution, trained on data from 1325 operational collaborative robots (cobots) from the Universal robots (UR) e-Series, classifies program lines concerning the expected lifespan of the robot, considering program line arguments, expected resource usage, and asserted joint stress. The model achieves a worst-case accuracy of 90.43% through 10-fold cross-validation with a 50% data split. We also present a selection of programming lines illustrating various robot program cases and an example of longevity improvement. Finally, we publish a dataset containing 56405 unique program line executions, aiming to enhance the sustainability and efficiency of robotic systems and support future research.

关键词： robots Software data models Hardware Stress Runtime Codes Degradation Collaborative robots Kinematics AI-based methods data sets for robot learning software middleware and programming environments

来源：评论

学校读者我要写书评

暂无评论

MotIF: Motion Instruction Fine-Tuning

引用

IEEE robotICS AND AUTOMATION LETTERS 2025年第3期10卷 2287-2294页

作者： Hwang, Minyoung Hejna, Joey Sadigh, Dorsa Bisk, Yonatan MIT Cambridge MA 02139 USA Stanford Univ Stanford CA 94305 USA Google DeepMind Mountain View CA 94043 USA Carnegie Mellon Univ Pittsburgh PA 15213 USA

While success in many robotics tasks can be determined by only observing the final state and how it differs from the initial state - e.g., if an apple is picked up - many tasks require observing the full motion of the robot to correctly determine success. For example, brushing hair requires repeated strokes that correspond to the contours and type of hair. Prior works often use off-the-shelf vision-language models (VLMs) as success detectors;however, when success depends on the full trajectory, VLMs struggle to make correct judgments for two reasons. First, modern VLMs often use single frames, and thus cannot capture changes over a full trajectory. Second, even if we provide state-of-the-art VLMs with an input of multiple frames, they still fail to correctly detect success due to a lack of robot data. Our key idea is to fine-tune VLMs using abstract representations that are able to capture trajectory-level information such as the path the robot takes by overlaying keypoint trajectories on the final image. We propose motion instruction fine-tuning (MotIF), a method that fine-tunes VLMs using the aforementioned abstract representations to semantically ground the robot's behavior in the environment. To benchmark and fine-tune VLMs for robotic motion understanding, we introduce the MotIF-1K dataset containing 653 human and 369 robot demonstrations across 13 task categories with motion descriptions. MotIF assesses the success of robot motion given task and motion instructions. Our model significantly outperforms state-of-the-art API-based single-frame VLMs and video LMs by at least twice in F1 score with high precision and recall, generalizing across unseen motions, tasks, and environments. Finally, we demonstrate practical applications of MotIF in ranking trajectories on how they align with task and motion descriptions.

关键词： robots Trajectory Visualization Semantics robot motion Grounding Detectors Shape Optical flow Tracking Intention recognition data sets for robot learning semantic scene understanding

来源：评论

学校读者我要写书评

暂无评论

ConditionNET: learning Preconditions and Effects for Execution Monitoring

引用

IEEE robotICS AND AUTOMATION LETTERS 2025年第2期10卷 1337-1344页

作者： Sliwowski, Daniel Lee, Dongheui Tech Univ Wien TU Wien Autonomous Syst Lab A-1040 Vienna Austria Inst Robot & Mechatron DLR German Aerosp Ctr D-82234 Wessling Germany

The introduction of robots into everyday scenarios necessitates algorithms capable of monitoring the execution of tasks. In this letter, we propose ConditionNET, an approach for learning the preconditions and effects of actions in a fully data-driven manner. We develop an efficient vision-language model and introduce additional optimization objectives during training to optimize for consistent feature representations. ConditionNET explicitly models the dependencies between actions, preconditions, and effects, leading to improved performance. We evaluate our model on two robotic datasets, one of which we collected for this letter, containing 406 successful and 138 failed teleoperated demonstrations of a Franka Emika Panda robot performing tasks like pouring and cleaning the counter. We show in our experiments that ConditionNET outperforms all baselines on both anomaly detection and phase prediction tasks. Furthermore, we implement an action monitoring system on a real robot to demonstrate the practical applicability of the learned preconditions and effects. Our results highlight the potential of ConditionNET for enhancing the reliability and adaptability of robots in real-world environments.

关键词： robot sensing systems Monitoring Feature extraction Hidden Markov models Planning Probabilistic logic Natural languages Image segmentation Anomaly detection Transformers Deep learning methods data sets for robot learning deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

WildScenes: A benchmark for 2D and 3D semantic segmentation in large-scale natural environments

引用

INTERNATIONAL JOURNAL OF robotICS RESEARCH 2025年第4期44卷 532-549页

作者： Vidanapathirana, Kavisha Knights, Joshua Hausler, Stephen Cox, Mark Ramezani, Milad Jooste, Jason Griffiths, Ethan Mohamed, Shaheer Sridharan, Sridha Fookes, Clinton Moghadam, Peyman CSIRO CSIRO Robot Data61 1 Technology Ct Pullenvale Qld 4069 Australia Queensland Univ Technol Brisbane Qld Australia

Recent progress in semantic scene understanding has primarily been enabled by the availability of semantically annotated bi-modal (camera and LiDAR) datasets in urban environments. However, such annotated datasets are also needed for natural, unstructured environments to enable semantic perception for applications, including conservation, search and rescue, environment monitoring, and agricultural automation. Therefore, we introduce WildScenes, a bi-modal benchmark dataset consisting of multiple large-scale, sequential traversals in natural environments, including semantic annotations in high-resolution 2D images and dense 3D LiDAR point clouds, and accurate 6-DoF pose information. The data is (1) trajectory-centric with accurate localization and globally aligned point clouds, (2) calibrated and synchronized to support bi-modal training and inference, and (3) containing different natural environments over 6 months to support research on domain adaptation. Our 3D semantic labels are obtained via an efficient, automated process that transfers the human-annotated 2D labels from multiple views into 3D point cloud sequences, thus circumventing the need for expensive and time-consuming human annotation in 3D. We introduce benchmarks on 2D and 3D semantic segmentation and evaluate a variety of recent deep-learning techniques to demonstrate the challenges in semantic segmentation in natural environments. We propose train-val-test splits for standard benchmarks as well as domain adaptation benchmarks and utilize an automated split generation technique to ensure the balance of class label distributions. The WildScenes benchmark webpage is https://***/WildScenes, and the data is publicly available at https://***/collection/csiro:61541.

关键词： Semantic scene understanding performance evaluation and benchmarking data sets for robotic vision data sets for robot learning

来源：评论

学校读者我要写书评

暂无评论

CEAR: Comprehensive Event Camera dataset for Rapid Perception of Agile Quadruped robots

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第10期9卷 8999-9006页

作者： Zhu, Shifan Xiong, Zixun Kim, Donghyun Univ Massachusetts Manning Coll Informat & Comp Sci Amherst MA 01003 USA

When legged robots perform agile movements, traditional RGB cameras often produce blurred images, posing a challenge for rapid perception. Event cameras have emerged as a promising solution for capturing rapid perception and coping with challenging lighting conditions thanks to their low latency, high temporal resolution, and high dynamic range. However, integrating event cameras into agile-legged robots is still largely unexplored. To bridge this gap, we introduce CEAR, a dataset comprising data from an event camera, an RGB-D camera, an IMU, a LiDAR, and joint encoders, all mounted on a dynamic quadruped, Mini Cheetah robot. This comprehensive dataset features more than 100 sequences from real-world environments, encompassing various indoor and outdoor environments, different lighting conditions, a range of robot gaits (e.g., trotting, bounding, pronking), as well as acrobatic movements like backflip. To our knowledge, this is the first event camera dataset capturing the dynamic and diverse quadruped robot motions under various setups, developed to advance research in rapid perception for quadruped robots.

关键词： Cameras robot vision systems robots robot sensing systems Legged locomotion Quadrupedal robots Laser radar data sets for SLAM data sets for event camera data sets for robot learning sensor fusion

来源：评论

学校读者我要写书评

暂无评论

MOTIONBENCHMAKER: A Tool to Generate and Benchmark Motion Planning datasets

引用

IEEE robotICS AND AUTOMATION LETTERS 2022年第2期7卷 882-889页

作者： Chamzas, Constantinos Quintero-Pena, Carlos Kingston, Zachary Orthey, Andreas Rakita, Daniel Gleicher, Michael Toussaint, Marc Kavraki, Lydia E. Rice Univ Dept Comp Sci Houston TX 77005 USA TU Learning & Intelligent Syst Lab Berlin Germany Univ Wisconsin Dept Comp Sci 1210 W Dayton St Madison WI 53706 USA

Recently, there has been a wealth of development in motion planning for robotic manipulation-new motion planners are continuously proposed, each with their own unique strengths and weaknesses. However, evaluating new planners is challenging and researchers often create their own ad-hoc problems for benchmarking, which is time-consuming, prone to bias, and does not directly compare against other state-of-the-art planners. We present MOTIONBENCHMAKER, an open-source tool to generate benchmarking datasets for realistic robot manipulation problems. MOTIONBENCHMAKER is designed to be an extensible, easy-to-use tool that allows users to both generate datasets and benchmark them by comparing motion planning algorithms. Empirically, we show the benefit of using MOTIONBENCHMAKER as a tool to procedurally generate datasets which helps in the fair evaluation of planners. We also present a suite of 40 prefabricated datasets, with 5 different commonly used robots in 8 environments, to serve as a common ground to accelerate motion planning research.

关键词： data sets for robot learning motion and path planning manipulation planning

来源：评论

学校读者我要写书评

暂无评论

MoSS: Monocular Shape Sensing for Continuum robots

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第2期9卷 1524-1531页

作者： Shentu, Chengnan Li, Enxu Chen, Chaojun Dewi, Puspita T. Lindell, David B. Burgner-Kahrs, Jessica Univ Toronto Dept Math & Computat Sci Continuum Robot Lab Mississauga ON L5L 1C6 Canada Univ Toronto Dept Comp Sci Toronto ON M5S 1A1 Canada Waabi Innovat Inc Toronto ON M5T 2S6 Canada Univ Toronto Dept Mech & Ind Engn Toronto ON M5S 1A1 Canada Univ Toronto Dept Comp Sci Toronto ON M5S 1A1 Canada

Continuum robots are promising candidates for interactive tasks in medical and industrial applications due to their unique shape, compliance, and miniaturization capability. Accurate and real-time shape sensing is essential for such tasks yet remains a challenge. Embedded shape sensing has high hardware complexity and cost, while vision-based methods require stereo setup and struggle to achieve real-time performance. This letter proposes a novel eye-to-hand monocular approach to continuum robot shape sensing. Utilizing a deep encoder-decoder network, our method, MoSSNet, eliminates the computation cost of stereo matching and reduces requirements on sensing hardware. In particular, MoSSNet comprises an encoder and three parallel decoders to uncover spatial, length, and contour information from a single RGB image, and then obtains the 3D shape through curve fitting. A two-segment tendon-driven continuum robot is used for data collection and testing, demonstrating accurate (mean shape error of 0.91 mm, or 0.36% of robot length) and real-time (70 fps) shape sensing on real-world data. Additionally, the method is optimized end-to-end and does not require fiducial markers, manual segmentation, or camera calibration.

关键词： robots Shape Sensors robot kinematics Three-dimensional displays robot vision systems Cameras Modeling control and learning for soft robots computer vision for medical robotics data sets for robot learning

来源：评论

学校读者我要写书评

暂无评论

Generation of GelSight Tactile Images for Sim2Real learning

引用

IEEE robotICS AND AUTOMATION LETTERS 2021年第2期6卷 4177-4184页

作者： Gomes, Daniel Fernandes Paoletti, Paolo Luo, Shan Univ Liverpool Dept Comp Sci SmARTLab Liverpool L69 3BX Merseyside England Univ Liverpool Sch Engn Liverpool L69 3GH Merseyside England

Most current works in Sim2Real learning for robotic manipulation tasks leverage camera vision that may be significantly occluded by robot hands during the manipulation. Tactile sensing offers complementary information to vision and can compensate for the information loss caused by the occlusions. However, the use of tactile sensing is restricted in the Sim2Real research due to no simulated tactile sensors being available. To mitigate the gap, we introduce a novel approach for simulating a GelSight tactile sensor in the commonly used Gazebo simulator. Similar to the real GelSight sensor, the simulated sensor can produce high-resolution images from depth-maps captured by a simulated optical sensor, and reconstruct the interaction between the touched object and an opaque soft membrane. It can indirectly sense forces, geometry, texture and other properties of the object and enables Sim2Real learning with tactile sensing. Preliminary experimental results have shown that the simulated sensor could generate realistic outputs similar to the ones captured by a real GelSight sensor. All the materials used in this letter are available at https://***/gelsight-simulation.

关键词： Deep learning methods data sets for robot learning force and tactile sensing transfer learning

来源：评论

学校读者我要写书评

暂无评论

Model Free Method of Screening Training data for Adversarial datapoints Through Local Lipschitz Quotient Analysis

引用

IEEE robotICS AND AUTOMATION LETTERS 2024年第12期9卷 11122-11129页

作者： Kamienski, Emily Asada, Harry MIT Dept Mech Engn Cambridge MA 02139 USA

It is often challenging to pick suitable data features for learning problems. Sometimes certain regions of the data are harder to learn because they are not well characterized by the selected data features. The challenge is amplified when resources for sensing and computation are limited and time-critical, yet reliable decisions must be made. For example, a robotic system for preventing falls of elderly people needs a real-time fall predictor, with low false positive and false negative rates, using a simple wearable sensor to activate a fall prevention mechanism. Here we present a methodology for assessing the learnability of data based on the Lipschitz quotient. We develop a procedure for determining which regions of the dataset contain adversarial data points, input data that look similar but belong to different target classes. Regardless of the learning model, it will be hard to learn such data. We then present a method for determining which additional feature(s) are most effective in improving the predictability of each of these regions. This is a model-independent data analysis that can be executed before constructing a prediction model through machine learning or other techniques. We demonstrate this method on two synthetic datasets and a dataset of human falls, which uses inertial measurement unit signals. For the fall dataset, we identified two groups of adversarial data points and improved the predictability of each group over the baseline dataset, as assessed by Lipschitz, by using 2 different sets of features. This work offers a valuable tool for assessing data learnability that can be applied to not only fall prediction problems, but also other robotics applications that learn from data.

关键词： data models Sensors Predictive models robots Machine learning Analytical models Training robot sensing systems Tuning Training data Deep learning methods data sets for robot learning Lipschitz quotient data preparation adversarial data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：