检索结果-内蒙古大学图书馆

UDTIRI: An Online Open-Source intelligent Road Inspection Benchmark Suite

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Guo, Sicen Li, Jiahang Feng, Yi Zhou, Dacheng Zhang, Denghuang Chen, Chen Su, Shuai Zhu, Xingyi Chen, Qijun Fan, Rui Group The College of Electronics & Information Engineering Shanghai Research Institute for Intelligent Autonomous Systems The State Key Laboratory of Intelligent Autonomous Systems Frontiers Science Center for Intelligent Autonomous Systems Tongji University Shanghai201804 China The Department of Control Science & Engineering Tongji University Shanghai201804 China The College of Electronics & Information Engineering Tongji University Shanghai201804 China The Department of Road & Airport Engineering The Key Laboratory of Road & Traffic Engineering of Ministry of Education Tongji University Shanghai200092 China

In the nascent domain of urban digital twins (UDT), the prospects for leveraging cutting-edge deep learning techniques are vast and compelling. Particularly within the specialized area of intelligent road inspection (IRI), a noticeable gap exists, underscored by the current dearth of dedicated research efforts and the lack of large-scale well-annotated datasets. To foster advancements in this burgeoning field, we have launched an online open-source benchmark suite, referred to as UDTIRI. Along with this article, we introduce the road pothole detection task, the first online competition published within this benchmark suite. This task provides a well-annotated dataset, comprising 1,000 RGB images and their pixel/instance-level ground-truth annotations, captured in diverse real-world scenarios under different illumination and weather conditions. Our benchmark provides a systematic and thorough evaluation of state-of-the-art object detection, semantic segmentation, and instance segmentation networks, developed based on either convolutional neural networks or Transformers. We anticipate that our benchmark will serve as a catalyst for the integration of advanced UDT techniques into IRI. By providing algorithms with a more comprehensive understanding of diverse road conditions, we seek to unlock their untapped potential and foster innovation in this critical domain. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

A Hierarchical Deep Reinforcement Learning Strategy for Collective Pursuit-Evasion Game with Partial Observations

学校读者我要写书评

暂无评论

IEEE Transactions on Artificial Intelligence

IEEE Transactions on Artificial Intelligence 2025年

作者： Wang, Xiaoxiao Yi, Peng Hong, Yiguang Tongji University Shanghai Research Institute for Intelligent Autonomous Systems Shanghai200092 China Tongji University School of Electronics and Information Shanghai200070 China Tongji University Control Science and Engineering Department Shanghai200070 China National Key Laboratory of Autonomous Intelligent Unmanned Systems China Ministry of Education Frontiers Science Center for Intelligent Autonomous Systems China

This paper addresses the problem of Collective Pursuit-Evasion Game (C-PEG) with partial observation, aiming to enhance cooperation among cluster pursuers, reduce the game space, and optimize the overall pursuer strategy efficiency in complex dynamic environments. Firstly, we consider a distributed communication model suitable for cluster collaboration and model the multi-agent cooperative task as a Collective Decentralized Partially Observable Markov Decision Process (C-Dec POMDP). Then we propose a Hierarchical Regularization Game Multi-agent Deep Deterministic Policy Gradient (HRG-MADDPG) algorithm, which includes High-Level Strategy (HLS) and Low-Level Strategy (LLS). The HLS takes the current observation from the pursuers’ sensors and the pursuers’ status as inputs, and outputs target allocation strategies for the coalitions and individuals to guide the LLS in executing specific pursuit tasks. The LLS is further divided into centralized training and decentralized execution phases. In the centralized training phase, the guiding strategy provided by the HLS and the introduced regularization auxiliary term are combined to promote decision-making through the fusion of favorable situations and auxiliary terms. In the decentralized execution phase, a reliable policy improvement method based on partially observable information is designed. All in all, the HRG-MADDPG algorithm provides a reliable method for collective pursuit-evasion. It is trained in a single environment and further validated in multiple different test environments. Simulation results confirm the effectiveness of the proposed method, showing the adaptability, scalability, and rapid responsiveness of the proposed framework in various practical applications. © 2020 IEEE.

关键词： Hierarchical systems

NeRF in robotics: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Guangming Pan, Lei Peng, Songyou Liu, Shaohui Xu, Chenfeng Miao, Yanzi Zhan, Wei Tomizuka, Masayoshi Pollefeys, Marc Wang, Hesheng The University of Cambridge ETH Zurich Switzerland The Department of Automation Shanghai Jiao Tong University Shanghai200240 China The Key Laboratory of System Control and Information Processing Ministry of Education of China China Engineering Research Center of Intelligent Control for Underground Space Ministry of Education School of Information and Control Engineering Advanced Robotics Research Center China University of Mining and Technology Xuzhou221116 China ETH Zurich Zurich Switzerland Mechanical Systems Control Laboratory University of California Berkeley United States

Meticulous 3D environment representations have been a longstanding goal in computer vision and robotics fields. The recent emergence of neural implicit representations has introduced radical innovation to this field as implicit representations enable numerous capabilities. Among these, the Neural Radiance Field (NeRF) has sparked a trend because of the huge representational advantages, such as simplified mathematical models, compact environment storage, and continuous scene representations. Apart from computer vision, NeRF has also shown tremendous potential in the field of robotics. Thus, we create this survey to provide a comprehensive understanding of NeRF in the field of robotics. By exploring the advantages and limitations of NeRF, as well as its current applications and future potential, we hope to shed light on this promising area of research. Our survey is divided into two main sections: The Application of NeRF in robotics and The Advance of NeRF in robotics, from the perspective of how NeRF enters the field of robotics. In the first section, we introduce and analyze some works that have been or could be used in the field of robotics from the perception and interaction perspectives. In the second section, we show some works related to improving NeRF's own properties, which are essential for deploying NeRF in the field of robotics. In the discussion section of the review, we summarize the existing challenges and provide some valuable future research directions for reference. © 2024, CC BY-NC-ND.

关键词： robotics

Self-supervised Multi-frame Monocular Depth Estimation with Pseudo-LiDAR Pose Enhancement

学校读者我要写书评

暂无评论

Self-supervised Multi-frame Monocular Depth Estimation with ...

IEEE International Conference on robotics and Automation (ICRA)

作者： Wenhua Wu Guangming Wang Jiquan Zhong Hesheng Wang Zhe Liu MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Department of Automation Key Laboratory of System Control and Information Processing of Ministry of Education Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education Shanghai Engineering Research Center of Intelligent Control and Management Insititute of Medical Robotics Shanghai Jiao Tong University Shanghai China

Depth estimation is one of the most important tasks in scene understanding. In the existing joint self-supervised learning approaches of depth-pose estimation, depth estimation and pose estimation networks are independent of each other. They only use the adjacent image frames for pose estimation and lack the use of the estimated geometric information. To enhance the depth-pose association, we propose a monocular multi-frame unsupervised depth estimation framework, named PLPE-Depth. There are a depth estimation network and two pose estimation networks with image input and pseudo-LiDAR input. The main idea of our approach is to use the pseudo-LiDAR reconstructed from the depth map to estimate the pose of adjacent frames. We propose depth re-estimation with a better pose between the image pose and the pseudo-LiDAR pose to improve the accuracy of estimation. Besides, we improve the reconstruction loss and design a pseudo-LiDAR pose enhancement loss to facilitate the joint learning. Our approach enhances the use of the estimated depth information and strengthens the coupling between depth estimation and pose estimation. Experiments on the KITTI dataset show that our depth estimation achieves state-of-the-art performance at low resolution. Our source codes will be released on https://***/IRMVLabIPLPE-Depth.

关键词：

Adaptive Block Elevation Mapping for Large-scale Scene

学校读者我要写书评

暂无评论

Adaptive Block Elevation Mapping for Large-scale Scene

IEEE International Conference on robotics and Biomimetics

作者： Yang Zhou Shiqiang Zhu Huang Huang Yuehua Li Jason Gu Research Center for Intelligent Robotics Research Institute of Interdisciplinary Innovation Zhejiang Laboratory Hangzhou China Zhejiang Engineering Research Center for Intelligent Robotics Hangzhou China Beijing Institute of Control Engineering Beijing China Beijing Research Institute of Zhejiang Laboratory Beijing China Department of Electrical and Computer Engineering Dalhousie University Halifax NS Canada

ISBN: (纸本)9781665481106

Dense map that contains the surrounding geometry and vision information of a robot is widely used for path planning, navigation, obstacle avoidance and other applications. Considering the performance of the processing unit mounted on the robot is limited, mapping algorithm has to make compromise by sacrificing speed and precision. It will be more challenging when the dense mapping scene is very large because the memory consumption will be greatly increased and the map is difficult to be extended if beyonding the initial map. To suppress the negative impact from the increased map scale, we proposed a novel block mapping approach to generate the dense map in large scale of scene. In this work, the elevation map is selected as the base dense map. The entire elevation map is segmented into numerous block maps of which size is much smaller than that of the entire map. The present moment of lidar and vision measurements are used to generate the local elevation map. The local elevation map is used to update block maps which are adaptively generated along the motion trajectory. A memory-disk interaction mechanism, which the block maps will be loaded to memory or saved to local disk when needed, is introduced. Our block mapping approach is tested on the KITTI datasets, and the results demonstrate that the mapping approach can stably operate in a large scale of scene with a much smaller consumption of memory.

关键词： Geometry Laser radar Navigation Motion segmentation Memory management Robot sensing systems Trajectory

Deep Domain Adaptation Regression for Force Calibration of Optical Tactile Sensors

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Zhuo Ou, Ni Jiang, Jiaqi Luo, Shan Robot Perception Lab Centre for Robotics Research Department of Engineering King’s College London LondonWC2R 2LS United Kingdom State Key Laboratory of Intelligent Control and Decision of Complex Systems Beijing Institute of Technology Beijing100081 China

Optical tactile sensors provide robots with rich force information for robot grasping in unstructured environments. The fast and accurate calibration of three-dimensional contact forces holds significance for new sensors and existing tactile sensors which may have incurred damage or aging. However, the conventional neural-network-based force calibration method necessitates a large volume of force-labeled tactile images to minimize force prediction errors, with the need for accurate Force/Torque measurement tools as well as a time-consuming data collection process. To address this challenge, we propose a novel deep domain-adaptation force calibration method, designed to transfer the force prediction ability from a calibrated optical tactile sensor to uncalibrated ones with various combinations of domain gaps, including marker presence, illumination condition, and elastomer modulus. Experimental results show the effectiveness of the proposed unsupervised force calibration method, with lowest force prediction errors of 0.102N (3.4% in full force range) for normal force, and 0.095N (6.3%) and 0.062N (4.1%) for shear forces along the x-axis and y-axis, respectively. This study presents a promising, general force calibration methodology for optical tactile sensors. Copyright © 2024, The Authors. All rights reserved.

关键词： Forecasting

TransForce: Transferable Force Prediction for Vision-based Tactile Sensors with Sequential Image Translation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Zhuo Ou, Ni Zhang, Xuyang Luo, Shan The Robot Perception Lab Centre for Robotics Research Department of Engineering King's College London LondonWC2R 2LS United Kingdom The State Key Laboratory of Intelligent Control and Decision of Complex Systems Beijing Institute of Technology Beijing100081 China

Vision-based tactile sensors (VBTSs) provide high-resolution tactile images crucial for robot in-hand manipulation. However, force sensing in VBTSs is underutilized due to the costly and time-intensive process of acquiring paired tactile images and force labels. In this study, we introduce a transferable force prediction model, TransForce, designed to leverage collected image-force paired data for new sensors under varying illumination colors and marker patterns while improving the accuracy of predicted forces, especially in the shear direction. Our model effectively achieves translation of tactile images from the source domain to the target domain, ensuring that the generated tactile images reflect the illumination colors and marker patterns of the new sensors while accurately aligning the elastomer deformation observed in existing sensors, which is beneficial to force prediction of new sensors. As such, a recurrent force prediction model trained with generated sequential tactile images and existing force labels is employed to estimate higher-accuracy forces for new sensors with lowest average errors of 0.69N (5.8% in full work range) in x-axis, 0.70N (5.8%) in y-axis, and 1.11N (6.9%) in z-axis compared with models trained with single images. The experimental results also reveal that pure marker modality is more helpful than the RGB modality in improving the accuracy of force in the shear direction, while the RGB modality show better performance in the normal direction. Copyright © 2024, The Authors. All rights reserved.

关键词： RGB color model

AI Enabled Automatic Mobile Robot intelligent Navigation in Construction with Obstacle Awareness

学校读者我要写书评

暂无评论

IEEE Transactions on Automation Science and engineering 2025年

作者： Zhang, Yinlong Liu, Yuanhao Cui, Yunge Zeng, Ziming Liang, Wei Chinese Academy of Sciences State Key Laboratory of Robotics Shenyang Institute of Automation Shenyang110016 China Chinese Academy of Sciences Key Laboratory of Networked Control Systems Shenyang110016 China University of Chinese Academy of Sciences Beijing101408 China Zeekr Group Zeekr Intelligent Driving Department Shanghai110078 China Shenzhen Polytechnic University School of Automotive and Transportation Engineering Shenzhen518055 China

In the construction industry, the integration of artificial intelligence (AI) and robotics has led to significant advancements in automating various tasks. One critical aspect is the intelligent navigation of automatic mobile robot (AMR) within construction sites, where dynamic environments pose challenges such as inaccurate robot state estimation, irregular and textureless obstacle detection. To solve these issues, this paper designs an AI enabled obstacle-aware AMR intelligent navigation approach while ensuring robot safety and efficiency. Specifically, the robot is equipped with the complementary RGB camera, inertial measurement unit (IMU) and wheel encoder to estimate the states (i.e., positions, orientations and velocities), which have been tightly fused in the optimization framework. Furthermore, the RGB-Depth images are jointly fed into the AI model. It combines of Mask-RCNN and multi-scale attention network, to detect and segment the obstacles, and estimate the corresponding relative depth. It should be noted that the system incorporates obstacle awareness mechanisms to dynamically switch the robot's velocities in response to obstacles in the environment, ensuring smooth and collision-free movement. Experimental results demonstrate the effectiveness and robustness of the proposed AI-enabled navigation system in real-world construction scenarios, showcasing its potential to enhance productivity and safety in construction automation. © 2004-2012 IEEE.

关键词： Construction industry

Cross-Camera Multi-Person Re-Identification Based on Local-Global Dynamic Feature Matching

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Zhou, Yanmin Chen, Wenji Li, Jie Zhang, Shiyuan Wang, Yixiang Lu, Ping He, Bin Department of Control Science & Engineering College of Electronics & Information Engineering Tongji University Shanghai201804 China National Key Laboratory of Autonomous Intelligent Unmanned Systems Shanghai201210 China Frontiers Science Center for Intelligent Autonomous Systems Shanghai201210 China Tongji University China

Cross-camera pedestrian tracking continuously tracks pedestrians in a monitoring network composed of multiple cameras. Due to the large degree of freedom of the human body and the influence of the environment variations, the pedestrian recognition is challenging for such tasks. We propose a method of dynamically matching local features of horizontal and vertical stripes (DFM+) to automatically align pedestrian features and obtain more stable pedestrian appearance features. In addition, we design a three-branched local-global dynamic feature matching network (3bDFM-Net) framework including the person orientation, and the local and global branches for cross-camera multi-person re-identification (ReId). Experiment results show that the accuracy of cross-camera multi-person tracking is improved by introducing the pedestrian orientation. The Rank-1 accuracies on the Market1501 and DukeMTMCReID datasets reach 96.2% and 90.1%, respectively. Further tests verified its capabilities of cross-camera tracking of pedestrians in the surveillance video of real scenes. © 2023, The Authors. All rights reserved.

关键词： Cameras