检索结果-内蒙古大学图书馆

DRO: deep Recurrent Optimizer for Video to Depth

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第5期8卷 2844-2851页

作者： Gu, Xiaodong Yuan, Weihao Dai, Zuozhuo Zhu, Siyu Tang, Chengzhou Dong, Zilong Tan, Ping Alibaba Grp Hangzhou 330501 Peoples R China Simon Fraser Univ Burnaby BC V5A 1S6 Canada

There are increasing interests of studying the video-to-depth (V2D) problem with machine learning techniques. While earlier methods directly learn a mapping from images to depth maps and camera poses, more recent works enforce multi-view geometry constraints through optimization embedded in the learning framework. This paper presents a novel optimization method based on recurrent neural networks to further exploit the potential of neural networks in V2D. Specifically, our neural optimizer alternately updates the depth and camera poses through iterations to minimize a feature-metric cost, and two gated recurrent units iteratively improve the results by tracing historical information. Extensive experimental results demonstrate that our method outperforms previous methods and is more efficient in computation and memory consumption than cost-volume-based methods. In particular, our self-supervised method outperforms previous supervised methods on the KITTI and ScanNet datasets. Our source code will be made public.

关键词： Costs Head Optimization Feature extraction Cameras Logic gates Task analysis deep learning for visual perception computer vision for transportation visual learning

来源：评论

学校读者我要写书评

暂无评论

DYS-SLAM: A real-time RGBD SLAM combined with optical flow and semantic information in a dynamic environment

引用

JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2023年第6期45卷 10349-10367页

作者： Fang, Yuhua Xie, Zhijun Chen, Kewei Huang, Guangyan Zarei, Roozbeh Xie, Yuntao Ningbo Univ Sch Fac Elect Engn & Comp Sci Ningbo Peoples R China Ningbo Univ Sch Fac Mech Engn & Mech Ningbo Peoples R China Deakin Univ Sch Informat Technol Melbourne Vic Australia Univ New South Wales Sch Comp Sci & Engn Sydney NSW Australia

Traditional Simultaneous Localization and Mapping application in dynamic situations is constrained by static assumptions. However, the majority of well-known dynamic SLAM systems use deep learning to identify dynamic objects, which creates the issue of trade-offs between accuracy and real-time. To tackle this issue, this work suggests a unique dynamic semantics method(DYS-SLAM) for semantic simultaneous localization and mapping that strikes a trade-off between high accuracy and high real-time performance. We propose M-LK, an enhanced Lucas-Kanade(LK) optical flow method. This technique keeps the continuous motion and greyscale consistency assumptions from the original method while switching out the spatial consistency assumption for a motion consistency assumption, reducing sensitivity to image gradients to identify dynamic feature points and regions efficiently. In order to increase segmentation accuracy while maintaining real-time performance, we develop a segmentation refinement scheme that projects 3D point cloud segmentation results into 2D object detection zones. A dense semantic octree graph is built in the interim to expedite the high-level process. Compared to the four equivalent dynamic SLAM approaches, experiments on the publicly available TUM RGB-D dataset demonstrate that the DYS-SLAM method offers competitive localization accuracy and satisfactory real-time performance in both high and low-dynamic scenarios.

关键词： visual SLAM object detection dynamic environment deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

CSGP: Closed-Loop Safe Grasp Planning via Attention-Based deep Reinforcement learning From Demonstrations

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2023年第6期8卷 3158-3165页

作者： Tang, Zixin Shi, Yifei Xu, Xin Natl Univ Def Technol Coll Intelligence Sci & Technol Changsha 410073 Peoples R China

Grasping is at the core of many robotic manipulation tasks. Despite the recent progress, closed-loop grasp planning in stacked scenes is still unsatisfactory, in terms of efficiency, stability, and most importantly, safety. In this letter, we present CSGP, a closed-loop safe grasp planning approach via attention-based deep reinforcement learning (DRL) from demonstrations, which is capable of learning safe grasping policies that make surrounding objects less disturbed or damaged during manipulation. In CSGP, a 6-DoF safe grasping policy network with a Next-Best-Region attention module is proposed to intrinsically identify the safe regions in the view, facilitating the learning of safe grasping actions. Moreover, we design a fully automatic pipeline in the simulator to collect safe grasping demonstrations, which are utilized to pre-train the policy with behavior cloning and fine-tune it with DRL. To effectively and stably improve the policy during fine-tuning, a DRL from demonstrations algorithm named Safe-DDPGfD is presented in CSGP with a truncated height-anneal exploration mechanism for safe exploration. Moreover, we provide a benchmark that contains scenes with multiple levels of stack layers for method evaluation. Simulation results demonstrate the state-of-the-art performance of our method, achieving the Overall score of 88% in our benchmark. Also, real-world robot grasping experiments also show the effectiveness of our method.

关键词： deep learning in grasping and manipulation deep learning for visual perception learning from demonstration

来源：评论

学校读者我要写书评

暂无评论

Grasp-and-Classify Robotic Sorting With Grasping Rectangle Correction and Weighted Nearest-Neighbor Relation Network

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2025年第6期10卷 6103-6110页

作者： Han, Dongxiao Li, Yuwen Shanghai Univ Sch Mechatron Engn & Automat Shanghai Key Lab Intelligent Mfg & Robot Shanghai 201900 Peoples R China

Robotic sorting in cluttered environments still faces significant challenges, especially with resource-constrained hardware. Traditional detect-and-grasp workflows usually require extensive image collection and annotation for model training, which can become impractical when the categories of the sorted objects frequently change. To overcome this issue, this article proposes a grasp-and-classify robotic sorting method with deep learning-based object grasping and classification algorithms which can be deployed on resource-constrained hardware platforms. To do this, a Grasping Rectangle Correction (GRC) algorithm is incorporated to adjust the grasping poses generated from the Generative Residual Convolutional Neural Network (GR-ConvNetv2). Then, an efficient Weighted Nearest-Neighbor Relation Network (WNNRNet) is developed for few-shot object classification. This model unifies deep Nearest Neighbor Neural Network (DN4) and Relation network to reduce overfitting through feature sharing, and the joint training with a weighted multi-task loss function can enhance the generalization capability of few-shot classification. Simulation tests have been carried out to validate the GRC and WNNRNet algorithms with Cornell, Jacquard, and MiniImageNet datasets. Finally, a robotic sorting system with a UR10 robot and a Kinect camera has been built to perform real-world sorting tests to demonstrate the effectiveness of the proposed method. Benefited from the efficient correction of the grasping pose with the GRC algorithm and the fact that the WNNRNet requires limited samples for training, the proposed method can be deployed on a consumer-level laptop for sorting stacked objects in scenarios with varying categories.

关键词： Grasping Robots Sorting Training Classification algorithms Computational modeling Adaptation models Grippers Libraries Hardware deep learning for visual perception deep learning in grasping and manipulation grasping

来源：评论

学校读者我要写书评

暂无评论

Metrically Scaled Monocular Depth Estimation through Sparse Priors for Underwater Robots

Metrically Scaled Monocular Depth Estimation through Sparse ...

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Ebner, Luca Billings, Gideon Williams, Stefan Swiss Fed Inst Technol Robot Syst Lab Zurich Switzerland Univ Sydney Australian Ctr Field Robot Sydney NSW Australia

ISBN: (纸本)9798350384581;9798350384574

In this work, we address the problem of real-time dense depth estimation from monocular images for mobile underwater vehicles. We formulate a deep learning model that fuses sparse depth measurements from triangulated features to improve the depth predictions and solve the problem of scale ambiguity. To allow prior inputs of arbitrary sparsity, we apply a dense parameterization method. Our model extends recent state-of-the-art approaches to monocular image based depth estimation, using an efficient encoder-decoder backbone and modern lightweight transformer optimization stage to encode global context. The network is trained in a supervised fashion on the forward-looking underwater dataset, FLSea. Evaluation results on this dataset demonstrate significant improvement in depth prediction accuracy by the fusion of the sparse feature priors. In addition, without any retraining, our method achieves similar depth prediction accuracy on a downward looking dataset we collected with a diver operated camera rig, conducting a survey of a coral reef. The method achieves real-time performance, running at 24 FPS on a NVIDIA Jetson Xavier NX, 160 FPS on a NVIDIA RTX 2080 GPU and 7 FPS on a single Intel i9-9900K CPU core, making it suitable for direct deployment on embedded GPU systems. The implementation of this work is made publicly available at https://***/ebnerluca/uw_depth.

关键词： Marine Robotics Computer Vision for Automation deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

Multi-task learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator

Multi-task Learning for Real-time Autonomous Driving Leverag...

引用

IEEE International Conference on Robotics and Automation (ICRA)

作者： Choi, Wonhyeok Shin, Mingyu Lee, Hyukzae Cho, Jaehoon Park, Jaehyeon Im, Sunghoon DGIST Dept Elect Engn & Comp Sci Daegu South Korea DGIST Dept Interdisciplinary Studies Artificial Intelli Daegu South Korea Hyundai Motor Co Autonomous Driving Ctr Seoul South Korea

ISBN: (纸本)9798350384581;9798350384574

Real-time processing is crucial in autonomous driving systems due to the imperative of instantaneous decision-making and rapid response. In real-world scenarios, autonomous vehicles are continuously tasked with interpreting their surroundings, analyzing intricate sensor data, and making decisions within split seconds to ensure safety through numerous computer vision tasks. In this paper, we present a new real-time multi-task network adept at three vital autonomous driving tasks: monocular 3D object detection, semantic segmentation, and dense depth estimation. To counter the challenge of negative transfer - the prevalent issue in multi-task learning - we introduce a task-adaptive attention generator. This generator is designed to automatically discern interrelations across the three tasks and arrange the task-sharing pattern, all while leveraging the efficiency of the hard-parameter sharing approach. To the best of our knowledge, the proposed model is pioneering in its capability to concurrently handle multiple tasks, notably 3D object detection, while maintaining real-time processing speeds. Our rigorously optimized network, when tested on the Cityscapes-3D datasets, consistently outperforms various baseline models. Moreover, an in-depth ablation study substantiates the efficacy of the methodologies integrated into our framework.

关键词： Autonomous Driving Real-time Multi-task learning deep learning for visual perception

来源：评论

学校读者我要写书评

暂无评论

PromptTAD: Object-Prompt Enhanced Traffic Anomaly Detection

引用

IEEE Robotics and Automation Letters 2025年第7期10卷 7174-7181页

作者： Qiu, Hao Yang, Xiaobo Gong, Xiaojin Zhejiang University College of Information Science and Electronic Engineering Zhejiang Hangzhou 310027 China Zhejiang University Faculty of the College of Information Science and Electronic Engineering Zhejiang Hangzhou 310027 China

Ego-centric Traffic Anomaly Detection (TAD) aims to identify abnormal events in videos captured by dashboard-mounted cameras in vehicles. Compared to anomaly detection in roadside surveillance videos, ego-centric TAD poses greater challenges due to the dynamic backgrounds caused by vehicle motion. Previous frame-level methods are often vulnerable to interference from these dynamic backgrounds and struggle to detect small objects located at a distance or off-center. To address these challenges, we propose an object-prompt enhanced method that integrates detected traffic objects into a frame-level TAD framework. Our approach introduces an object-prompt scheme comprising an object prompt encoder, along with two cross-attention-based aggregation modules: an instance-wise aggregation module for fusing information between object instances and the scene, and a relation-wise aggregation module for capturing relationships inter-objects. Additionally, we design an instance-level loss to supervise anomaly detection at the object level. Our method effectively mitigates interference from dynamic backgrounds, improves the detection of distant or off-center anomalies, and enables precise spatial localization of anomalies. Experimental results on the DoTA and DADA-2000 datasets demonstrate that our method achieves state-of-the-art performance. © 2016 IEEE.

关键词： deep learning for visual perception object detection segmentation and categorization

来源：评论

学校读者我要写书评

暂无评论

Assessing Monocular Depth Estimation Networks for UAS Deployment in Rainforest Environments

Assessing Monocular Depth Estimation Networks for UAS Deploy...

引用

2024 International Conference on Intelligent Robots and Systems

作者： Tangellapalli, Srisai Anirudh Peschel, Joshua Sangha, Harman Singh Duncan, Brittany A. Univ Nebraska Nimbus Lab Lincoln NE 68588 USA Iowa State Univ Agr & Biosyst Engn Ames IA USA

ISBN: (纸本)9798350377712;9798350377705

The primary objective of this study was to utilize state-of-the-art deep learning-based monocular depth estimation models to assist UAS pilots in rainforest canopy data collection and navigation. Monocular depth estimation models provide a complementary technique to other depth measurement and estimation techniques to extend the range and improve measurements. Several state-of-the-art models were evaluated using a novel dataset composed of data from a simulated rainforest environment. In the evaluation, MiDaS outperformed the other models, and a segmentation pipeline was designed using this model to identify the highest areas of the canopies. The segmentation pipeline was evaluated using 1080p and 360p input videos from the simulated rainforest dataset. It was able to achieve an IoU of 0.848 and 0.826 and an F1 score of 0.915 and 0.902 at each resolution, respectively. We incorporated the proposed depth-estimation-based segmentation pipeline into an example application and deployed it on an edge system. Experimental results display the capabilities of a UAS using the segmentation pipeline for rainforest data collection.

关键词： deep learning for visual perception Vision-Based Navigation Aerial Systems: perception and Autonomy

来源：评论

学校读者我要写书评

暂无评论

A Real 3D Embodied Dataset for Robotic Active visual learning

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 6646-6652页

作者： Zhao, Qianfan Zhang, Lu Wu, Lingxi Qiao, Hong Liu, Zhiyong Chinese Acad Sci Inst Automat State Key Lab Management & Control Complex Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Artificial Intelligence Beijing 100190 Peoples R China Chinese Acad Sci Ctr Excellence Brain Sci & Intelligence Technol Shanghai 200031 Peoples R China Univ Calif Santa Barbara Dept Math Coll Letters & Sci Santa Barbara CA 93106 USA

Active interaction with environments is one of the striking characteristics of robotic active vision, which allows robots to move to facilitate visual tasks. Recently, several embodied AI platforms have been proposed as the synthetic environments to study robotic active vision, without needing to interact in real world. However, by using synthetic data, model trained on these platforms will suffer performance degradation when applied in reality. In this letter, a real 3D embodied dataset is proposed for robotic active visual learning. The proposed dataset consists of real point cloud data densely collected in 7 real-world indoor scenes. In our embodied dataset, researchers are able to simulate the movements and interactions of robots in indoor environments and obtain real visual data, which will not lead to performance degradation in reality. Furthermore, we proposed a 3D divergency policy that can guide robots to move and collect data to improve visual performance in novel environments. The proposed policy is designed following a simple fact: a good 3D detector should produce consistent 3D detection results for the same object from different viewpoints. Therefore, our policy encourages the robot to explore the area where the detector generates different 3D bounding boxes for the same object and helps the robot improve its visual performance in novel scenes.

关键词： Data sets for robotic vision deep learning for visual perception reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Robust Sim2Real 3D Object Classification Using Graph Representations and a deep Center Voting Scheme

引用

IEEE ROBOTICS AND AUTOMATION LETTERS 2022年第3期7卷 8028-8035页

作者： Weibel, Jean-Baptiste Patten, Timothy Vincze, Markus TU Wien Vis Robot Lab Automat & Control Inst A-1040 Vienna Austria Univ Technol Sydney Fac Engn & Informat Technol Robot Inst Ultimo 2007 Australia

While object semantic understanding is essential for service robotic tasks, 3D object classification is still an open problem. learning from artificial 3D models alleviates the cost of the annotation necessary to approach this problem, but today's methods still struggle with the differences between artificial and real 3D data. We conjecture that one of the causes of this issue is the fact that today's methods learn directly from point coordinates, which makes them highly sensitive to scale changes. We propose to learn from a graph of reproducible object parts whose scale is more reliable. In combination with a voting scheme, our approach achieves significantly more robust classification and improves upon state-of-the-art by up to 16% when transferring from artificial to real objects.

关键词： deep learning for visual perception recognition visual learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：