检索结果-内蒙古大学图书馆

Domain Adaptation in visual Reinforcement Learning via Self-Expert Imitation with Purifying Latent Feature

学校读者我要写书评

暂无评论

Domain Adaptation in Visual Reinforcement Learning via Self-...

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Lin Chen Jianan Huang Zhen Zhou Yaonan Wang Yang Mo Zhiqiang Miao Kai Zeng Mingtao Feng Danwei Wang School of Electrical and Information Engineering Hunan University Changsha China National Engineering Research Center for Robot Visual Perception and Control Technology Changsha China School of Computer Science and Technology Xidian University Xian China School of Electrical and Electrical Engineering Nanyang Technological University Singapore

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Generalizing visual reinforcement learning is fundamental to robot visual navigation, involving the acquisition of a policy from interactions with source environments to facilitate adaptation to analogous, yet unfamiliar target environments. Recent advancements capitalize on data augmentation techniques, self-supervised learning methods, and the generative adversarial network framework to train policy neural networks with enhanced generalizability. However, current methods, upon extracting domain-general latent features, further utilize these features to train the reinforcement learning policy, resulting in a decline in the performance of the learned policy guiding the agent to accomplish tasks. To tackle these challenges, a framework of self-expert imitation with purifying latent features was devised, empowering the policy to achieve robust and stable zero-shot generalization performance in visually similar domains previously unseen, without diminishing the performance of guiding the agent to accomplish tasks. The extraction method of domain-general latent features is proposed to enhance their quality based on the variational autoencoder. Extensive experiments have shown that our policy, compared with state-of-the-art counterparts, does not diminish the performance of the policy guiding the agent to accomplish tasks after generalization.

关键词： Training visualization Navigation Semantic segmentation Autoencoders Neural networks Reinforcement learning Self-supervised learning Feature extraction Intelligent robots

PanoVPR: Towards Unified Perspective-to-Equirectangular visual Place Recognition via Sliding Windows across the Panoramic View

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shi, Ze Shi, Hao Yang, Kailun Yin, Zhe Lin, Yining Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou China School of Robotics and with the National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha China Shanghai SUPREMIND Technology Co Ltd Shanghai China

— visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose PanoVPR, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at PanoVPR. Copyright © 2023, The Authors. All rights reserved.

关键词： Query processing

Steady Tracker: Tracking a Target Stably Using a Quadrotor

学校读者我要写书评

暂无评论

Steady Tracker: Tracking a Target Stably Using a Quadrotor

IEEE International Conference on robotics and Biomimetics

作者： Hongwen Li Hang Zhong Yongsheng Lv Jianjun Sha Yu Long Yaonan Wang Harbin Engineering University Harbin Qingdao Innovation and Development Center of Harbin Engineering University Qingdao China School of Robotics National Engineering Research Center for Robot Visual Perception and Control Hunan University College of Electrical and Information Engineering Hunan University Changsha China

ISBN: (纸本)9781665481106

Maneuvering target tracking of Unmanned Aerial Vehicle(UAV) in cluttered environments is a challenging issue owing to the unknown motion intention of the target and the complex moving environments. As the complexity of the environment increases, stable and secure target tracking is increasingly difficult to guarantee. To address the issue, this paper proposes a stable quadrotor tracking solution. The proposed solution contains two parts: target motion prediction and tracking path searching. The target motion prediction method predicts the future target motion based on the obtained target observations while considering observation noise and prediction errors. The tracking path searching method utilizes a sampling-based search method, using the homotopy of paths to ensure that the tracking path and the target position are in the same space. Finally, simulations, real-world experiments and statistical analysis verify the correctness and effectiveness of the proposed approach.

关键词： Target tracking Statistical analysis Prediction methods Search problems Turning Prediction algorithms Convex functions

PanoVpr: Towards Unified Perspective-to-Equirectangular visual Place Recognition via Sliding Windows Across the Panoramic View

学校读者我要写书评

暂无评论

PanoVpr: Towards Unified Perspective-to-Equirectangular Visu...

International Conference on Intelligent Transportation

作者： Ze Shi Hao Shi Kailun Yang Zhe Yin Yining Lin Kaiwei Wang State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou China Shanghai SUPREMIND Technology Co. Ltd Shanghai China National Engineering Research Center of Robot Visual Perception and Control Technology School of Robotics Hunan University Changsha China

visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose PanoVPR, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at PanoVPR.

关键词：

Towards Source-Free Domain Adaptive Semantic Segmentation Via Importance-Aware and Prototype-Contrast Learning

学校读者我要写书评

暂无评论

IEEE Transactions on Intelligent Vehicles

IEEE Transactions on Intelligent Vehicles 2024年 1-13页

作者： Cao, Yihong Zhang, Hui Lu, Xiao Xiao, Zheng Yang, Kailun Wang, Yaonan College of Computer Science and Electronic Engineering Hunan University Changsha China National Engineering Research Center of Robot Vision Perception and Control Technology School of Robotics Hunan University Changsha China College of Engineering and Design Hunan Normal University Changsha China

Domain adaptive semantic segmentation enables robust pixel- wise understanding in real-world driving scenes. Source-free domain adaptation, as a more practical technique, addresses the concerns of data privacy and storage limitations in typical unsupervised domain adaptation methods, making it especially relevant in the context of intelligent vehicles. It utilizes a well-trained source model and unlabeled target data to achieve adaptation in the target domain. However, in the absence of source data and target labels, current solutions cannot sufficiently reduce the impact of domain shift and fully leverage the information from the target data. In this paper, we propose an end-to-end source-free domain adaptation semantic segmentation method via Importance-Aware and Prototype-Contrast (IAPC) learning. The proposed IAPC framework effectively extracts domain-invariant knowledge from the well-trained source model and learns domain-specific knowledge from the unlabeled target domain. Specifically, considering the problem of domain shift in the prediction of the target domain by the source model, we put forward an importance-aware mechanism for the biased target prediction probability distribution to extract domain-invariant knowledge from the source model. We further introduce a prototype-contrast strategy, which includes a prototype-symmetric cross-entropy loss and a prototype-enhanced cross-entropy loss, to learn target intra-domain knowledge without relying on labels. A comprehensive variety of experiments on two domain adaptive semantic segmentation benchmarks demonstrates that the proposed end-to-end IAPC solution outperforms existing state-of-the-art methods. The source code is publicly available at https://***/yihong-97/Source-free-IAPC. IEEE

关键词： Semantics

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiao, Jianbin Cheng, Xina Chen, Weijie Yin, Xiaoting Shi, Hao Yang, Kailun School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as human-computer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable self-attention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multi-perspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves state-of-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose. Copyright © 2024, The Authors. All rights reserved.

关键词： Human computer interaction

Towards Single-Lens controllable Depth-of-Field Imaging via Depth-Aware Point Spread Functions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qian, Xiaolong Jiang, Qi Gao, Yao Gao, Shaohua Yi, Zhonghua Sun, Lei Wei, Kai Li, Haifeng Yang, Kailun Wang, Kaiwei Bai, Jian State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. To the best of our knowledge, we are the first to explore the single-lens controllable DoF imaging solution. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at https://***/XiaolongQian/DCDI. Copyright © 2024, The A

关键词： Optical transfer function

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

Towards Precise 3D Human Pose Estimation with Multi-Perspect...

International Joint Conference on Neural Networks (IJCNN)

作者： Jianbin Jiao Xina Cheng Weijie Chen Xiaoting Yin Hao Shi Kailun Yang School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as humancomputer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable selfattention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multiperspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves stateof-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose.

关键词： Training Three-dimensional displays Correlation Computational modeling Source coding Pose estimation Video sequences

Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gao, Yao Jiang, Qi Gao, Shaohua Sun, Lei Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

Recently, joint design approaches that simultaneously optimize optical systems and downstream algorithms through data-driven learning have demonstrated superior performance over traditional separate design approaches. However, current joint design approaches heavily rely on the manual identification of initial lenses, posing challenges and limitations, particularly for compound lens systems with multiple potential starting points. In this work, we present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications;and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution in all search results. Extensive experimental results illustrate that QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability, which automatically provides compound lens based computational imaging systems with higher imaging quality compared to existing paradigms. The source code will be made publicly available at https://***/LiGpy/QGSO. Copyright © 2024, The Authors. All rights reserved.

关键词： Imaging systems