检索结果-内蒙古大学图书馆

Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation

学校读者我要写书评

暂无评论

Fourier Prompt Tuning for Modality-Incomplete Scene Segmenta...

IEEE Symposium on Intelligent Vehicle

作者： Ruiping Liu Jiaming Zhang Kunyu Peng Yufan Chen Ke Cao Junwei Zheng M. Saquib Sarfraz Kailun Yang Rainer Stiefelhagen Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Germany Mercedes-Benz Tech Innovation Germany School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

ISBN: (数字)9798350348811

ISBN: (纸本)9798350348828

关键词： Training Rain Source coding Semantic segmentation Semantics Switches Benchmark testing

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jiao, Jianbin Cheng, Xina Chen, Weijie Yin, Xiaoting Shi, Hao Yang, Kailun School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as human-computer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable self-attention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multi-perspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves state-of-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose. Copyright © 2024, The Authors. All rights reserved.

关键词： Human computer interaction

Towards Single-Lens controllable Depth-of-Field Imaging via Depth-Aware Point Spread Functions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qian, Xiaolong Jiang, Qi Gao, Yao Gao, Shaohua Yi, Zhonghua Sun, Lei Wei, Kai Li, Haifeng Yang, Kailun Wang, Kaiwei Bai, Jian State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

controllable Depth-of-Field (DoF) imaging commonly produces amazing visual effects based on heavy and expensive high-end lenses. However, confronted with the increasing demand for mobile scenarios, it is desirable to achieve a lightweight solution with Minimalist Optical Systems (MOS). This work centers around two major limitations of MOS, i.e., the severe optical aberrations and uncontrollable DoF, for achieving single-lens controllable DoF imaging via computational methods. A Depth-aware controllable DoF Imaging (DCDI) framework is proposed equipped with All-in-Focus (AiF) aberration correction and monocular depth estimation, where the recovered image and corresponding depth map are utilized to produce imaging results under diverse DoFs of any high-end lens via patch-wise convolution. To address the depth-varying optical degradation, we introduce a Depth-aware Degradation-adaptive Training (DA2T) scheme. At the dataset level, a Depth-aware Aberration MOS (DAMOS) dataset is established based on the simulation of Point Spread Functions (PSFs) under different object distances. Additionally, we design two plug-and-play depth-aware mechanisms to embed depth information into the aberration image recovery for better tackling depth-aware degradation. Furthermore, we propose a storage-efficient Omni-Lens-Field model to represent the 4D PSF library of various lenses. With the predicted depth map, recovered image, and depth-aware PSF map inferred by Omni-Lens-Field, single-lens controllable DoF imaging is achieved. To the best of our knowledge, we are the first to explore the single-lens controllable DoF imaging solution. Comprehensive experimental results demonstrate that the proposed framework enhances the recovery performance, and attains impressive single-lens controllable DoF imaging results, providing a seminal baseline for this field. The source code and the established dataset will be publicly available at https://***/XiaolongQian/DCDI. Copyright © 2024, The A

关键词： Optical transfer function

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

学校读者我要写书评

暂无评论

Towards Precise 3D Human Pose Estimation with Multi-Perspect...

International Joint Conference on Neural Networks (IJCNN)

作者： Jianbin Jiao Xina Cheng Weijie Chen Xiaoting Yin Hao Shi Kailun Yang School of Artificial Intelligence Xidian University China State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

3D human pose estimation captures the human joint points in three-dimensional space while keeping the depth information and physical structure. That is essential for applications that require precise pose information, such as humancomputer interaction, scene understanding, and rehabilitation training. Due to the challenges in data collection, mainstream datasets of 3D human pose estimation are primarily composed of multi-view video data collected in laboratory environments, which contains rich spatial-temporal correlation information besides the image frame content. Given the remarkable selfattention mechanism of transformers, capable of capturing the spatial-temporal correlation from multi-view video datasets, we propose a multi-stage framework for 3D sequence-to-sequence (seq2seq) human pose detection. Firstly, the spatial module represents the human pose feature by intra-image content, while the frame-image relation module extracts temporal relationships and 3D spatial positional relationship features between the multiperspective images. Secondly, the self-attention mechanism is adopted to eliminate the interference from non-human body parts and reduce computing resources. Our method is evaluated on Human3.6M, a popular 3D human pose detection dataset. Experimental results demonstrate that our approach achieves stateof-the-art performance on this dataset. The source code will be available at https://***/WUJINHUAN/3D-human-pose.

关键词： Training Three-dimensional displays Correlation Computational modeling Source coding Pose estimation Video sequences

Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gao, Yao Jiang, Qi Gao, Shaohua Sun, Lei Yang, Kailun Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China School of Robotics Hunan University Changsha410012 China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China

Recently, joint design approaches that simultaneously optimize optical systems and downstream algorithms through data-driven learning have demonstrated superior performance over traditional separate design approaches. However, current joint design approaches heavily rely on the manual identification of initial lenses, posing challenges and limitations, particularly for compound lens systems with multiple potential starting points. In this work, we present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications;and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution in all search results. Extensive experimental results illustrate that QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability, which automatically provides compound lens based computational imaging systems with higher imaging quality compared to existing paradigms. The source code will be made publicly available at https://***/LiGpy/QGSO. Copyright © 2024, The Authors. All rights reserved.

关键词： Imaging systems

Beyond the Field-of-View: Enhancing Scene Visibility and perception with Clip-Recurrent Transformer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shi, Hao Jiang, Qi Yang, Kailun Yin, Xiaoting Wang, Ze Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation the National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd Shanghai201210 China

Vision sensors are widely applied in vehicles, robots, and roadside infrastructure. However, due to limitations in hardware cost and system size, camera Field-of-View (FoV) is often restricted and may not provide sufficient coverage. Nevertheless, from a spatiotemporal perspective, it is possible to obtain information beyond the camera’s physical FoV from past video streams. In this paper, we propose the concept of online video inpainting for autonomous vehicles to expand the field of view, thereby enhancing scene visibility, perception, and system safety. To achieve this, we introduce the FlowLens architecture, which explicitly employs optical flow and implicitly incorporates a novel clip-recurrent transformer for feature propagation. FlowLens offers two key features: 1) FlowLens includes a newly designed Clip-Recurrent Hub with 3D-Decoupled Cross Attention (DDCA) to progressively process global information accumulated over time. 2) It integrates a multi-branch Mix Fusion Feed Forward Network (MixF3N) to enhance the precise spatial flow of local features. To facilitate training and evaluation, we derive the KITTI360 dataset with various FoV mask, which covers both outer- and inner FoV expansion scenarios. We also conduct both quantitative assessments and qualitative comparisons of beyond-FoV semantics and beyond-FoV object detection across different models. We illustrate that employing FlowLens to reconstruct unseen scenes even enhances perception within the field of view by providing reliable semantic context. Extensive experiments and user studies involving offline and online video inpainting, as well as beyond-FoV perception tasks, demonstrate that FlowLens achieves state-of-the-art performance. The source code and dataset are made publicly available at https://github. com/MasterHow/FlowLens. Copyright © 2022, The Authors. All rights reserved.

关键词： Optical flows

Skeleton-Based Human Action Recognition with Noisy Labels

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xu, Yi Peng, Kunyu Wen, Di Liu, Ruiping Zheng, Junwei Chen, Yufan Zhang, Jiaming Roitberg, Alina Yang, Kailun Stiefelhagen, Rainer Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Germany Institute for Artificial Intelligence University of Stuttgart Germany School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

Understanding human actions from body poses is critical for assistive robots sharing space with humans in order to make informed and safe decisions about the next interaction. However, precise temporal localization and annotation of activity sequences is time-consuming and the resulting labels are often noisy. If not effectively addressed, label noise negatively affects the model’s training, resulting in lower recognition quality. Despite its importance, addressing label noise for skeleton-based action recognition has been overlooked so far. In this study, we bridge this gap by implementing a framework that augments well-established skeleton-based human action recognition methods with label-denoising strategies from various research areas to serve as the initial benchmark. Observations reveal that these baselines yield only marginal performance when dealing with sparse skeleton data. Consequently, we introduce a novel methodology, NoiseEraSAR, which integrates global sample selection, co-teaching, and Cross-Modal Mixture-of-Experts (CM-MOE) strategies, aimed at mitigating the adverse impacts of label noise. Our proposed approach demonstrates better performance on the established benchmark, setting new state-of-the-art standards. The source code for this study is accessible at https://***/xuyizdby/NoiseEraSAR. Copyright © 2024, The Authors. All rights reserved.

关键词： Musculoskeletal system

LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Ze Yang, Kailun Shi, Hao Li, Peng Gao, Fei Bai, Jian Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China Huzhou Institute of Zhejiang University Zhejiang University China

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using (u, v, 1)T become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a visual Inertial SLAM framework for cameras with extremely Large FoV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method Copyright © 2022, The Authors. All rights reserved.

关键词： Cameras

A Graph-Based SLAM Method Assisted by visual Marker in the Degenerate Scenes

学校读者我要写书评

暂无评论

A Graph-Based SLAM Method Assisted by Visual Marker in the D...

International Conference on Advanced robotics and Mechatronics (ICARM)

作者： Jieqingxin Zhang Hui Zhang Xidong Zhou Bo Chen Xiangchuan Wang Tao Xu College of Electrical and Information Engineering Changsha University of Science and Technology Hunan China College of Robotics also with Robot Visual Perception and Control Technology National Engineering Research Center Hunan University Hunan China College of Electrical and Information Engineering Hunan University Hunan China

In the post-COVID-19 pandemic era, hospitals and other places have an urgent need for mobile robots with autonomous disinfection ability, and robots need to complete SLAM tasks to realize autonomous navigation. Lidar is widely used for indoor SLAM. However, due to the lack of geometric structure in the indoor environment, two-dimensional lidar information degrades, rendering the robot unable to obtain effective positioning. Therefore, we leverage the easy identification and high robustness of ARTag to fuse vision and range sensor information. We introduce ARTag as visual marker to assist positioning, establish observation window to screen the acquired ARTag pose. We employ the pose graph optimization method to optimize the visual markers and laser scanning results in the back end. This reduces the positioning errors caused by the degradation of lidar information and reduces the frequency of optimization by improving the back end optimization strategy. This method is applied in a UltraViolet C (UVC) Disinfection robot experiment. Experimental results show that our method effectively improves the positioning accuracy and robustness of the robot in the environments with degraded laser information.

关键词：