检索结果-内蒙古大学图书馆

Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Jiazheng Chen, Xiang Zhang, Yuxi Liu, Min Wang, Yaonan Zhang, Hang College of Electrical and Information Engineering Hunan University Hunan Changsha China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Hunan Changsha China Cornell University United States

Surgical navigation based on multimodal image registration has played a significant role in providing intraoperative guidance to surgeons by showing the relative position of the target area to critical anatomical structures during surgery. However, due to the differences between multimodal images and intraoperative image deformation caused by tissue displacement and removal during the surgery, effective registration of preoperative and intraoperative multimodal images faces significant challenges. To address the multimodal image registration challenges in Learn2Reg 2024, an unsupervised multimodal medical image registration method based on multilevel correlation balanced optimization (MCBO) is designed to solve these problems. First, the features of each modality are extracted based on the modality independent neighborhood descriptor, and the multimodal images is mapped to the feature space. Second, a multilevel pyramidal fusion optimization mechanism is designed to achieve global optimization and local detail complementation of the deformation field through dense correlation analysis and weight-balanced coupled convex optimization for input features at different scales. For preoperative medical images in different modalities, the alignment and stacking of valid information between different modalities is achieved by the maximum fusion between deformation fields. Our method focuses on the ReMIND2Reg task in Learn2Reg 2024, and to verify the generality of the method, we also tested it on the COMULIS3DCLEM task. Based on the results, our method achieved second place in the validation of both two tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： Convex optimization

Materobot: Material Recognition in Wearable robotics for People with visual Impairments

学校读者我要写书评

暂无评论

MateRobot: Material Recognition in Wearable Robotics for Peo...

IEEE International Conference on robotics and Automation (ICRA)

作者： Junwei Zheng Jiaming Zhang Kailun Yang Kunyu Peng Rainer Stiefelhagen Institute for Robotics and Anthropomatics Karlsruhe Institute of Technology Karlsruhe Germany Department of Engineering Science University of Oxford UK School of Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha China

ISBN: (数字)9798350384574

ISBN: (纸本)9798350384581

People with visual Impairments (PVI) typically recognize objects through haptic perception. Knowing objects and materials before touching is desired by the target users but under-explored in the field of human-centered robotics. To fill this gap, in this work, a wearable vision-based robotic system, MATErobot, is established for PVI to recognize materials and object categories beforehand. To address the computational constraints of mobile platforms, we propose a lightweight yet accurate model MATEViT to perform pixel-wise semantic segmentation, simultaneously recognizing both objects and materials. Our methods achieve respective 40.2% and 51.1% of mIoU on COCOStuff-10K and DMS datasets, surpassing the previous method with +5.7% and +7.0% gains. Moreover, on the field test with participants, our wearable system reaches a score of 28 in the NASA-Task Load Index, indicating low cognitive demands and ease of use. Our MATErobot demonstrates the feasibility of recognizing material property through visual cues and offers a promising step towards improving the functionality of wearable robots for PVI. The source code has been made publicly available at MATErobot.

关键词： visualization Source coding Semantic segmentation visual impairment Semantics Wearable robots Mobile applications

SF-TIM: A Simple Framework for Enhancing Quadrupedal robot Jumping Agility by Combining Terrain Imagination and Measurement

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Ze Li, Yang Xu, Long Shi, Hao Ma, Zunwang Chu, Zhen Li, Chao Gao, Fei Yang, Kailun Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University China School of Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China DeepRobotics Co. Ltd. China

Dynamic jumping on high platforms and over gaps differentiates legged robots from wheeled counterparts. Compared to walking on rough terrains, dynamic locomotion on abrupt surfaces requires fusing proprioceptive and exteroceptive perception for explosive movements. In this paper, we propose SF-TIM (Simple Framework combining Terrain Imagination and Measurement), a single-policy method that enhances quadrupedal robot jumping agility, while preserving their fundamental blind walking capabilities. In addition, we introduce a terrain-guided reward design specifically to assist quadrupedal robots in high jumping, improving their performance in this task. To narrow the simulation-to-reality gap in quadrupedal robot learning, we introduce a stable and high-speed elevation map generation framework, enabling zero-shot simulation-to-reality transfer of locomotion ability. Our algorithm has been deployed and validated on both the small-/large-size quadrupedal robots, demonstrating its effectiveness in real-world applications: the robot has successfully traversed various high platforms and gaps, showing the robustness of our proposed approach. A demo video has been made available at https://***/SF-TIM. Copyright © 2024, The Authors. All rights reserved.

关键词： Multipurpose robots

PanoVPR: Towards Unified Perspective-to-Equirectangular visual Place Recognition via Sliding Windows across the Panoramic View

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shi, Ze Shi, Hao Yang, Kailun Yin, Zhe Lin, Yining Wang, Kaiwei State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou China School of Robotics and with the National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha China Shanghai SUPREMIND Technology Co Ltd Shanghai China

— visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose PanoVPR, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at PanoVPR. Copyright © 2023, The Authors. All rights reserved.

关键词： Query processing

Safety-Critical control of Quadrotor UAVs with control Barrier Functions

学校读者我要写书评

暂无评论

Safety-Critical Control of Quadrotor UAVs with Control Barri...

IEEE International Conference on robotics and Biomimetics

作者： Tianyi Yang Zhiqiang Miao Guo Yi Yaonan Wang College of Electrical and Information Engineering Hunan University Chang-sha China National Engineering Research Center of Robot Visual Perception and Control Changsha China School of Information and Electrical Engineering Hunan University of Science and Technology Changsha China

ISBN: (纸本)9781665481106

For quadrotors, imposing multiple dynamic constraints on the state simultaneously to achieve safe control is a challenging problem. In this paper, a cascaded control archi-tecture based on quadratic programming method is proposed to generate minimally-invasive and collision-free control actions. This architecture consists of exponential control barrier functions(ECBFs) to construct a non-conservative forward invariant safety region and geometric nonlinear PID attitude control with considering quadrotor dynamics to avoid the singularities of Euler-angles and the ambiguity of quaternions. The feasibility and the effectiveness of the proposed cascaded control architecture is demonstrated through numerical simulations.

关键词： Attitude control Quaternions Biomimetics Numerical simulation Safety Quadratic programming robots

OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Teng, Fei Zhang, Jiaming Peng, Kunyu Wang, Yaonan Stiefelhagen, Rainer Yang, Kailun The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany The Institute for Visual Computing ETH Zurich Zurich8092 Switzerland

Light field cameras are capable of capturing intricate angular and spatial details. This allows for acquiring complex light patterns and details from multiple angles, significantly enhancing the precision of image semantic segmentation. However, two significant issues arise: (1) The extensive angular information of light field cameras contains a large amount of redundant data, which is overwhelming for the limited hardware resources of intelligent agents. (2) A relative displacement difference exists in the data collected by different micro-lenses. To address these issues, we propose an Omni-Aperture Fusion model (OAFuser) that leverages dense context from the central view and extracts the angular information from sub-aperture images to generate semantically consistent results. To simultaneously streamline the redundant information from the light field cameras and avoid feature loss during network propagation, we present a simple yet very effective Sub-Aperture Fusion Module (SAFM). This module efficiently embeds sub-aperture images in angular features, allowing the network to process each sub-aperture image with a minimal computational demand of only (∼1GFlops). Furthermore, to address the mismatched spatial information across viewpoints, we present a Center Angular Rectification Module (CARM) to realize feature resorting and prevent feature occlusion caused by misalignment. The proposed OAFuser achieves state-of-the-art performance on four UrbanLF datasets in terms of all evaluation metrics and sets a new record of 84.93% in mIoU on the UrbanLF-Real Extended dataset, with a gain of +3.69%. The source code for OAFuser is available at https://***/FeiBryantkit/OAFuser. Impact Statement-To solve the data abundance problem, we have reduced the significant computational consumption of light field cameras while not introducing any additional parameters. The proposed method has practical value for the deployment and application of light field cameras. The proposed

关键词： Semantic Segmentation

PanoVpr: Towards Unified Perspective-to-Equirectangular visual Place Recognition via Sliding Windows Across the Panoramic View

学校读者我要写书评

暂无评论

PanoVpr: Towards Unified Perspective-to-Equirectangular Visu...

International Conference on Intelligent Transportation

作者： Ze Shi Hao Shi Kailun Yang Zhe Yin Yining Lin Kaiwei Wang State Key Laboratory of Modern Optical Instrumentation Zhejiang University Hangzhou China Shanghai SUPREMIND Technology Co. Ltd Shanghai China National Engineering Research Center of Robot Visual Perception and Control Technology School of Robotics Hunan University Changsha China

visual place recognition has gained significant attention in recent years as a crucial technology in autonomous driving and robotics. Currently, the two main approaches are the perspective view retrieval (P2P) paradigm and the equirectangular image retrieval (E2E) paradigm. However, it is practical and natural to assume that users only have consumer-grade pinhole cameras to obtain query perspective images and retrieve them in panoramic database images from map providers. To address this, we propose PanoVPR, a perspective-to-equirectangular (P2E) visual place recognition framework that employs sliding windows to eliminate feature truncation caused by hard cropping. Specifically, PanoVPR slides windows over the entire equirectangular image and computes feature descriptors for each window, which are then compared to determine place similarity. Notably, our unified framework enables direct transfer of the backbone from P2P methods without any modification, supporting not only CNNs but also Transformers. To facilitate training and evaluation, we derive the Pitts250k-P2E dataset from the Pitts250k and establish YQ360, latter is the first P2E visual place recognition dataset collected by a mobile robot platform aiming to simulate real-world task scenarios better. Extensive experiments demonstrate that PanoVPR achieves state-of-the-art performance and obtains 3.8% and 8.0% performance gain on Pitts250k-P2E and YQ360 compared to the previous best method, respectively. Code and datasets will be publicly available at PanoVPR.

关键词：

Full Point Encoding for Local Feature Aggregation in 3D Point Clouds

学校读者我要写书评

暂无评论

arXiv 2023年

作者： He, Yong Yu, Hongshan Yang, Zhengeng Liu, Xiaoyan Sun, Wei Mian, Ajmal National Engineering Laboratory for Robot Visual Perception and Control Technology College of Electrical and Information Engineering Hunan University Lushan South Rd. Yuelu Dist. Changsha410082 China Department of Computer Science The University of Western Australia WA6009 Australia

Point cloud processing methods exploit local point features and global context through aggregation which does not explicity model the internal correlations between local and global features. To address this problem, we propose full point encoding which is applicable to convolution and transformer architectures. Specifically, we propose Full Point Convolution (FPConv) and Full Point Transformer (FPTransformer) architectures. The key idea is to adaptively learn the weights from local and global geometric connections, where the connections are established through local and global correlation functions respectively. FPConv and FPTransformer simultaneously model the local and global geometric relationships as well as their internal correlations, demonstrating strong generalization ability and high performance. FPConv is incorporated in classical hierarchical network architectures to achieve local and global shape-aware learning. In FPTransformer, we introduce full point position encoding in self-attention, that hierarchically encodes each point position in the global and local receptive field. We also propose a shape aware downsampling block which takes into account the local shape and the global context. Experimental comparison to existing methods on benchmark datasets show the efficacy of FPConv and FPTransformer for semantic segmentation, object detection, classification, and normal estimation tasks. In particular, we achieve state-of-the-art semantic segmentation results of 76% mIoU on S3DIS 6-fold and 72.2% on S3DIS Area 5. Copyright © 2023, The Authors. All rights reserved.

关键词： Convolution

Skeleton-Based Human Action Recognition with Noisy Labels

学校读者我要写书评

暂无评论

Skeleton-Based Human Action Recognition with Noisy Labels

IEEE/RSJ International Conference on Intelligent robots and Systems (IROS)

作者： Yi Xu Kunyu Peng Di Wen Ruiping Liu Junwei Zheng Yufan Chen Jiaming Zhang Alina Roitberg Kailun Yang Rainer Stiefelhagen Anthropomatics and Robotics Karlsruhe Institute of Technology Germany Institute for Artificial Intelligence University of Stuttgart Germany School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Understanding human actions from body poses is critical for assistive robots sharing space with humans in order to make informed and safe decisions about the next interaction. However, precise temporal localization and annotation of activity sequences is time-consuming and the resulting labels are often noisy. If not effectively addressed, label noise negatively affects the model’s training, resulting in lower recognition quality. Despite its importance, addressing label noise for skeleton-based action recognition has been overlooked so far. In this study, we bridge this gap by implementing a framework that augments well-established skeleton-based human action recognition methods with label-denoising strategies from various research areas to serve as the initial benchmark. Observations reveal that these baselines yield only marginal performance when dealing with sparse skeleton data. Consequently, we introduce a novel methodology, NoiseEraSAR, which integrates global sample selection, co-teaching, and Cross-Modal Mixture-of-Experts (CM-MOE) strategies, aimed at mitigating the adverse impacts of label noise. Our proposed approach demonstrates better performance on the established benchmark, setting new state-of-the-art standards. The source code for this study will be made accessible at https://***/xuyizdby/NoiseEraSAR.

关键词： Training Source coding Noise Training data Benchmark testing Skeleton Noise measurement Human activity recognition Streams Standards