检索结果-内蒙古大学图书馆

GenMapping: Unleashing the Potential of Inverse Perspective Mapping for Robust Online HD Map Construction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Siyu Yang, Kailun Shi, Hao Wang, Song Yao, You Li, Zhiyong The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University Hangzhou310027 China The College of Computer Science Zhejiang University Hangzhou310027 China Shanghai Supremind Technology Company Ltd. Shanghai201210 China The USC Viterbi School of Engineering The University of Southern California Los AngelesCA90089 United States

Online High-Definition (HD) maps have emerged as the preferred option for autonomous driving, overshadowing the counterpart offline HD maps due to flexible update capability and lower maintenance costs. However, contemporary online HD map models embed parameters of visual sensors into training, resulting in a significant decrease in generalization performance when applied to visual sensors with different parameters. Inspired by the inherent potential of Inverse Perspective Mapping (IPM), where camera parameters are decoupled from the training process, we have designed a universal map generation framework, GenMapping. The framework is established with a triadic synergy architecture, including principal and dual auxiliary branches. When faced with a coarse road image with local distortion translated via IPM, the principal branch learns robust global features under the state space models. The two auxiliary branches are a dense perspective branch and a sparse prior branch. The former exploits the correlation information between static and moving objects, whereas the latter introduces the prior knowledge of OpenStreetMap (OSM). The triple-enhanced merging module is crafted to synergistically integrate the unique spatial features from all three branches. To further improve generalization capabilities, a Cross-View Map Learning (CVML) scheme is leveraged to realize joint learning within the common space. Additionally, a Bidirectional Data Augmentation (BiDA) module is introduced to mitigate reliance on datasets concurrently. A thorough array of experimental results shows that the proposed model surpasses current state-of-the-art methods in both semantic mapping and vectorized mapping, while also maintaining a rapid inference speed. Moreover, in cross-dataset experiments, the generalization of semantic mapping is improved by 17.3% in mIoU, while vectorized mapping is improved by 12.1% in mAP. The source code will be publicly available at https://***/lynn-yu/GenMappin

关键词： Photomapping

Multi-Expert Adversarial Attack Detection in Person Re-identification Using Context Inconsistency

学校读者我要写书评

暂无评论

Multi-Expert Adversarial Attack Detection in Person Re-ident...

International Conference on Computer Vision (ICCV)

作者： Xueping Wang Shasha Li Min Liu Yaonan Wang Amit K. Roy-Chowdhury College of Electrical and Information Engineering Hunan University China National Engineering Laboratory for Robot Visual Perception and Control Technology China University of California

ISBN: (纸本)9781665428132

The success of deep neural networks (DNNs) has promoted the widespread applications of person re-identification (ReID). However, ReID systems inherit the vulnerability of DNNs to malicious attacks of visually in-conspicuous adversarial perturbations. Detection of adversarial attacks is, therefore, a fundamental requirement for robust ReID systems. In this work, we propose a Multi-Expert Adversarial Attack Detection (MEAAD) approach to achieve this goal by checking context inconsistency, which is suitable for any DNN-based ReID systems. Specifically, three kinds of context inconsistencies caused by adversarial attacks are employed to learn a detector for distinguishing the perturbed examples, i.e., a) the embedding distances between a perturbed query person image and its top-K retrievals are generally larger than those between a benign query image and its top-K retrievals, b) the embedding distances among the top-K retrievals of a perturbed query image are larger than those of a benign query image, c) the top-K retrievals of a benign query image obtained with multiple expert ReID models tend to be consistent, which is not preserved when attacks are present. Extensive experiments on the Market1501 and DukeMTMC-ReID datasets show that, as the first adversarial attack detection approach for ReID, MEAAD effectively detects various adversarial attacks and achieves high ROC-AUC (over 97.5%).

关键词： Deep learning Computer vision Perturbation methods Neural networks Detectors Feature extraction Context modeling

Tackling Real-world Complexity: Hierarchical Modeling and Dynamic Prompting for Multimodal Long Document Classification

学校读者我要写书评

暂无评论

IEEE Transactions on Circuits and Systems for Video technology 2025年

作者： Liu, Tengfei Hu, Yongli Li, Mingjie Yi, Junfei Chang, Xiaojun Gao, Junbin Yin, Baocai Beijing University of Technology Beijing Key Laboratory of Multimedia and Intelligent Software Technology Beijing Institute of Artificial Intelligence School of Information Science and Technology Beijing100124 China Stanford University Department of Radiation Oncology StanfordCA94305 United States Hunan University National Engineering Laboratory of Robot Visual Perception and Control Technology College of Electrical and Information Engineering Hunan Changsha410082 China University of Technology Sydney ReLER Laboratory Australian Artificial Intelligence Institute Faculty of Engineering and Information Technology UltimoNSW2007 Australia The University of Sydney Business School The University of Sydney Discipline of Business Analytics CamperdownNSW2006 Australia

With the rapid growth of internet content, multimodal long document data has become increasingly prominent, drawing significant attention from researchers. However, most existing methods primarily focus on scenarios where all modalities are present, often overlooking more challenging and realistic cases involving missing image modality. To address this limitation, we propose a robust multimodal long document classification (MLDC) framework that integrates hierarchical modeling and dynamic prompting to handle complex multimodal long document data. Our approach begins by leveraging hierarchical modeling combined with an Adaptive Correlation Multimodal Transformer (ACMT) to effectively capture relationships between text and images at both section and sentence levels. We also introduce a Dynamic Prompt Generation (DPG) module at both levels to enhance the model's robustness in handling missing image data. By evaluating sample uncertainty, the DPG module dynamically adjusts both the number of prompts and the prompts themselves, allowing the model to better adapt to the varying needs of different samples. Finally, a Hierarchical Heterogeneous Graph (HHG) is introduced to enhance feature interactions across levels, further improving the coherence and accuracy of the model. Extensive experiments on four multi-modal long document datasets demonstrate that our model shows superior performance compared to existing state-of-the-art MLDC classification methods in various conditions. © 1991-2012 IEEE.

关键词： dynamic prompt generation hierarchical heterogeneous graph Multimodal long document classification

DTCLMapper: Dual Temporal Consistent Learning for Vectorized HD Map Construction

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Siyu Lin, Jiacheng Shi, Hao Zhang, Jiaming Wang, Song Yao, You Li, Zhiyong Yang, Kailun The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China The College of Computer Science and Electronic Engineering Hunan University Changsha410082 China The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany The College of Computer Science Zhejiang University Hangzhou310027 China The USC Viterbi School of Engineering The University of Southern California Los AngelesCA90089 United States

Temporal information plays a pivotal role in Bird’s-Eye-View (BEV) driving scene understanding, which can alleviate the visual information sparsity. However, the indiscriminate temporal fusion method will cause the barrier of feature redundancy when constructing vectorized High-Definition (HD) maps. In this paper, we revisit the temporal fusion of vectorized HD maps, focusing on temporal instance consistency and temporal map consistency learning. To improve the representation of instances in single-frame maps, we introduce a novel method, DTCLMapper. This approach uses a dual-stream temporal consistency learning module that combines instance embedding with geometry maps. In the instance embedding component, our approach integrates temporal Instance Consistency Learning (ICL), ensuring consistency from vector points and instance features aggregated from points. A vectorized points pre-selection module is employed to enhance the regression efficiency of vector points from each instance. Then aggregated instance features obtained from the vectorized points preselection module are grounded in contrastive learning to realize temporal consistency, where positive and negative samples are selected based on position and semantic information. The geometry mapping component introduces Map Consistency Learning (MCL) designed with self-supervised learning. The MCL enhances the generalization capability of our consistent learning approach by concentrating on the global location and distribution constraints of the instances. Extensive experiments on well-recognized benchmarks indicate that the proposed DTCLMapper achieves state-of-the-art performance in vectorized mapping tasks, reaching 61.9% and 65.1% mAP scores on the nuScenes and Argoverse datasets, respectively. The source code is available at https://***/lynn-yu/DTCLMapper. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Single-Leader-Dual-Follower Teleoperation in Object-Holding Task with Internal Force Regulation 26

学校读者我要写书评

暂无评论

Single-Leader-Dual-Follower Teleoperation in Object-Holding ...

26th International Conference on Automation and Computing, ICAC 2021

作者： Huang, Darong Jiang, Yiming Yang, Chenguang South China University of Technology School of Automation Science and Engineering Guangzhou510641 China Hunan University National Engineering Laboratory for Robot Visual Perception and Control Changsha410082 China University of the West of England Bristol Robotics Laboratory BristolBS16 1QY United Kingdom

ISBN: (纸本)9781860435577

robot teleoperation attracts growing attention of researchers in many domains. Plenty of factors contribute to the good performance of a smart teleoperation system, and one crucial factor is that it provides an environmental feedback for the operator. Hence, in this paper, we offer a solution to help operator to perceive the contact force of the tele-robot. In addition, we introduce a strategy to cooperatively teleoperate two follower robots by using only single leader1 robot, which helps to reduce the control burden for the operator, as well as enables the teleoperation system to avoid the problem caused by the mismatched degree of freedom (DoF). Moreover, an impedance model based technique is employed to regulate the holding internal force during object-holding tasks. Experiments are conducted gradually, using a Touch X device as the leader robot and two arms of the Baxter robot as the tele-robots. Experimental results validate the feasibility of the proposed teleoperation system. © 2021 Chinese Automation and Computing Society in the UK-CACSUK.

关键词： Remote control

LF-VISLAM: A SLAM Framework for Large Field-of-View Cameras with Negative Imaging Plane on Mobile Agents

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Ze Yang, Kailun Shi, Hao Li, Peng Gao, Fei Bai, Jian Wang, Kaiwei State Key Laboratory of Extreme Photonics and Instrumentation Zhejiang University China School of Robotics Hunan University China National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University China State Key Laboratory of Industrial Control Technology Zhejiang University China Huzhou Institute of Zhejiang University Zhejiang University China

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using (u, v, 1)T become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a visual Inertial SLAM framework for cameras with extremely Large FoV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method Copyright © 2022, The Authors. All rights reserved.

关键词： Cameras

Multi-Expert Adversarial Attack Detection in Person Re-identification Using Context Inconsistency

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Xueping Li, Shasha Liu, Min Wang, Yaonan Roy-Chowdhury, Amit K. College of Electrical and Information Engineering Hunan University China National Engineering Laboratory for Robot Visual Perception and Control Technology China University of California Riverside United States

The success of deep neural networks (DNNs) has promoted the widespread applications of person reidentification (ReID). However, ReID systems inherit the vulnerability of DNNs to malicious attacks of visually inconspicuous adversarial perturbations. Detection of adversarial attacks is, therefore, a fundamental requirement for robust ReID systems. In this work, we propose a Multi-Expert Adversarial Attack Detection (MEAAD) approach to achieve this goal by checking context inconsistency, which is suitable for any DNN-based ReID systems. Specifically, three kinds of context inconsistencies caused by adversarial attacks are employed to learn a detector for distinguishing the perturbed examples, i.e., a) the embedding distances between a perturbed query person image and its top-K retrievals are generally larger than those between a benign query image and its top-K retrievals, b) the embedding distances among the top-K retrievals of a perturbed query image are larger than those of a benign query image, c) the top-K retrievals of a benign query image obtained with multiple expert ReID models tend to be consistent, which is not preserved when attacks are present. Extensive experiments on the Market1501 and DukeMTMC-ReID datasets show that, as the first adversarial attack detection approach for ReID, MEAAD effectively detects various adversarial attacks and achieves high ROC-AUC (over 97.5%). Copyright © 2021, The Authors. All rights reserved.

关键词： Deep neural networks

Computational Imaging for Machine perception: Transferring Semantic Segmentation beyond Aberrations

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jiang, Qi Shi, Hao Gao, Shaohua Zhang, Jiaming Yang, Kailun Sun, Lei Ni, Huajian Wang, Kaiwei The State Key Laboratory of Extreme Photonics and Instrumentation The National Engineering Research Center of Optical Instrumentation Zhejiang University Hangzhou310027 China The Institute for Anthropomatics and Robotics Karlsruhe Institute of Technology Karlsruhe76131 Germany The School of Robotics The National Engineering Research Center of Robot Visual Perception and Control Technology Hunan University Changsha410082 China Shanghai SUPREMIND Technology Company Ltd Shanghai201210 China

Semantic scene understanding with Minimalist Optical Systems (MOS) in mobile and wearable applications remains a challenge due to the corrupted imaging quality induced by optical aberrations. However, previous works only focus on improving the subjective imaging quality through the Computational Imaging (CI) technique, ignoring the feasibility of advancing semantic segmentation. In this paper, we pioneer the investigation of Semantic Segmentation under Optical Aberrations (SSOA) with MOS. To benchmark SSOA, we construct Virtual Prototype Lens (VPL) groups through optical simulation, generating Cityscapes-ab and KITTI-360-ab datasets under different behaviors and levels of aberrations. We look into SSOA via an unsupervised domain adaptation perspective to address the scarcity of labeled aberration data in real-world scenarios. Further, we propose Computational Imaging Assisted Domain Adaptation (CIADA) to leverage prior knowledge of CI for robust performance in SSOA. Based on our benchmark, we conduct experiments on the robustness of classical segmenters against aberrations. In addition, extensive evaluations of possible solutions to SSOA reveal that CIADA achieves superior performance under all aberration distributions, bridging the gap between computational imaging and downstream applications for MOS. The project page is at https://***/zju-jiangqi/CIADA. Copyright © 2022, The Authors. All rights reserved.

关键词： Aberrations

FlowDriveNet: An End-to-End Network for Learning Driving Policies from Image Optical Flow and LiDAR Point Flow

学校读者我要写书评

暂无评论

FlowDriveNet: An End-to-End Network for Learning Driving Pol...

IEEE International Conference on robotics and Automation (ICRA)

作者： Shuai Wang Jiahu Qin Menglin Li Yaonan Wang University of Science and Technology of China Hefei China College of Electrical and Information Engineering Hunan University Changsha China National Engineering Laboratory for Robot Visual Perception and Control Technology Changsha China

Learning driving policies using an end-to-end network has been proved a promising solution for autonomous driving. Due to the lack of a benchmark driver behavior dataset that contains both the visual and the LiDAR data, existing works solely focus on learning driving from visual sensors. Besides, most works are limited to predict steering angle yet neglect the more challenging vehicle speed control problem. In this paper, we propose a novel end-to-end network, FlowDriveNet, which takes advantages of sequential visual data and LiDAR data jointly to predict steering angle and vehicle speed. The main challenges of this problem are how to efficiently extract driving-related information from images and point clouds, and how to fuse them effectively. To tackle these challenges, we propose a concept of point flow and declare that image optical flow and LiDAR point flow are significant motion cues for driving policy learning. Specifically, we first create an enhanced dataset that consists of images, point clouds and corresponding human driver behaviors. Then, in FlowDriveNet, a deep but efficient visual feature extraction module and a point feature extraction module are utilized to extract spatial features from optical flow and point flow, respectively. Additionally, a novel temporal fusion and prediction module is designed to fuse temporal information from the extracted spatial feature sequences and predict vehicle driving commands. Finally, a series of ablation experiments verify the importance of optical flow and point flow and comparison experiments show that our flow-based method outperforms the existing image-based approaches on the task of driving policy learning.

关键词： Image motion analysis Computer vision visualization Laser radar Fuses Predictive models Feature extraction