检索结果-内蒙古大学图书馆

CREAM: Weakly Supervised Object Localization via Class RE-Activation Mapping

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Jilan Hou, Junlin Zhang, Yuejie Feng, Rui Zhao, Rui-Wei Zhang, Tao Lu, Xuequan Gao, Shang School of Computer Science Shanghai Key Lab of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University China Academy for Engineer and Technology Fudan University China Shanghai University of Finance and Economics China Deakin University Australia

Weakly Supervised Object Localization (WSOL) aims to localize objects with image-level supervision. Existing works mainly rely on Class Activation Mapping (CAM) derived from a classification model. However, CAM-based methods usually focus on the most discriminative parts of an object (i.e., incomplete localization problem). In this paper, we empirically prove that this problem is associated with the mixup of the activation values between less discriminative foreground regions and the background. To address it, we propose Class RE-Activation Mapping (CREAM), a novel clustering-based approach to boost the activation values of the integral object regions. To this end, we introduce class-specific foreground and background context embeddings as cluster centroids. A CAM-guided momentum preservation strategy is developed to learn the context embeddings during training. At the inference stage, the re-activation mapping is formulated as a parameter estimation problem under Gaussian Mixture Model, which can be solved by deriving an unsupervised Expectation-Maximization based soft-clustering algorithm. By simply integrating CREAM into various WSOL approaches, our method significantly improves their performance. CREAM achieves the state-of-the-art performance on CUB, ILSVRC and OpenImages benchmark datasets. Code will be available at https://***/Jazzcharles/CREAM. © 2022, CC BY.

关键词： Object recognition

CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zheng, Tianlun Chen, Zhineng Fang, Shancheng Xie, Hongtao Jiang, Yu-Gang School of Computer Science Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan Universtiy Shanghai200438 China School of Information Science and Technology University of Science and Technology of China Hefei230026 China

The Transformer-based encoder-decoder framework is becoming popular in scene text recognition, largely because it naturally integrates recognition clues from both visual and semantic domains. However, recent studies show that the two kinds of clues are not always well registered and therefore, feature and character might be misaligned in difficult text (e.g., with a rare shape). As a result, constraints such as character position are introduced to alleviate this problem. Despite certain success, visual and semantic are still separately modeled and they are merely loosely associated. In this paper, we propose a novel module called Multi-Domain Character Distance Perception (MDCDP) to establish a visually and semantically related position embedding. MDCDP uses the position embedding to query both visual and semantic features following the cross-attention mechanism. The two kinds of clues are fused into the position branch, generating a content-aware embedding that well perceives character spacing and orientation variants, character semantic affinities, and clues tying the two kinds of information. They are summarized as the multi-domain character distance. We develop CDistNet that stacks multiple MDCDPs to guide a gradually precise distance modeling. Thus, the feature-character alignment is well built even various recognition difficulties are presented. We verify CDistNet on ten challenging public datasets and two series of augmented Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

HUMAN SIMULACRA: BENCHMARKING THE PERSONIFICATION OF LARGE LANGUAGE MODELS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xie, Qiujie Feng, Qiming Zhang, Tianqi Li, Qingqiu Yang, Linyi Zhang, Yuejie Feng, Rui He, Liang Gao, Shang Zhang, Yue School of Computer Science Shanghai Key Lab of Intelligent Information Processing Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University China Tongji University China University College London United Kingdom Huawei Noah’s Ark Lab Canada Shanghai Key Laboratory of Mental Health and Psychological Crisis Intervention China Deakin University Australia Westlake University China Westlake Institute for Advanced Study China

Large Language Models (LLMs) are recognized as systems that closely mimic aspects of human intelligence. This capability has attracted the attention of the social science community, who see the potential in leveraging LLMs to replace human participants in experiments, thereby reducing research costs and complexity. In this paper, we introduce a benchmark for LLMs personification, including a strategy for constructing virtual characters’ life stories from the ground up, a Multi-Agent Cognitive Mechanism capable of simulating human cognitive processes, and a psychology-guided evaluation method to assess human simulations from both self and observational perspectives. Experimental results demonstrate that our constructed simulacra can produce personified responses that align with their target characters. We hope this work will serve as a benchmark in the field of human simulation, paving the way for future research. © 2024, CC BY-NC-SA.

关键词： Digital elevation model

Deep Learning-Based Object Pose Estimation: A Comprehensive Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Jian Sun, Wei Yang, Hui Zeng, Zhiwen Liu, Chongpei Zheng, Jin Liu, Xingyu Rahmani, Hossein Sebe, Nicu Mian, Ajmal The National Engineering Research Center for Robot Visual Perception and Control Technology College of Electrical and Information Engineering the State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body Hunan University Changsha410082 China The School of Architecture and Art Central South University Changsha410082 China The Department of Automation Tsinghua University Beijing100084 China The School of Computing and Communications Lancaster University LA1 4YW United Kingdom The Department of Information Engineering and Computer Science University of Trento Trento38123 Italy The Department of Computer Science The University of Western Australia WA6009 Australia

Object pose estimation is a fundamental computer vision problem with broad applications in augmented reality and robotics. Over the past decade, deep learning models, due to their superior accuracy and robustness, have increasingly supplanted conventional algorithms reliant on engineered point pair features. Nevertheless, several challenges persist in contemporary methods, including their dependency on labeled training data, model compactness, robustness under challenging conditions, and their ability to generalize to novel unseen objects. A recent survey discussing the progress made on different aspects of this area, outstanding challenges, and promising future directions, is missing. To fill this gap, we discuss the recent advances in deep learning-based object pose estimation, covering all three formulations of the problem, i.e., instance-level, category-level, and unseen object pose estimation. Our survey also covers multiple input data modalities, degrees-of-freedom of output poses, object properties, and downstream tasks, providing the readers with a holistic understanding of this field. Additionally, it discusses training paradigms of different domains, inference modes, application areas, evaluation metrics, and benchmark datasets, as well as reports the performance of current state-of-the-art methods on these benchmarks, thereby facilitating the readers in selecting the most suitable method for their application. Finally, the survey identifies key challenges, reviews the prevailing trends along with their pros and cons, and identifies promising directions for future research. We also keep tracing the latest works at Awesome-Object-Pose-Estimation. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

Feature Rotation Invariance Learning for Point Cloud Analysis

学校读者我要写书评

暂无评论

Feature Rotation Invariance Learning for Point Cloud Analysi...

Smart World Congress (SWC), IEEE

作者： Lu Shi Qi Cao Guoqing Zhang Jin Yi Yansen Huang Yigang Cen State Key Laboratory of Advanced Rail Autonomous Operation the School of Computer Science and Technology and Visual Intellgence +X International Cooperation Joint Laboratory of MOE Beijing Jiaotong University Beijing China the School of Computing Science University of Glasgow Singapore With the Key Laboratory of Big Data & Artificial Intelligence in Transportation Ministry of Education the State Key Laboratory of Advanced Rail Autonomous Operation and the School of Computer and Information Technology Beijing Jiaotong University Beijing China the College of Civil Engineering Guizhou University Guizhou Lianjian Civil Engineering Quality Testing Monitoring Center Co. LTD Guizhou China

ISBN: (数字)9798331520861

ISBN: (纸本)9798331520878

While deep learning has significantly advanced point cloud analysis, extracting effective features from their disordered structure remains challenging. Existing approaches often rely on complex network architectures or manual feature engineering to achieve rotation invariance, which increases inference latency. This paper offers a novel perspective on this challenge, showing that rotation-sensitive models can achieve performance comparable to rotation-invariant models when trained with rotation-augmented data. Given the significant differences in the feature distributions of rotation-sensitive and rotation-invariant models, we propose Feature Rotation Invariance Learning (FRIL), a framework that leverages Distance Metric Learning and rotation-based constraints to help rotation-sensitive models extract rotation-invariant features. This approach eliminates the need for additional modules, enabling robust performance against rotations. Extensive experiments demonstrate that FRIL consistently outperforms existing rotation-invariant methods across various visual tasks.

关键词： Point cloud compression Representation learning Analytical models Solid modeling visualization Three-dimensional displays Semantics Cloning Feature extraction Data models

M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Junke Wu, Zuxuan Ouyang, Wenhao Han, Xintong Chen, Jingjing Lim, Ser-Nam Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Huya Inc Meta AI

The widespread dissemination of Deepfakes demands effective approaches that can detect perceptually convincing forged images. In this paper, we aim to capture the subtle manipulation artifacts at different scales using transformer models. In particular, we introduce a Multi-modal Multi-scale TRansformer (M2TR), which operates on patches of different sizes to detect local inconsistencies in images at different spatial levels. M2TR further learns to detect forgery artifacts in the frequency domain to complement RGB information through a carefully designed cross modality fusion block. In addition, to stimulate Deepfake detection research, we introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods. We conduct extensive experiments to verify the effectiveness of the proposed method, which outperforms state-of-the-art Deepfake detection methods by clear margins. Copyright © 2021, The Authors. All rights reserved.

关键词： Frequency domain analysis

Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better

学校读者我要写书评

暂无评论

Revisiting Adversarial Robustness Distillation: Robust Soft ...

International Conference on computer Vision (ICCV)

作者： Bojia Zi Shihao Zhao Xingjun Ma Yu-Gang Jiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan Univeristy Shanghai Collaborative Innovation Center on Intelligent Visual Computing School of Information Technology Deakin University Geelong Australia

ISBN: (纸本)9781665428132

Adversarial training is one effective approach for training robust deep neural networks against adversarial attacks. While being able to bring reliable robustness, adversarial training (AT) methods in general favor high capacity models, i.e., the larger the model the better the robustness. This tends to limit their effectiveness on small models, which are more preferable in scenarios where storage or computing resources are very limited (e.g., mobile devices). In this paper, we leverage the concept of knowledge distillation to improve the robustness of small models by distilling from adversarially trained large models. We first revisit several state-of-the-art AT methods from a distillation perspective and identify one common technique that can lead to improved robustness: the use of robust soft labels – predictions of a robust model. Following this observation, we propose a novel adversarial robustness distillation method called Robust Soft Label Adversarial Distillation (RSLAD) to train robust small student models. RSLAD fully exploits the robust soft labels produced by a robust (adversarially-trained) large teacher model to guide the student’s learning on both natural and adversarial examples in all loss terms. We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack. We also provide a set of understandings on our RSLAD and the importance of robust soft labels for adversarial robustness distillation. Code: https://***/zibojia/RSLAD.

关键词： Training Deep learning computer vision Codes Computational modeling Neural networks Predictive models

Revisiting adversarial robustness distillation: Robust soft labels make student better

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zi, Bojia Zhao, Shihao Ma, Xingjun Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan Univeristy Shanghai Collaborative Innovation Center on Intelligent Visual Computing School of Information Technology Deakin University Geelong Australia

Adversarial training is one effective approach for training robust deep neural networks against adversarial attacks. While being able to bring reliable robustness, adversarial training (AT) methods in general favor high capacity models, i.e., the larger the model the better the robustness. This tends to limit their effectiveness on small models, which are more preferable in scenarios where storage or computing resources are very limited (e.g., mobile devices). In this paper, we leverage the concept of knowledge distillation to improve the robustness of small models by distilling from adversarially trained large models. We first revisit several state-of-the-art AT methods from a distillation perspective and identify one common technique that can lead to improved robustness: the use of robust soft labels - predictions of a robust model. Following this observation, we propose a novel adversarial robustness distillation method called Robust Soft Label Adversarial Distillation (RSLAD) to train robust small student models. RSLAD fully exploits the robust soft labels produced by a robust (adversarially-trained) large teacher model to guide the student's learning on both natural and adversarial examples in all loss terms. We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack. We also provide a set of understandings on our RSLAD and the importance of robust soft labels for adversarial robustness distillation. Code: https://***/zibojia/RSLAD. © 2021, CC BY.

关键词： Distillation

MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-visual Event Localization and Video Parsing

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yu, Jiashuo Cheng, Ying Zhao, Rui-Wei Feng, Rui Zhang, Yuejie School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Academy for Engineering and Technology Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

Recognizing and localizing events in videos is a fundamental task for video understanding. Since events may occur in auditory and visual modalities, multimodal detailed perception is essential for complete scene comprehension. Most previous works attempted to analyze videos from a holistic perspective. However, they do not consider semantic information at multiple scales, which makes the model difficult to localize events in different lengths. In this paper, we present a Multimodal Pyramid Attentional Network (MM-Pyramid) for event localization. Specifically, we first propose the attentive feature pyramid module. This module captures temporal pyramid features via several stacking pyramid units, each of them is composed of a fixed-size attention block and dilated convolution block. We also design an adaptive semantic fusion module, which leverages a unit-level attention block and a selective fusion block to integrate pyramid features interactively. Extensive experiments on audio-visual event localization and weakly-supervised audio-visual video parsing tasks verify the effectiveness of our approach. Copyright © 2021, The Authors. All rights reserved.

关键词： Semantics