检索结果-内蒙古大学图书馆

Hybrid Data-Free Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Jialiang Chen, Shuo Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Jiangsu Key Laboratory of Image and Video Understanding for Social Security China Center for Advanced Intelligence Project RIKEN Japan Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods. Code is available at https://***/tangjialiang97/HiDFD. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Yuanxin Meng, Fandong Lin, Zheng Li, Jiangnan Fu, Peng Cao, Yanan Wang, Weiping Zhou, Jie Institute of Information Engineering Chinese Academy of Sciences China MOE Key Laboratory of Computational Linguistics Peking University China School of Computer Science Peking University China School of Cyber Security University of Chinese Academy of Sciences China Pattern Recognition Center WeChat AI Tencent Inc China

Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend to rely on the dataset bias and struggle to generalize to out-of-distribution (OOD) data. In response to the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning. However, these results are only obtained in the in-distribution (ID) setting. In this paper, we extend the study on PLMs subnetworks to the OOD setting, investigating whether sparsity and robustness to dataset bias can be achieved simultaneously. To this end, we conduct extensive experiments with the pre-trained BERT model on three natural language understanding (NLU) tasks. Our results demonstrate that sparse and robust subnetworks (SRNets) can consistently be found in BERT, across the aforementioned three scenarios, using different training and compression methods. Furthermore, we explore the upper bound of SRNets using the OOD information and show that there exist sparse and almost unbiased BERT subnetworks. Finally, we present 1) an analytical study that provides insights on how to promote the efficiency of SRNets searching process and 2) a solution to improve subnetworks’ performance at high sparsity. Copyright © 2022, The Authors. All rights reserved.

关键词： Efficiency

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Lei Lin, Yankai Ren, Xuancheng Zhao, Guangxiang Li, Peng Zhou, Jie Sun, Xu MOE Key Lab of Computational Linguistics School of Computer Science Peking University China Gaoling School of Artificial Intelligence Renmin University of China Beijing China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Tsinghua University China Pattern Recognition Center WeChat AI Tencent Inc. China

Investigating better ways to reuse the released pre-trained language models (PLMs) can significantly reduce the computational cost and the potential environmental side-effects. This paper explores a novel PLM reuse paradigm, Knowledge Integration (KI). Without human annotations available, KI aims to merge the knowledge from different teacher-PLMs, each of which specializes in a different classification problem, into a versatile student model. To achieve this, we first derive the correlation between virtual golden supervision and teacher predictions. We then design a Model Uncertainty–aware Knowledge Integration (MUKI) framework to recover the golden supervision for the student. Specifically, MUKI adopts Monte-Carlo Dropout to estimate model uncertainty for the supervision integration. An instance-wise re-weighting mechanism based on the margin of uncertainty scores is further incorporated, to deal with the potential conflicting supervision from teachers. Experimental results demonstrate that MUKI achieves substantial improvements over baselines on benchmark datasets. Further analysis shows that MUKI can generalize well for merging teacher models with heterogeneous architectures, and even teachers major in cross-lingual datasets.1 © 2022, CC BY.

关键词： Uncertainty analysis

Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Xuannan Cui, Xing Li, Peipei Li, Zekun Huang, Huaibo Xia, Shuhan Zhang, Miaoxuan Zou, Yueying He, Ran School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China School of Computer Science University of California Santa Barbara United States State Key Laboratory of Multimodal Artificial Intelligence Systems CASIA New Laboratory of Pattern Recognition CASIA School of Artificial Intelligence University of Chinese Academy of Sciences Beijing100190 China

The rapid evolution of multimodal foundation models has led to significant advancements in cross-modal understanding and generation across diverse modalities, including text, images, audio, and video. However, these models remain susceptible to jailbreak attacks, which can bypass built-in safety mechanisms and induce the production of potentially harmful content. Consequently, understanding the methods of jailbreak attacks and existing defense mechanisms is essential to ensure the safe deployment of multimodal generative models in real-world scenarios, particularly in security-sensitive applications. To provide comprehensive insight into this topic, this survey reviews jailbreak and defense in multimodal generative models. First, given the generalized lifecycle of multimodal jailbreak, we systematically explore attacks and corresponding defense strategies across four levels: input, encoder, generator, and output. Based on this analysis, we present a detailed taxonomy of attack methods, defense mechanisms, and evaluation frameworks specific to multimodal generative models. Additionally, we cover a wide range of input-output configurations, including modalities such as Any-to-Text, Any-to-vision, and Any-to-Any within generative systems. Finally, we highlight current research challenges and propose potential directions for future research. The open-source repository corresponding to this work can be found at https://***/liuxuannan/Awesome-Multimodal-Jailbreak. Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

A novel TSK fuzzy system incorporating multiview collaborative transfer learning for personalized epileptic EEG detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Li, Andong Deng, Zhaohong Lou, Qiongdan Choi, Kup-Sze Shen, Hongbin Wang, Shitong School of Artificial Intelligence and Computer Science Jiangnan University Jiangsu Key Laboratory of Digital Design and Software Technology Wuxi214122 China Centre for Smart Health Hong Kong Polytechnic University Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai200240 China Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China

—In clinical practice, electroencephalography (EEG) plays an important role in the diagnosis of epilepsy. EEG-based computer-aided diagnosis of epilepsy can greatly improve the accuracy of epilepsy detection while reducing the workload of physicians. However, there are many challenges in practical applications for personalized epileptic EEG detection (i.e., training of detection model for a specific person), including the difficulty in extracting effective features from one single view, the undesirable but common scenario of lacking sufficient training data in practice, and the no guarantee of identically distributed training and test data. To solve these problems, we propose a TSK fuzzy system-based epilepsy detection algorithm that integrates multi-view collaborative transfer learning. To address the challenge due to the limitation of single-view features, multi-view learning ensures the diversity of features by extracting them from different views. The lack of training data for building a personalized detection model is tackled by leveraging the knowledge from the source domain (reference scene) to enhance the performance of the target domain (current scene of interest), where mismatch of data distributions between the two domains is resolved with adaption technique based on maximum mean discrepancy. Notably, the transfer learning and multi-view feature extraction are performed at the same time. Furthermore, the fuzzy rules of the TSK fuzzy system equip the model with strong fuzzy logic inference capability. Hence, the proposed method has the potential to detect epileptic EEG signals effectively, which is demonstrated with the positive results from a large number of experiments on the CHB-MIT dataset. © 2021, CC BY-NC-ND.

关键词： Electrophysiology

MFHI: Taking Modality-free Human Identification as Zero-shot Learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Liu, Zhizhe Zhang, Xingxing Zhu, Zhenfeng Zheng, Shuai Zhao, Yao Cheng, Jian Institute of Information Science Beijing Jiaotong University Beijing100044 China Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing Jiaotong University Beijing100044 China Department of Computer Science and Technology Tsinghua University Beijing100084 China National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing China

Human identification is an important topic in event detection, person tracking, and public security. There have been numerous methods proposed for human identification, such as face identification, person re-identification, and gait identification. Typically, existing methods predominantly classify a queried image to a specific identity in an image gallery set (I2I). This is seriously limited for the scenario where only a textual description of the query or an attribute gallery set is available in a wide range of video surveillance applications (A2I or I2A). However, very few efforts have been devoted towards modality-free identification, i.e., identifying a query in a gallery set in a scalable way. In this work, we take an initial attempt, and formulate such a novel Modality-Free Human Identification (named MFHI) task as a generic zero-shot learning model in a scalable way. Meanwhile, it is capable of bridging the visual and semantic modalities by learning a discriminative prototype of each identity. In addition, the semantics-guided spatial attention is enforced on visual modality to obtain interpretable representations with both high global category-level and local attribute-level discrimination. Finally, we design and conduct an extensive group of experiments on two common challenging identification tasks, including face identification and person re-identification, demonstrating that our method outperforms a wide variety of state-of-the-art methods on modality-free human identification. Copyright © 2020, The Authors. All rights reserved.

关键词： Semantics

Journal of Network Intelligence

学校读者我要写书评

暂无评论

Journal of Network Intelligence 2022年第4期7卷 848-861页

作者： Weng, Wei Li, Tong Liao, Jian-Chao Guo, Feng Chen, Fen Wei, Bo-Wen College of Computer and Information Engineering Xiamen University of Technology Fujian Key Laboratory of Pattern Recognition and Image Understanding Xiamen361024 China College of Computer and Information Engineering Xiamen University of Technology Xiamen361024 China Fujian Newland Auto-ID Tech. Co. Ltd Fuzhou350015 China Xiamen Fuyun Information Tech. Co. Ltd Xiamen361008 China

Graph clustering is a fundamental method for studying complex networks. Some existing approaches focus on the graph data without attributed information. However, graph data in the real world generally have attribute information, which can better describe complex networks. Recently, deep learning-based methods are widely used for attributed graph clustering. DAEGC is a goal-directed method, which utilizes graph attention encoders for latent information learning and performs competitively in attributed graph clustering. To better aggregate the structure and node attribute information in DAEGC, we employ a lightweight attention mechanism to capture the relationship between nodes and their neighbors, learn hidden representations, and use a self-training module to train the model for optimal results. Extensive experiments show that our proposed method improves DAEGC and achieves superior performance compared with state-of-the-art algorithms in attributed graph clustering. © 2022, Taiwan Ubiquitous Information CO LTD. All rights reserved.

关键词： Complex networks

Learning Flexible Binary Code for Linear Projection Based Hashing with Random Forest

学校读者我要写书评

暂无评论

Learning Flexible Binary Code for Linear Projection Based Ha...

International Conference on pattern recognition

作者： Shuze Du Wei Zhang Shifeng Chen Yafei Wen Chengdu Institute of Computer Applications Chinese Academy of Sciences Sichuan China Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Media Lab. Huawei Technologies Co. Ltd. China Shenzhen Key Lab of Computer Vision and Pattern Recognition Chinese Academy of Sciences Guangdong China The Chinese University of Hong Kong Hong Kong University of Chinese Academy of Sciences Beijing China

ISBN: (纸本)9781479952106

Existing linear projection based hashing methods have witnessed many progresses in finding the approximate nearest neighbor(s) of a given query. They perform well when using a short code. But their code length depends on the original data dimension, thus their performance can not be further improved with higher number of bits for low dimensional data. In addition, in the case of high dimensional data, it is not a good choice to produce each bit by a sign function. In this paper, we propose a novel random forest based approach to cope with the above shortcomings. The bits are obtained by recording the paths when a point traversing each tree in the forest. Then we propose a new metric to calculate the similarity between any two codes. Experimental results on two large benchmark datasets show that our approach outperforms its counterparts and demonstrate its superiority over the existing state-of-the-art hashing methods for descriptor retrieval.

关键词： Vegetation Vectors Binary codes Encoding Computational modeling Radio frequency Principal component analysis

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction

学校读者我要写书评

暂无评论

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreferen...

2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022

作者： Wang, Xiaozhi Chen, Yulin Ding, Ning Peng, Hao Wang, Zimu Lin, Yankai Han, Xu Hou, Lei Li, Juanzi Liu, Zhiyuan Li, Peng Zhou, Jie Department of Computer Science and Technology BNRist Tsinghua University Beijing China Shenzhen International Graduate School Tsinghua University Beijing China THU-Siemens Ltd. China Joint Research Center for Industrial Intelligence and IoT Tsinghua University Beijing China Tsinghua University Beijing China Xi'an Jiaotong-Liverpool University Suzhou China Gaoling School of Artificial Intelligence Renmin University of China Beijing China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Pattern Recognition Center WeChat AI Tencent Inc China

The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two drawbacks of existing datasets limit event relation extraction (ERE) tasks: (1) Small scale. Due to the annotation complexity, the data scale of existing datasets is limited, which cannot well train and evaluate data-hungry models. (2) Absence of unified annotation. Different types of event relations naturally interact with each other, but existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. To address these issues, we construct a unified large-scale human-annotated ERE dataset MAVEN-ERE with improved annotation schemes. It contains 103, 193 event coreference chains, 1, 216, 217 temporal relations, 57, 992 causal relations, and 15, 841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude. Experiments show that ERE on MAVEN-ERE is quite challenging, and considering relation interactions with joint learning can improve performances. The dataset and source codes can be obtained from https://***/THU-KEG/MAVEN-ERE. © 2022 Association for Computational Linguistics.

关键词： Extraction