检索结果-内蒙古大学图书馆

Revisiting adversarial robustness distillation: Robust soft labels make student better

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zi, Bojia Zhao, Shihao Ma, Xingjun Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan Univeristy Shanghai Collaborative Innovation Center on Intelligent Visual Computing School of Information Technology Deakin University Geelong Australia

Adversarial training is one effective approach for training robust deep neural networks against adversarial attacks. While being able to bring reliable robustness, adversarial training (AT) methods in general favor high capacity models, i.e., the larger the model the better the robustness. This tends to limit their effectiveness on small models, which are more preferable in scenarios where storage or computing resources are very limited (e.g., mobile devices). In this paper, we leverage the concept of knowledge distillation to improve the robustness of small models by distilling from adversarially trained large models. We first revisit several state-of-the-art AT methods from a distillation perspective and identify one common technique that can lead to improved robustness: the use of robust soft labels - predictions of a robust model. Following this observation, we propose a novel adversarial robustness distillation method called Robust Soft label Adversarial Distillation (RSLAD) to train robust small student models. RSLAD fully exploits the robust soft labels produced by a robust (adversarially-trained) large teacher model to guide the student's learning on both natural and adversarial examples in all loss terms. We empirically demonstrate the effectiveness of our RSLAD approach over existing adversarial training and distillation methods in improving the robustness of small models against state-of-the-art attacks including the AutoAttack. We also provide a set of understandings on our RSLAD and the importance of robust soft labels for adversarial robustness distillation. Code: https://***/zibojia/RSLAD. © 2021, CC BY.

关键词： Distillation

A Dual Prompt Learning Framework for Few-Shot Dialogue State Tracking

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yuting Lei, Wenqiang Huang, Pei Cao, Juan Li, Jintao Chua, Tat-Seng University of Chinese Academy of Sciences Beijing China Sichuan University Sichuan China Stanford University California United States Key Lab of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China National University of Singapore Singapore

Dialogue state tracking (DST) module is an important component for task-oriented dialog systems to understand users' goals and needs. Collecting dialogue state labels including slots and values can be costly, especially with the wide application of dialogue systems in more and more new-rising domains. In this paper, we focus on how to utilize the language understanding and generation ability of pre-trained language models for DST. We design a dual prompt learning framework for few-shot DST. Specifically, we consider the learning of slot generation and value generation as dual tasks, and two prompts are designed based on such a dual structure to incorporate task-related knowledge of these two tasks respectively. In this way, the DST task can be formulated as a language modeling task efficiently under few-shot settings. Experimental results on two task-oriented dialogue datasets show that the proposed method not only outperforms existing state-of-the-art few-shot methods, but also can generate unseen slots. It indicates that DST-related knowledge can be probed from PLM and utilized to address low-resource DST efficiently with the help of prompt learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Computational linguistics

Harnessing Hierarchical label Distribution Variations in Test Agnostic Long-tail Recognition

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Zhiyong Xu, Qianqian Wang, Zitai Li, Sicong Han, Boyu Bao, Shilong Cao, Xiaochun Huang, Qingming School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Tech. CAS Institute of Information Engineering CAS School of Cyber Security University of Chinese Academy of Sciences China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China BDKM University of Chinese Academy of Sciences China

This paper explores test-agnostic long-tail recognition, a challenging long-tail task where the test label distributions are unknown and arbitrarily imbalanced. We argue that the variation in these distributions can be broken down hierarchically into global and local levels. The global ones reflect a broad range of diversity, while the local ones typically arise from milder changes, often focused on a particular neighbor. Traditional methods predominantly use a Mixture-of-Expert (MoE) approach, targeting a few fixed test label distributions that exhibit substantial global variations. However, the local variations are left unconsidered. To address this issue, we propose a new MoE strategy, DirMixE, which assigns experts to different Dirichlet meta-distributions of the label distribution, each targeting a specific aspect of local variations. Additionally, the diversity among these Dirichlet meta-distributions inherently captures global variations. This dual-level approach also leads to a more stable objective function, allowing us to sample different test distributions better to quantify the mean and variance of performance outcomes. Theoretically, we show that our proposed objective benefits from enhanced generalization by virtue of the variance-based regularization. Comprehensive experiments across multiple benchmarks confirm the effectiveness of DirMixE. The code is available at https: //***/scongl/DirMixE. © 2024, CC BY.

关键词：

Design of low-profile wide-band high-gain circularly polarized antenna based on metasurface

学校读者我要写书评

暂无评论

Design of low-profile wide-band high-gain circularly polariz...

International Conference on Numerical Electromagnetic Modeling and Optimization for RF, Microwave, and Terahertz Applications (NEMO)

作者： Wei Zhang Liang Zhang Xianliang Wu key lab of intelligent computing & signal processing ministry of education anhui university Anhui University Hefei China key lab of intelligent computing & signal processing ministry of education anhui university AnHui Province Key Laboratory of simulation and design for Electronic information system Hefei Normal university Hefei Normal University Hefei China

ISBN: (数字)9781728169668

ISBN: (纸本)9781728169675

This paper proposes a wide-band low-profile high-gain microstrip slot antenna loaded with a mushroom-type metasurface structure. By directly loading the metamaterial structure on the upper surface of the slot antenna, low profile loading characteristics are realized and the antenna bandwidth is expanded. And by inserting the metal line array in the metamaterial dielectric board to improve the antenna gain. Change the gap width through the metasurface element with corner cut to produce a phase difference and realize circularly polarized. Simulation test and analysis of antenna through HFSS simulation software. The results show that the antenna with this metasurface has a profile height of only 0.09×λ(λ =27 mm), a relative impedance band-width of 20.4% (S 11

关键词： Slot antennas Microstrip antennas Metasurfaces Metamaterials Microstrip Loaded antennas Gain

Characterization and construction of optimal binary linear codes with one-dimensional hull

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Shitao Shi, Minjia Kim, Jon-Lark The School of Mathematical Sciences Anhui University Hefei230601 China The Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education The School of Mathematical Sciences Anhui University Hefei230601 China State Key Laboratory of Information Security Institute of Information Engineering Chinese Academy of Sciences Beijing100093 China The Department of Mathematics Sogang University Seoul Korea Republic of

The hull of a linear code over finite fields is the intersection of the code and its dual, and linear codes with small hulls have applications in computational complexity and information protection. Linear codes with the smallest hull are LCD codes, which have been widely studied. Recently, several papers were devoted to related LCD codes over finite fields with size greater than 3 to linear codes with one-dimensional or higher dimensional hull. Therefore, an interesting and non-trivial problem is to study binary linear codes with one-dimensional hull with connection to binary LCD codes. The objective of this paper is to study some properties of binary linear codes with one-dimensional hull, and establish their relation with binary LCD codes. Some interesting inequalities are thus obtained. Using such a characterization, we study the largest minimum distance done(n, k) among all binary linear [n, k] codes with one-dimensional hull. We determine the largest minimum distances done(n, n − k) for k ≤ 5 and done(n, k) for k ≤ 4 or 14 ≤ n ≤ 24. We partially determine the exact value of done(n, k) for k = 5 or 25 ≤ n ≤ *** Codes 94B05, 15B05, 12E10 Copyright © 2022, The Authors. All rights reserved.

关键词： Optimal systems

High order SF-SFDTD method for solving the time-dependent Schrödinger equation

学校读者我要写书评

暂无评论

High order SF-SFDTD method for solving the time-dependent Sc...

International Applied Computational Electromagnetics Society Symposium (ACES)

作者： Pan Pan Guoda Xie Zhixiang Huang Guilin Hou Ziheng Song School of Electronic and Information Engineering Anhui university Hefei China Education of Anhui Province. Anhui Province Key Laboratory of Target Recognition and Feature Extraction Luan Anhui China Ministry of Education. the Key Laboratory of Intelligent Computing and Signal Processing Hefei China Key Laboratory of Electromagnetic Environmental Sensing Anhui Province Anhui University Hefei China Anhui university Hefei China

A high-order spatial filtering-symplectic finite difference time domain (SF-SFDTD) scheme with controllable stability condition is proposed to solve the time-dependent Schrödinger equation (TDSE). Firstly, the higher-order symplectic framework for the discretization of Schrödinger equation is described, and then the spatial filtering method is carried out to extend the stability of the standard SFDTD method. Finally, a numerical example is given to evaluate the correctness of the SF-SFDTD method in numerical solving Schrödinger equation.

关键词： Filtering Filtering algorithms Computational electromagnetics Mathematical models Time-domain analysis Numerical stability Standards

Chosen Plaintext Attack on Jpeg Image Encryption with Adaptive key and Run Consistency

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： He, Hongjie Yuan, Yuan Chen, Fan Ye, Yuyun Tai, Heng-Ming Key Lab of Signal & Information Processing Southwest Jiaotong University Sichuan Province Chengdu611756 China Department of Electrical and Computer Engineering University of Tulsa TulsaOK74104 United States School of Computing and Artificial Intelligence Southwest Jiaotong University Chengdu611756 China

A JPEG image encryption with an adaptive key generated by a preserved characteristic not only lowers the cost of key management but also enhances the ability to resist various attacks. However, there are still security risks under the chosen-plaintext attack (CPA) conditions. This paper designs a novel CPA scheme to obtain details of the original image. First, the adaptive key can be reproduced from the encrypted image, so that the plaintext images with the same adaptive key can be constructed. Second, the existence of run consistency of MCUs (RCM), which facilitates rapid estimation. In addition, the single permutation for the runs of MCUs with RCM is designed for more accurate estimation. Experimental results show that two plaintext images are sufficient to get the image details. For an image of high quality factors (QFs≥90) or with rich complex texture, the accuracy ratio of estimated permutation sequence is close to 100%. © 2022, The Authors. All rights reserved.

关键词： Cryptography

A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancement

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yuting Huang, Pei Cao, Juan Li, Jintao Lin, Yun Dong, Jin Song Ma, Feifei Zhang, Jian Key Lab of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China Beijing China National University of Singapore Singapore Laboratory of Parallel Software and Computational Science ISCAS Beijing China

Recent years have seen the wide application of NLP models in crucial areas such as finance, medical treatment, and news media, raising concerns of the model robustness and vulnerabilities. In this paper, we propose a novel prompt-based adversarial attack to compromise NLP models and robustness enhancement technique. We first construct malicious prompts for each instance and generate adversarial examples via mask-and-filling under the effect of a malicious purpose. Our attack technique targets the inherent vulnerabilities of NLP models, allowing us to generate samples even without interacting with the victim NLP model, as long as it is based on pre-trained language models (PLMs). Furthermore, we design a prompt-based adversarial training method to improve the robustness of PLMs. As our training method does not actually generate adversarial samples, it can be applied to large-scale training sets efficiently. The experimental results show that our attack method can achieve a high attack success rate with more diverse, fluent and natural adversarial examples. In addition, our robustness enhancement method can significantly improve the robustness of models to resist adversarial attacks. Our work indicates that prompting paradigm has great potential in probing some fundamental flaws of PLMs and fine-tuning them for downstream tasks. Copyright © 2022, The Authors. All rights reserved.

关键词： Natural language processing systems

Self-Mutual Distillation Learning for Continuous Sign Language Recognition

学校读者我要写书评

暂无评论

Self-Mutual Distillation Learning for Continuous Sign Langua...

International Conference on Computer Vision (ICCV)

作者： Aiming Hao Yuecong Min Xilin Chen Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (纸本)9781665428132

In recent years, deep learning moves video-based Continuous Sign Language Recognition (CSLR) significantly forward. Currently, a typical network combination for CSLR includes a visual module, which focuses on spatial and short-temporal information, followed by a contextual module, which focuses on long-temporal information, and the Connectionist Temporal Classification (CTC) loss is adopted to train the network. However, due to the limitation of chain rules in back-propagation, the visual module is hard to adjust for seeking optimized visual features. As a result, it enforces that the contextual module focuses on contextual information optimization only rather than balancing efficient visual and contextual information. In this paper, we propose a Self-Mutual Knowledge Distillation (SMKD) method, which enforces the visual and contextual modules to focus on short-term and long-term information and enhances the discriminative power of both modules simultaneously. Specifically, the visual and contextual modules share the weights of their corresponding classifiers, and train with CTC loss simultaneously. Moreover, the spike phenomenon widely exists with CTC loss. Although it can help us choose a few of the key frames of a gloss, it does drop other frames in a gloss and makes the visual feature saturation in the early stage. A gloss segmentation is developed to relieve the spike phenomenon and decrease saturation in the visual module. We conduct experiments on two CSLR bench-marks: PHOENIX14 and PHOENIX14-T. Experimental results demonstrate the effectiveness of the SMKD.

关键词： Training Deep learning Visualization Computer vision Gesture recognition Assistive technologies Benchmark testing