检索结果-内蒙古大学图书馆

PREP: Pre-training with Temporal Elapse Inference for Popularity Prediction

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cao, Qi Shen, Huawei Liu, Yuanhao Gao, Jinhua Cheng, Xueqi Data Intelligence System Research Center Institute of Computing Technology Chinese Academy of Sciences China Cas Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China

Predicting the popularity of online content is a fundamental problem in various applications. One practical challenge takes roots in the varying length of observation time or prediction horizon, i.e., a good model for popularity prediction is desired to handle various prediction settings. However, most existing methods adopt a separate training paradigm for each prediction setting and the obtained model for one setting is difficult to be generalized to others, causing a great waste of computational resources and a large demand for downstream labels. To solve the above issues, we propose a novel pre-training framework for popularity prediction, namely PREP, aiming to pre-train a general representation model from the readily available unlabeled diffusion data, which can be effectively transferred into various prediction settings. We design a novel pretext task for pre-training, i.e., temporal elapse inference for two randomly sampled time slices of popularity dynamics, impelling the representation model to learn intrinsic knowledge about popularity dynamics. Experimental results conducted on two real datasets demonstrate the generalization and efficiency of the pre-training framework for different popularity prediction task settings. Copyright © 2021, The Authors. All rights reserved.

关键词： Forecasting

Self-Supervised GANs with label Augmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hou, Liang Shen, Huawei Cao, Qi Cheng, Xueqi Data Intelligence System Research Center Institute of Computing Technology Chinese Academy of Sciences China CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China

Recently, transformation-based self-supervised learning has been applied to generative adversarial networks (GANs) to mitigate catastrophic forgetting in the discriminator by introducing a stationary learning environment. However, the separate self-supervised tasks in existing self-supervised GANs cause a goal inconsistent with generative modeling due to the fact that their self-supervised classifiers are agnostic to the generator distribution. To address this problem, we propose a novel self-supervised GAN that unifies the GAN task with the self-supervised task by augmenting the GAN labels (real or fake) via self-supervision of data transformation. Specifically, the original discriminator and self-supervised classifier are unified into a label-augmented discriminator that predicts the augmented labels to be aware of both the generator distribution and the data distribution under every transformation, and then provide the discrepancy between them to optimize the generator. Theoretically, we prove that the optimal generator could converge to replicate the real data distribution. Empirically, we show that the proposed method significantly outperforms previous self-supervised and data augmentation GANs on both generative modeling and representation learning across benchmark datasets. Copyright © 2021, The Authors. All rights reserved.

关键词： Generative adversarial networks

On the Effectiveness of Function-Level Vulnerability Detectors for Inter-Procedural Vulnerabilities 24

学校读者我要写书评

暂无评论

On the Effectiveness of Function-Level Vulnerability Detecto...

44th ACM/IEEE International Conference on Software Engineering, ICSE 2024

作者： Li, Zhen Wang, Ning Zou, Deqing Li, Yating Zhang, Ruqian Xu, Shouhuai Zhang, Chao Jin, Hai Hubei Key Laboratory of Distributed System Security Hubei Engineering Research Center on Big Data Security Cluster and Grid Computing Lab National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Hong Kong Jin YinHu Laboratory Wuhan China School of Cyber Science and Engineering Huazhong University of Science and Technology Wuhan China University of Colorado Colorado Springs Department of Computer Science Colorado Springs Colorado United States Institute for Network Sciences and Cyberspace Tsinghua University Beijing China School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

ISBN: (纸本)9798400702174

Software vulnerabilities are a major cyber threat and it is important to detect them. One important approach to detecting vulnerabilities is to use deep learning while treating a program function as a whole, known as function-level vulnerability detectors. However, the limitation of this approach is not understood. In this paper, we investigate its limitation in detecting one class of vulnerabilities known as inter-procedural vulnerabilities, where the to-be-patched statements and the vulnerability-triggering statements belong to different functions. For this purpose, we create the first Inter -Procedural Vulnerability dataset (InterPVD) based on C/C++ open-source software, and we propose a tool dubbed VulTrigger for identifying vulnerability-triggering statements across functions. Experimental results show that VulTrigger can effectively identify vulnerability-triggering statements and inter-procedural vulnerabilities. Our findings include: (i) inter-procedural vulnerabilities are prevalent with an average of 2.8 inter-procedural layers;and (ii) function-level vulner-ability detectors are much less effective in detecting to-be-patched functions of inter-procedural vulnerabilities than detecting their counterparts of intra-procedural vulnerabilities. © 2024 ACM.

关键词： Open source software

Match-ignition: Plugging PageRank into transformer for long-form text matching

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Pang, Liang Lan, Yanyan Cheng, Xueqi Data Intelligence System Research Center Institute of Computing Technology Chinese Academy of Sciences Beijing China Institute for AI Industry Research Tsinghua University Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Neural text matching models have been widely used in community question answering, information retrieval, and dialogue. However, these models designed for short texts cannot well address the long-form text matching problem, because there are many contexts in long-form texts can not be directly aligned with each other, and it is difficult for existing models to capture the key matching signals from such noisy data. Besides, these models are computationally expensive for simply use all textual data indiscriminately. To tackle the effectiveness and efficiency problem, we propose a novel hierarchical noise filtering model, namely Match-Ignition. The main idea is to plug the well-known PageRank algorithm into the Transformer, to identify and filter both sentence and word level noisy information in the matching process. Noisy sentences are usually easy to detect because previous work has shown that their similarity can be explicitly evaluated by the word overlapping, so we directly use PageRank to filter such information based on a sentence similarity graph. Unlike sentences, words rely on their contexts to express concrete meanings, so we propose to jointly learn the filtering and matching process, to well capture the critical word-level matching signals. Specifically, a word graph is first built based on the attention scores in each self-attention block of Transformer, and key words are then selected by applying PageRank on this graph. In this way, noisy words will be filtered out layer by layer in the matching process. Experimental results show that Match-Ignition outperforms both SOTA short text matching models and recent long-form text matching models. We also conduct detailed analysis to show that Match-Ignition efficiently captures important sentences and words, to facilitate the long-form text matching process. © 2021, CC BY.

关键词：

Slimmable generative adversarial networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Hou, Liang Yuan, Zehuan Huang, Lei Shen, Huawei Cheng, Xueqi Wang, Changhu CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences ByteDance AI Lab SKLSDE Institute of Artificial Intelligence Beihang University

Generative adversarial networks (GANs) have achieved remarkable progress in recent years, but the continuously growing scale of models makes them challenging to deploy widely in practical applications. In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power. In this paper, we introduce slimmable GANs (SlimGANs), which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime. Specifically, we leverage multiple discriminators that share partial parameters to train the slimmable generator. To facilitate the consistency between generators of different widths, we present a stepwise inplace distillation technique that encourages narrow generators to learn from wide ones. As for class-conditional generation, we propose a sliceable conditional batch normalization that incorporates the label information into different widths. Our methods are validated, both quantitatively and qualitatively, by extensive experiments and a detailed ablation study. Copyright © 2020, The Authors. All rights reserved.

关键词： Generative adversarial networks

Transductive learning for unsupervised text style transfer

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xiao, Fei Pang, Liang Lan, Yanyan Wang, Yan Shen, Huawei Cheng, Xueqi Data Intelligence System Research Center Cas Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Institute for Ai Industry Research Tsinghua University University of Chinese Academy of Sciences Tencent Ai Lab

Unsupervised style transfer models are mainly based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases. However, the lacking of parallel corpus hinders the ability of these inductive learning methods on this task. As a result, it is likely to cause severe inconsistent style expressions, like the salad is rude. To tackle this problem, we propose a novel transductive learning approach in this paper, based on a retrieval-based context-aware style representation. Specifically, an attentional encoderdecoder with a retriever framework is utilized. It involves top-K relevant sentences in the target style in the transfer process. In this way, we can learn a context-aware style embedding to alleviate the above inconsistency problem. In this paper, both sparse (BM25) and dense retrieval functions (MIPS) are used, and two objective functions are designed to facilitate joint learning. Experimental results show that our method outperforms several strong baselines. The proposed transductive learning approach is general and effective to the task of unsupervised style transfer, and we will apply it to the other two typical methods in the future. © 2021, CC BY.

关键词： Embeddings

Ranking enhanced dialogue generation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Hao, Changying Pang, Liang Lan, Yanyan Sun, Fei Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China Alibaba Group Beijing China

How to effectively utilize the dialogue history is a crucial problem in multi-turn dialogue generation. Previous works usually employ various neural network architectures (e.g., recurrent neural networks, attention mechanisms, and hierarchical structures) to model the history. However, a recent empirical study by Sankar et al. has shown that these architectures lack the ability of understanding and modeling the dynamics of the dialogue history. For example, the widely used architectures are insensitive to perturbations of the dialogue history, such as words shuffling, utterances missing, and utterances reordering. To tackle this problem, we propose a Ranking Enhanced Dialogue generation framework in this paper. Despite the traditional representation encoder and response generation modules, an additional ranking module is introduced to model the ranking relation between the former utterance and consecutive utterances. Specifically, the former utterance and consecutive utterances are treated as query and corresponding documents, and both local and global ranking losses are designed in the learning process. In this way, the dynamics in the dialogue history can be explicitly captured. To evaluate our proposed models, we conduct extensive experiments on three public datasets, i.e., bAbI, PersonaChat, and JDC. Experimental results show that our models produce better responses in terms of both quantitative measures and human judgments, as compared with the state-of-the-art dialogue generation models. Furthermore, we give some detailed experimental analysis to show where and how the improvements come from. Copyright © 2020, The Authors. All rights reserved.

关键词： Recurrent neural networks

Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Yu-An Zhang, Ruqing Zhang, Mingkun Chen, Wei de Rijke, Maarten Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China University of Amsterdam Amsterdam Netherlands

Neural ranking models (NRMs) have shown great success in information retrieval (IR). But their predictions can easily be manipulated using adversarial examples, which are crafted by adding imperceptible perturbations to legitimate documents. This vulnerability raises significant concerns about their reliability and hinders the widespread deployment of NRMs. By incorporating adversarial examples into training data, adversarial training has become the de facto defense approach to adversarial attacks against NRMs. However, this defense mechanism is subject to a trade-off between effectiveness and adversarial robustness. In this study, we establish theoretical guarantees regarding the effectiveness-robustness tradeoff in NRMs. We decompose the robust ranking error into two components, i.e., a natural ranking error for effectiveness evaluation and a boundary ranking error for assessing adversarial robustness. Then, we define the perturbation invariance of a ranking model and prove it to be a differentiable upper bound on the boundary ranking error for attainable computation. Informed by our theoretical analysis, we design a novel perturbation-invariant adversarial training (PIAT) method for ranking models to achieve a better effectiveness-robustness trade-off. We design a regularized surrogate loss, in which one term encourages the effectiveness to be maximized while the regularization term encourages the output to be smooth, so as to improve adversarial robustness. Experimental results on several ranking models demonstrate the superiority of PITA compared to existing adversarial defenses. Copyright © 2023, The Authors. All rights reserved.

关键词： Errors

aNMM: Ranking short answer texts with attention-based neural matching model

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yang, Liu Ai, Qingyao Guo, Jiafeng Bruce Croft, W. Center for Intelligent Information Retrieval University of Massachusetts AmherstMA United States CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

As an alternative to question answering methods based on feature engineering, deep learning approaches such as convolutional neural networks (CNNs) and Long Short-Term Memory Models (LSTMs) have recently been proposed for semantic matching of questions and answers. To achieve good results, however, these models have been combined with additional features such as word overlap or BM25 scores. Without this combination, these models perform significantly worse than methods based on linguistic feature engineering. In this paper, we propose an attention based neural matching model for ranking short answer text. We adopt value-shared weighting scheme instead of position-shared weighting scheme for combining different matching signals and incorporate question term importance learning using question attention network. Using the popular benchmark TREC QA data, we show that the relatively simple aNMM model can significantly outperform other neural network models that have been used for the question answering task, and is competitive with models that are combined with additional features. When aNMM is combined with additional features, it outperforms all baselines. Copyright © 2018, The Authors. All rights reserved.

关键词： Semantics