检索结果-内蒙古大学图书馆

Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ma, Xinyu Guo, Jiafeng Zhang, Ruqing Fan, Yixing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Dense retrieval has shown promising results in many information retrieval (IR) related tasks, whose foundation is high-quality text representation learning for effective search. Some recent studies have shown that autoencoder-based language models are able to boost the dense retrieval performance using a weak decoder. However, we argue that 1) it is not discriminative to decode all the input texts and, 2) even a weak decoder has the bypass effect on the encoder. Therefore, in this work, we introduce a novel contrastive span prediction task to pre-train the encoder alone, but still retain the bottleneck ability of the autoencoder. The key idea is to force the encoder to generate the text representation close to its own random spans while far away from others using a group-wise contrastive loss. In this way, we can 1) learn discriminative text representations efficiently with the group-wise contrastive learning over spans and, 2) avoid the bypass effect of the decoder thoroughly. Comprehensive experiments over publicly available retrieval benchmark datasets show that our approach can outperform existing pre-training methods for dense retrieval significantly. Code and pre-trained models will be available at the URL. Copyright © 2022, The Authors. All rights reserved.

关键词： Signal encoding

Query understanding via intent description generation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhang, Ruqing Guo, Jiafeng Fan, Yixing Lan, Yanyan Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Query understanding is a fundamental problem in information retrieval (IR), which has attracted continuous attention through the past decades. Many different tasks have been proposed for understanding users’ search queries, e.g., query classification or query clustering. However, it is not that precise to understand a search query at the intent class/cluster level due to the loss of many detailed information. As we may find in many benchmark datasets, e.g., TREC and SemEval, queries are often associated with a detailed description provided by human annotators which clearly describes its intent to help evaluate the relevance of the documents. If a system could automatically generate a detailed and precise intent description for a search query, like human annotators, that would indicate much better query understanding has been achieved. In this paper, therefore, we propose a novel Query-to-Intent-Description (Q2ID) task for query understanding. Unlike those existing ranking tasks which leverage the query and its description to compute the relevance of documents, Q2ID is a reverse task which aims to generate a natural language intent description based on both relevant and irrelevant documents of a given query. To address this new task, we propose a novel Contrastive Generation model, namely CtrsGen for short, to generate the intent description by contrasting the relevant documents with the irrelevant documents given a query. We demonstrate the effectiveness of our model by comparing with several state-of-the-art generation models on the Q2ID task. We discuss the potential usage of such Q2ID technique through an example application. Copyright © 2020, The Authors. All rights reserved.

关键词：

LegoNet: A Fast and Exact Unlearning Architecture

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yu, Sihao Sun, Fei Guo, Jiafeng Zhang, Ruqing Cheng, Xueqi University of Chinese Academy of Sciences Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Machine unlearning aims to erase the impact of specific training samples upon deleted requests from a trained model. Retraining the model on the retained data after deletion is an effective but not efficient way due to the huge number of model parameters and re-training samples. To speed up, a natural way is to reduce such parameters and samples. However, such a strategy typically leads to a loss in model performance, which poses the challenge that increasing the unlearning efficiency while maintaining acceptable performance. In this paper, we present a novel network, namely LegoNet, which adopts the framework of "fixed encoder + multiple adapters". We fix the encoder (i.e., the backbone for representation learning) of LegoNet to reduce the parameters that need to be re-trained during unlearning. Since the encoder occupies a major part of the model parameters, the unlearning efficiency is significantly improved. However, fixing the encoder empirically leads to a significant performance drop. To compensate for the performance loss, we adopt the ensemble of multiple adapters, which are independent sub-models adopted to infer the prediction by the encoding (i.e., the output of the encoder). Furthermore, we design an activation mechanism for the adapters to further trade off unlearning efficiency against model performance. This mechanism guarantees that each sample can only impact very few adapters, so during unlearning, parameters and samples that need to be re-trained are both reduced. The empirical experiments verify that LegoNet accomplishes fast and exact unlearning while maintaining acceptable performance, synthetically outperforming unlearning baselines. © 2022, CC BY-NC-ND.

关键词： Efficiency

Controlling risk of web question answering

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Su, Lixin Guo, Jiafeng Fan, Yixing Lan, Yanyan Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Web question answering (QA) has become an indispensable component in modern search systems, which can significantly improve users’ search experience by providing a direct answer to users’ information need. This could be achieved by applying machine reading comprehension (MRC) models over the retrieved passages to extract answers with respect to the search query. With the development of deep learning techniques, state-of-the-art MRC performances have been achieved by recent deep methods. However, existing studies on MRC seldom address the predictive uncertainty issue, i.e., how likely the prediction of an MRC model is wrong, leading to uncontrollable risks in real-world Web QA applications. In this work, we first conduct an in-depth investigation over the risk of Web QA. We then introduce a novel risk control framework, which consists of a qualify model for uncertainty estimation using the probe idea, and a decision model for selectively output. For evaluation, we introduce risk-related metrics, rather than the traditional EM and F1 in MRC, for the evaluation of risk-aware Web QA. The empirical results over both the real-world Web QA dataset and the academic MRC benchmark collection demonstrate the effectiveness of our approach. Copyright © 2019, The Authors. All rights reserved.

关键词： Deep learning

Visual Named Entity Linking: A New dataset and A Baseline

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Sun, Wenxiang Fan, Yixing Guo, Jiafeng Zhang, Ruqing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Visual Entity Linking (VEL) is a task to link regions of images with their corresponding entities in Knowledge Bases (KBs), which is beneficial for many computer vision tasks such as image retrieval, image caption, and visual question answering. While existing tasks in VEL either rely on textual data to complement a multi-modal linking or only link objects with general entities, which fails to perform named entity linking on large amounts of image data. In this paper, we consider a purely Visual-based Named Entity Linking (VNEL) task, where the input only consists of an image. The task is to identify objects of interest (i.e., visual entity mentions) in images and link them to corresponding named entities in KBs. Since each entity often contains rich visual and textual information in KBs, we thus propose three different sub-tasks, i.e., visual to visual entity linking (V2VEL), visual to textual entity linking (V2TEL), and visual to visual-textual entity linking (V2VTEL). In addition, we present a high-quality human-annotated visual person linking dataset, named WIKIPerson. Based on WIKIPerson, we establish a series of baseline algorithms for the solution of each sub-task, and conduct experiments to verify the quality of the proposed datasets and the effectiveness of baseline methods. We envision this work to be helpful for soliciting more works regarding VNEL in the future. Copyright © 2022, The Authors. All rights reserved.

关键词： Image retrieval

Locally smoothed neural networks

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Pang, Liang Lan, Yanyan Xu, Jun Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Convolutional Neural networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel locally smoothed neural network (LSNN) in this paper. The main idea is to represent the weight matrix of the locally connected layer as the product of the kernel and the smoother, where the kernel is shared over different local receptive fields, and the smoother is for determining the importance and relations of different local receptive fields. Specifically, a multi-variate Gaussian function is utilized to generate the smoother, for modeling the location relations among different local receptive fields. Furthermore, the content information can also be leveraged by setting the mean and precision of the Gaussian function according to the content. Experiments on some variant of MNIST clearly show our advantages over CNN and locally connected layer. Copyright © 2017, The Authors. All rights reserved.

关键词： Convolution

A linguistic study on relevance modeling in information retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fan, Yixing Guo, Jiafeng Ma, Xinyu Zhang, Ruqing Lan, Yanyan Cheng, Xueqi University of Chinese Academy of Sciences Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Relevance plays a central role in information retrieval (IR), which has received extensive studies starting from the 20th century. The definition and the modeling of relevance has always been critical challenges in both information science and computer science research areas. Along with the debate and exploration on relevance, IR has already become a core task in many real-world applications, such as Web search engines, question answering systems, conversational bots, and so on. While relevance acts as a unified concept in all these retrieval tasks, the inherent definitions are quite different due to the heterogeneity of these tasks. This raises a question to us: Do these different forms of relevance really lead to different modeling focuses? To answer this question, in this work, we conduct an empirical study on relevance modeling in three representative IR tasks, i.e., document retrieval, answer retrieval, and response retrieval. Specifically, we attempt to study the following two questions: 1) Does relevance modeling in these tasks really show differences in terms of natural language understanding (NLU)? We employ 16 linguistic tasks to probe a unified retrieval model over these three retrieval tasks to answer this question. 2) If there do exist differences, how can we leverage the findings to enhance the relevance modeling? We proposed three intervention methods to investigate how to leverage different modeling focuses of relevance to improve these IR tasks. We believe the way we study the problem as well as our findings would be beneficial to the IR community. © 2021, CC BY.

关键词： Information retrieval

Rank internet service quality using multiple features: A machine learning approach

学校读者我要写书评

暂无评论

Rank internet service quality using multiple features: A mac...

6th International Conference on Semantics, Knowledge and Grid, SKG 2010

作者： Tu, Dandan Shu, Chengchun Shi, Jingwei Zhu, Tao Wang, Shuang Yu, Haiyan Key Laboratory of Network Science and Technology ICT Institute of Computing Technology CAS P.O.Box 2704 100190 Beijing China NEC Labs China 11F Bldg Innovation Plaza Tsinghua Science Park Beijing 100084 China

ISBN: (纸本)9780769541891

This paper addresses the problem of Ranking Internet service quality by taking a machine learning approach using multiple service features. Ranking helps find good services for applications that use services as building blocks. Unlike other ranking problems, the goodness of Internet service qualities is dependent upon multiple key features. The key features vary across different service categories and have unequally discriminative natures. This paper divides the ranking problem into four subtasks including categorizing services according to the service functionalities, identifying key features that determine the quality, denoising for feature measurement values and computing global ranking scores with multiple key features, which are cast into machine learning problems and solved using techniques classification, feature selection, clustering, and regression respectively. In particular, we propose in this paper an efficient dense-block based denoising method for subjective features, and a Supported Vector Regression based method for computing global ranking scores. Experimental results on both synthetic and real data show that the proposed approach can quantitatively identify the key features across service categories, discard noisy measurement values in 10 times faster, and compute the global ranking scores using multiple features with low mean squared errors for both linear and nonlinear ranking functions. © 2010 IEEE.

关键词： Mean square error

Adversarial Immunization for Certifiable Robustness on Graphs

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Tao, Shuchang Shen, Huawei Cao, Qi Hou, Liang Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Despite achieving strong performance in semi-supervised node classification task, graph neural networks (GNNs) are vulnerable to adversarial attacks, similar to other deep learning models. Existing researches focus on developing either robust GNN models or attack detection methods against adversarial attacks on graphs. However, little research attention is paid to the potential and practice of immunization to adversarial attacks on graphs. In this paper, we propose and formulate the graph adversarial immunization problem, i.e., vaccinating an affordable fraction of node pairs, connected or unconnected, to improve the certifiable robustness of graph against any admissible adversarial attack. We further propose an effective algorithm, called AdvImmune, which optimizes with meta-gradient in a discrete way to circumvent the computationally expensive combinatorial optimization when solving the adversarial immunization problem. Experiments are conducted on two citation networks and one social network. Experimental results demonstrate that the proposed AdvImmune method remarkably improves the ratio of robust nodes by 12%, 42%, 65%, with an affordable immune budget of only 5% edges. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph neural networks