检索结果-内蒙古大学图书馆

Evaluating Interpolation and Extrapolation Performance of Neural Retrieval Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhan, Jingtao Xie, Xiaohui Mao, Jiaxin Liu, Yiqun Guo, Jiafeng Zhang, Min Ma, Shaoping Department of Computer Science and Technology Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China Beijing Key Laboratory of Big Data Management and Analysis Methods Gaoling School of Artificial Intelligence Renmin University of China Beijing100872 China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

A retrieval model should not only interpolate the training data but also extrapolate well to the queries that are different from the training data. While neural retrieval models have demonstrated impressive performance on ad-hoc search benchmarks, we still know little about how they perform in terms of interpolation and extrapolation. In this paper, we demonstrate the importance of separately evaluating the two capabilities of neural retrieval models. Firstly, we examine existing ad-hoc search benchmarks from the two perspectives. We investigate the distribution of training and test data and find a considerable overlap in query entities, query intent, and relevance labels. This finding implies that the evaluation on these test sets is biased toward interpolation and cannot accurately reflect the extrapolation capacity. Secondly, we propose a novel evaluation protocol to separately evaluate the interpolation and extrapolation performance on existing benchmark datasets. It resamples the training and test data based on query similarity and utilizes the resampled dataset for training and evaluation. Finally, we leverage the proposed evaluation protocol to comprehensively revisit a number of widely-adopted neural retrieval models. Results show models perform differently when moving from interpolation to extrapolation. For example, representation-based retrieval models perform almost as well as interaction-based retrieval models in terms of interpolation but not extrapolation. Therefore, it is necessary to separately evaluate both interpolation and extrapolation performance and the proposed resampling method serves as a simple yet effective evaluation tool for future IR studies. Copyright © 2022, The Authors. All rights reserved.

关键词： Extrapolation

Semantic models for the first-stage retrieval: a comprehensive review

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Guo, Jiafeng Cai, Yinqiong Fan, Yixing Sun, Fei Zhang, Ruqing Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences NO. 6 Kexueyuan South Road Haidian District Beijing100190 China Alibaba Group Beijing100102 China

Multi-stage ranking pipelines have been a practical solution in modern search systems, where the first-stage retrieval is to return a subset of candidate documents, and latter stages attempt to re-rank those candidates. Unlike re-ranking stages going through quick technique shifts during past decades, the first-stage retrieval has long been dominated by classical term-based models. Unfortunately, these models suffer from the vocabulary mismatch problem, which may block re-ranking stages from relevant documents at the very beginning. Therefore, it has been a long-term desire to build semantic models for the first-stage retrieval that can achieve high recall efficiently. Recently, we have witnessed an explosive growth of research interests on the first-stage semantic retrieval models. We believe it is the right time to survey current status, learn from existing methods, and gain some insights for future development. In this paper, we describe the current landscape of the first-stage retrieval models under a unified framework to clarify the connection between classical term-based retrieval methods, early semantic retrieval methods and neural semantic retrieval methods. Moreover, we identify some open challenges and envision some future directions, with the hope of inspiring more researches on these important yet less investigated topics. © 2021, CC BY-NC-ND.

关键词： Surveys

A Deep Look into Neural Ranking Models for Information Retrieval

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Guo, Jiafeng Fan, Yixing Pang, Liang Yang, Liu Ai, Qingyao Zamani, Hamed Wu, Chen Croft, W. Bruce Cheng, Xueqi University of Chinese Academy of Sciences Beijing China Cas Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China Center for Intelligent Information Retrieval University of Massachusetts Amherst AmherstMA United States

Ranking models lie at the heart of research on information retrieval (IR). During the past decades, different techniques have been proposed for constructing ranking models, from traditional heuristic methods, probabilistic methods, to modern machine learning methods. Recently, with the advance of deep learning technology, we have witnessed a growing body of work in applying shallow or deep neural networks to the ranking problem in IR, referred to as neural ranking models in this paper. The power of neural ranking models lies in the ability to learn from the raw text inputs for the ranking problem to avoid many limitations of hand-crafted features. Neural networks have sufficient capacity to model complicated tasks, which is needed to handle the complexity of relevance estimation in ranking. Since there have been a large variety of neural ranking models proposed, we believe it is the right time to summarize the current status, learn from existing methodologies, and gain some insights for future development. In contrast to existing reviews, in this survey, we will take a deep look into the neural ranking models from different dimensions to analyze their underlying assumptions, major design principles, and learning strategies. We compare these models through benchmark tasks to obtain a comprehensive empirical understanding of the existing techniques. We will also discuss what is missing in the current literature and what are the promising and desired future directions. Copyright © 2019, The Authors. All rights reserved.

关键词： Information retrieval

Interpreting dense retrieval as mixture of topics

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhan, Jingtao Mao, Jiaxin Liu, Yiqun Guo, Jiafeng Zhang, Min Ma, Shaoping Department of Computer Science and Technology Institute for Artificial Intelligence Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China Beijing Key Laboratory of Big Data Management and Analysis Methods Gaoling School of Artificial Intelligence Renmin University of China Beijing100872 China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Dense Retrieval (DR) reaches state-of-the-art results in first-stage retrieval, but little is known about the mechanisms that contribute to its success. Therefore, in this work, we conduct an interpretation study of recently proposed DR models. Specifically, we first discretize the embeddings output by the document and query encoders. Based on the discrete representations, we analyze the attribution of input tokens. Both qualitative and quantitative experiments are carried out on public test collections. Results suggest that DR models pay attention to different aspects of input and extract various high-level topic representations. Therefore, we can regard the representations learned by DR models as a mixture of high-level topics. Copyright © 2021, The Authors. All rights reserved.

关键词： Mixtures

Classification Based on Multilayer Extreme Learning Machine for Motor Imagery Task from EEG Signals

学校读者我要写书评

暂无评论

Procedia Computer science 2016年 88卷 176-184页

作者： Lijuan Duan Menghu Bao Jun Miao Yanhui Xu Juncheng Chen College of Computer Science and Technology Beijing University of Technology Beijing 100124 China Beijing Key Laboratory of Trusted Computing National Engineering Laboratory for Critical Technologies of Information Security Classified Protection Beijing 100124 China Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing 100190 China Beijing Key Laboratory on Integration and Analysis of Large-scale Stream Data Beijing 100124 China

Classification of motor imagery electroencephalogram (EEG) is one of the most important technologies for BCI. To improve the accuracy, this paper introduces a classification system based on Multilayer Extreme Learning Machine (ML-ELM). In the system, the combination of PCA and LDA is chosen as the method of feature extraction and the ML-ELM is used to classify. The ML-ELM has not only the advantage which ELM has but also better performance than ELM. In the experiment, our method is compared with the methods based on ELM, such as kernel-ELM, Constrained-ELM and V-ELM, and some state-of–the-art methods on the same dataset. The experimental results show that ML-ELM is much more suitable for motor imagery EEG data and has better performance than the others.

关键词： Electroencephalogram Classification Motor Imagery Extreme Learning Machine Multilayer Extreme Learning Machine

Learning discrete representations via constrained clustering for effective and efficient dense retrieval

学校读者我要写书评

暂无评论

arXiv 2021年

Dense Retrieval (DR) has achieved state-of-the-art first-stage ranking effectiveness. However, the efficiency of most existing DR models is limited by the large memory cost of storing dense vectors and the time-consuming nearest neighbor search (NNS) in vector space. Therefore, we present RepCONC, a novel retrieval model that learns discrete Representations via CONstrained Clustering. RepCONC jointly trains dual-encoders and the Product Quantization (PQ) method to learn discrete document representations and enables fast approximate NNS with compact indexes. It models quantization as a constrained clustering process, which requires the document embeddings to be uniformly clustered around the quantization centroids and supports end-to-end optimization of the quantization method and dual-encoders. We theoretically demonstrate the importance of the uniform clustering constraint in RepCONC and derive an efficient approximate solution for constrained clustering by reducing it to an instance of the optimal transport problem. Besides constrained clustering, RepCONC further adopts a vector-based inverted file system (IVF) to support highly efficient vector search on CPUs. Extensive experiments on two popular ad-hoc retrieval benchmarks show that RepCONC achieves better ranking effectiveness than competitive vector quantization baselines under different compression ratio settings. It also substantially outperforms a wide range of existing retrieval models in terms of retrieval effectiveness, memory efficiency, and time efficiency. Copyright © 2021, The Authors. All rights reserved.

关键词： Signal encoding

A quantum-inspired classical algorithm for separable non-negative matrix factorization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Chen, Zhihuai Li, Yinan Sun, Xiaoming Yuan, Pei Zhang, Jialin CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China Centrum Wiskunde & Informatica and QuSoft Science Park 123 Amsterdam1098XG Netherlands

Non-negative Matrix Factorization (NMF) asks to decompose a (entry-wise) non-negative matrix into the product of two smaller-sized nonnegative matrices, which has been shown intractable in general. In order to overcome this issue, separability assumption is introduced which assumes all data points are in a conical hull. This assumption makes NMF tractable and is widely used in text analysis and image processing, but still impractical for huge-scale datasets. In this paper, inspired by recent development on dequantizing techniques, we propose a new classical algorithm for separable NMF problem. Our new algorithm runs in polynomial time in the rank and logarithmic in the size of input matrices, which achieves an exponential speedup in the low-rank setting. Copyright © 2019, The Authors. All rights reserved.

关键词： Non-negative matrix factorization

Account matching across heterogeneous networks

学校读者我要写书评

暂无评论

Account matching across heterogeneous networks

International Conference on Game Theory for networks, GameNets

作者： Qiang Liu Jingyuan Li Yuanzhuo Wang Guoliang Xing Yan Ren University of Chinese Academy of Sciences Beijing China CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China National Computer Network Emergency Response Technical Team Coordination Center of China Beijing China

ISBN: (纸本)9781479979035

Due to the development of web services, many social network sites, as well as online shopping sites have been booming in the past decade, where it is a common phenomenon that people are likely to use multiple services at the same time. Discovering the correspondence between accounts of a same individual is a crucial prerequisite for many interesting cross network applications, such as improving the recommendation performance of the online shopping sites by using extra information from social network services. In this paper, we propose a gametheoretic method to identify correlation accounts of individuals between social network sites and online shopping sites with stable matching model, incorporating accounts profiles as well as historical behaviors. The results show that our method identifies up to 70% of correlation accounts between Facebook and eBay, one of the most popular social network sites and online shopping sites in the world, respectively.

关键词： Games Lead

ProtoEM: A Prototype-Enhanced Matching Framework for Event Relation Extraction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hu, Zhilei Li, Zixuan Xu, Daozhu Bai, Long Jin, Cheng Jin, Xiaolong Guo, Jiafeng Cheng, Xueqi CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences China School of Computer Science and Technology University of Chinese Academy of Sciences China State Key Laboratory of Geo-Information Engineering Xi’an710054 China Xi’an Research Institute of Surveying and Mapping Xi’an710054 China

Event Relation Extraction (ERE) aims to extract multiple kinds of relations among events in texts. However, existing methods singly categorize event relations as different classes, which are inadequately capturing the intrinsic semantics of these relations. To comprehensively understand their intrinsic semantics, in this paper, we obtain prototype representations for each type of event relation and propose a Prototype-Enhanced Matching (ProtoEM) framework for the joint extraction of multiple kinds of event relations. Specifically, ProtoEM extracts event relations in a two-step manner, i.e., prototype representing and prototype matching. In the first step, to capture the connotations of different event relations, ProtoEM utilizes examples to represent the prototypes corresponding to these relations. Subsequently, to capture the interdependence among event relations, it constructs a dependency graph for the prototypes corresponding to these relations and utilized a Graph Neural network (GNN)-based module for modeling. In the second step, it obtains the representations of new event pairs and calculates their similarity with those prototypes obtained in the first step to evaluate which types of event relations they belong to. Experimental results on the MAVEN-ERE dataset demonstrate that the proposed ProtoEM framework can effectively represent the prototypes of event relations and further obtain a significant improvement over baseline models. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics