检索结果-内蒙古大学图书馆

aNMM: Ranking short answer texts with attention-based neural matching model

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yang, Liu Ai, Qingyao Guo, Jiafeng Bruce Croft, W. Center for Intelligent Information Retrieval University of Massachusetts AmherstMA United States CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

As an alternative to question answering methods based on feature engineering, deep learning approaches such as convolutional neural networks (CNNs) and Long Short-Term Memory Models (LSTMs) have recently been proposed for semantic matching of questions and answers. To achieve good results, however, these models have been combined with additional features such as word overlap or BM25 scores. Without this combination, these models perform significantly worse than methods based on linguistic feature engineering. In this paper, we propose an attention based neural matching model for ranking short answer text. We adopt value-shared weighting scheme instead of position-shared weighting scheme for combining different matching signals and incorporate question term importance learning using question attention network. Using the popular benchmark TREC QA data, we show that the relatively simple aNMM model can significantly outperform other neural network models that have been used for the question answering task, and is competitive with models that are combined with additional features. When aNMM is combined with additional features, it outperforms all baselines. Copyright © 2018, The Authors. All rights reserved.

关键词： Semantics

A generalization theory based on independent and task-identically distributed assumption

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zheng, Guanhua Sang, Jitao Li, Houqiang Yu, Jian Xu, Changsheng University of Science and Technology of China School of Computer and Information Technology Beijing Key Laboratory of Traffic Data Analysis and Mining Beijing Jiaotong University Beijing100044 China Chinese Academy of Sciences Key Laboratory of Technology in Geo-Spatial Information Processing and Application System Hefei230026 China National Lab of Pattern Recognition Institute of Automation CAS Beijing100190 China University of Chinese Academy of Sciences

—Existing generalization theories analyze the generalization performance mainly based on the model complexity and training process. The ignorance of the task properties, which results from the widely used IID assumption, makes these theories fail to interpret many generalization phenomena or guide practical learning tasks. In this paper, we propose a new Independent and Task-Identically Distributed (ITID) assumption, to consider the task properties into the data generating process. The derived generalization bound based on the ITID assumption identifies the significance of hypothesis invariance in guaranteeing generalization performance. Based on the new bound, we introduce a practical invariance enhancement algorithm from the perspective of modifying data distributions. Finally, we verify the algorithm and theorems in the context of image classification task on both toy and real-world datasets. The experimental results demonstrate the reasonableness of the ITID assumption and the effectiveness of new generalization theory in improving practical generalization performance. Copyright © 2019, The Authors. All rights reserved.

关键词： Classification (of information)

Learning a deep listwise context model for ranking refinement

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Ai, Qingyao Bi, Keping Guo, Jiafeng Croft, W. Bruce College of Information and Computer Sciences University of Massachusetts Amherst AmherstMA United States CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China

Learning to rank has been intensively studied and widely applied in information retrieval. Typically, a global ranking function is learned from a set of labeled data, which can achieve good performance on average but may be suboptimal for individual queries by ignoring the fact that relevant documents for different queries may have different distributions in the feature space. Inspired by the idea of pseudo relevance feedback where top ranked documents, which we refer as the local ranking context, can provide important information about the query's characteristics, we propose to use the inherent feature distributions of the top results to learn a Deep Listwise Context Model that helps us fine tune the initial ranked list. Specifically, we employ a recurrent neural network to sequentially encode the top results using their feature vectors, learn a local context model and use it to re-rank the top results. There are three merits with our model: (1) Our model can capture the local ranking context based on the complex interactions between top results using a deep neural network;(2) Our model can be built upon existing learning-to-rank methods by directly using their extracted feature vectors;(3) Our model is trained with an attention-based loss function, which is more effective and efficient than many existing listwise methods. Experimental results show that the proposed model can significantly improve the state-of-the-art learning to rank methods on benchmark retrieval corpora. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep neural networks

On the relationship between energy complexity and other boolean function measures

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Sun, Xiaoming Sun, Yuan Wu, Kewen Xia, Zhiyu CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China School of Electronics Engineering and Computer Science Peking University China

In this work we investigate energy complexity, a Boolean function measure related to circuit complexity. Given a circuit C over the standard basis {∨2, ∧2, ¬}, the energy complexity of C, denoted by EC(C), is the maximum number of its activated inner gates over all inputs. The energy complexity of a Boolean function f, denoted by EC(f), is the minimum of EC(C) over all circuits C computing f. Recently, K. Dinesh et al. [3] gave EC(f) an upper bound in terms of the decision tree complexity, EC(f) = O(D(f)3). They also showed that EC(f) ≤ 3n−1, where n is the input size. For the lower bound, they show that EC(f) ≥ 13 psens(f), where psens(f) is the positive sensitivity. They asked whether EC(f) can be lower bounded by a polynomial of D(f). We improve both the upper and lower bounds in this paper. For upper bounds, We show that EC(f) ≤ min{ 12 D(f)2 + O(D(f)), n + 2 D(f) − 2}. For the lower bound, we answer Dinesh et al.'s question by proving that EC(f) = Ω(pD(f)). For non-degenerated functions, we also give another lower bound EC(f) = Ω(log n) where n is the input size. These two lower bounds are incomparable to each other. Besides, we examine the energy complexity of OR functions and ADDRESS functions, which implies the tightness of our two lower bounds respectively. In addition, the former one answers another open question in [3] asking for non-trivial lower bound for energy complexity of OR functions. Copyright © 2018, The Authors. All rights reserved.

关键词： Decision trees

Structured decomposition for reversible Boolean functions

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Jiang, Jiaqing Sun, Xiaoming Sun, Yuan Wu, Kewen Xia, Zhiyu CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China School of Electronics Engineering and Computer Science Peking University Beijing China

Reversible Boolean function is a one-to-one function which maps n-bit input to n-bit output. Reversible logic synthesis has been widely studied due to its relationship with low-energy computation as well as quantum computation. In this work, we give a structured decomposition for even reversible Boolean functions (RBF). Specifically, for n ≥ 6, any even n-bit RBF can be decomposed to 7 blocks of (n − 1)-bit RBF, where 7 is a constant independent of n;and the positions of those blocks have large degree of freedom. Moreover, if the (n − 1)-bit RBFs are required to be even as well, we show for n ≥ 10, n-bit RBF can be decomposed to 10 even (n − 1)-bit RBFs. For simplicity, we say our decomposition has block depth 7 and even block depth 10. Our result improves Selinger’s work in block depth model, by reducing the constant from 9 to 7;and from 13 to 10 when the blocks are limited to be even. We emphasize that our setting is a bit different from Selinger’s. In Selinger’s constructive proof, each block is one of two specific positions and thus the decomposition has an alternating structure. We relax this restriction and allow each block to act on arbitrary (n − 1) bits. This relaxation keeps the block structure and provides more candidates when choosing positions of blocks. Copyright © 2018, The Authors. All rights reserved.

关键词： Logic gates

Relative Importance Sampling for off-Policy Actor-Critic in Deep Reinforcement Learning

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Humayoo, Mahammad Cheng, Xueqi CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100190 China

Off-policy learning is more unstable compared to on-policy learning in reinforcement learning (RL). One of the reasons for instability of off-policy learning is a discrepancy between target (π) and behavior (b) policy distribution. The discrepancy between π and b distribution can be alleviated by employing the smooth variant of importance sampling (IS), such as relative importance sampling (RIS).The RIS has parameter β ∈ [0, 1] that controls the smoothness. To cope with the instability of off-policy learning, we present the first relative importance sampling-off-policy actor-critic (RIS-off-PAC) model-free algorithms in RL. In our method, A network yields the target policy (actor), the value function (critic) assessing the current policy (π) using samples drawn from the behavior policy. We use action values generated from the behavior policy in reward function to train our algorithms rather than from the target policy. We also use deep neural networks to train both the actor and critic. We evaluated our algorithms on a number of OpenAI Gym problems and demonstrated better or comparable performance to several state-of-the-art RL baselines. Copyright © 2018, The Authors. All rights reserved.

关键词： Reinforcement learning

Locally smoothed neural networks 9

学校读者我要写书评

暂无评论

Locally smoothed neural networks

9th Asian Conference on Machine Learning, ACML 2017

作者： Pang, Liang Lan, Yanyan Xu, Jun Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Convolutional Neural networks (CNN) and the locally connected layer are limited in capturing the importance and relations of different local receptive fields, which are often crucial for tasks such as face verification, visual question answering, and word sequence prediction. To tackle the issue, we propose a novel locally smoothed neural network (LSNN) in this paper. The main idea is to represent the weight matrix of the locally connected layer as the product of the kernel and the smoother, where the kernel is shared over different local receptive fields, and the smoother is for determining the importance and relations of different local receptive fields. Specifically, a multi-variate Gaussian function is utilized to generate the smoother, for modeling the location relations among different local receptive fields. Furthermore, the content information can also be leveraged by setting the mean and precision of the Gaussian function according to the content. Experiments on some variant of MNIST clearly show our advantages over CNN and locally connected layer. © 2017 L. Pang†∗, Y. Lan†, J. Xu†, J. Guo† & X. Cheng†.

关键词： Convolution

Modeling Diverse Relevance Patterns in Ad-hoc Retrieval

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Fan, Yixing Guo, Jiafeng Lan, Yanyan Xu, Jun Zhai, Chengxiang Cheng, Xueqi University of Chinese Academy of Sciences Beijing China CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China Department of Computer Science University of Illinois at Urbana-Champaign IL United States

Assessing relevance between a query and a document is challenging in ad-hoc retrieval due to its diverse patterns, i.e., a document could be relevant to a query as a whole or partially as long as it provides sufficient information for users’ need. Such diverse relevance patterns require an ideal retrieval model to be able to assess relevance in the right granularity adaptively. Unfortunately, most existing retrieval models compute relevance at a single granularity, either document-wide or passage-level, or use fixed combination strategy, restricting their ability in capturing diverse relevance patterns. In this work, we propose a data-driven method to allow relevance signals at different granularities to compete with each other for final relevance assessment. Specifically, we propose a HIerarchical Neural maTching model (HiNT) which consists of two stacked components, namely local matching layer and global decision layer. The local matching layer focuses on producing a set of local relevance signals by modeling the semantic matching between a query and each passage of a document. The global decision layer accumulates local signals into different granularities and allows them to compete with each other to decide the final relevance score. Experimental results demonstrate that our HiNT model outperforms existing state-of-the-art retrieval models significantly on benchmark ad-hoc retrieval datasets. Copyright © 2018, The Authors. All rights reserved.

关键词： Semantics

Accurate Recovery of Internet Traffic data: A Sequential Tensor Completion Approach

学校读者我要写书评

暂无评论

Accurate Recovery of Internet Traffic Data: A Sequential Ten...

作者： Xie, Kun Wang, Lele Wang, Xin Xie, Gaogang Wen, Jigang Zhang, Guangxing Cao, Jiannong Zhang, Dafang College of Computer Science and Electronics Engineering Hunan University Changsha410006 China CAS Key Laboratory of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing100864 China Department of Electrical and Computer Engineering State University of New York at Stony Brook Stony BrookNY11794 United States Network Research Center Institute of Computing Technology Chinese Academy of Sciences Beijing100864 China Department of Computing Hong Kong Polytechnic University Hong Kong Hong Kong

The inference of traffic volume of the whole network from partial traffic measurements becomes increasingly critical for various network engineering tasks, such as capacity planning and anomaly detection. Previous studies indicate that the matrix completion is a possible solution for this problem. However, as a 2-D matrix cannot sufficiently capture the spatial-temporal features of traffic data, these approaches fail to work when the data missing ratio is high. To fully exploit hidden spatial-temporal structures of the traffic data, this paper models the traffic data as a 3-way traffic tensor and formulates the traffic data recovery problem as a low-rank tensor completion problem. However, the high computation complexity incurred by the conventional tensor completion algorithms prevents its practical application for the traffic data recovery. To reduce the computation cost, we propose a novel sequential tensor completion algorithm, which can efficiently exploit the tensor decomposition result based on the previous traffic data to derive the tensor decomposition upon arriving of new data. Furthermore, to better capture the changes of data correlation over time, we propose a dynamic sequential tensor completion algorithm. To the best of our knowledge, we are the first to propose sequential tensor completion algorithms to significantly speed up the traffic data recovery process. This facilitates the modeling of Internet traffic with the tensor to well exploit the hidden structures of traffic data for more accurate missing data inference. We have done extensive simulations with the real traffic trace as the input. The simulation results demonstrate that our algorithms can achieve significantly better performance compared with the literature tensor and matrix completion algorithms even when the data missing ratio is high. © 1993-2012 IEEE.

关键词： Tensors