检索结果-内蒙古大学图书馆

Momentum Decoding: Open-ended Text Generation As Graph Exploration

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Lan, Tian Su, Yixuan Liu, Shuhang Huang, Heyan Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China

Open-ended text generation with autoregressive language models (LMs) is one of the core tasks in natural language processing. However, maximization-based decoding methods (e.g., greedy/beam search) often lead to the degeneration problem, i.e., the generated text is unnatural and contains undesirable repetitions. Existing solutions to this problem either introduce randomness prone to incoherence or require a look-ahead mechanism that demands extra computational overhead. In this study, we formulate open-ended text generation from a new perspective, i.e., we view it as an exploration process within a directed graph. Thereby, we understand the phenomenon of degeneration as circular loops within the directed graph. Based on our formulation, we propose a novel decoding method—momentum decoding—which encourages the LM to greedily explore new nodes outside the current graph. Meanwhile, it also allows the LM to return to the existing nodes with a momentum downgraded by a pre-defined resistance function. We extensively test our approach on three benchmarks from different domains through automatic and human evaluations. The results show that momentum decoding performs comparably with the current state of the art while enjoying notably improved inference speed and computation FLOPs. Furthermore, we conduct a detailed analysis to reveal the merits and inner workings of our approach. © 2022, CC BY.

关键词： Momentum

Singleton detection for coreference resolution via multi-window and multi-filter CNN 1

学校读者我要写书评

暂无评论

13th China Workshop on Machine Translation, CWMT 2017

作者： Li, Kenan Huang, Heyan Guo, Yuhang Jian, Ping School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Application 5 South Zhongguancun Street Beijing100081 China Beijing Advanced Innovation Center for Imaging Technology Capital Normal University Beijing100048 China

ISBN: (数字)9789811071348

ISBN: (纸本)9789811071331

Mention detection is the first and a key stage in most of coreference resolution systems. Singleton mentions are the ones which appear only once and are not mentioned in the following texts. Singleton mentions always affect the performance of coreference resolution systems. To remove the singleton ones from the automatically predicted mentions, we propose a novel singleton detection method based on multi-window and multi-filter convolutional neural network (MMCNN). The MMCNN model can detect singleton mentions with less use of hand-designed features and more sentence information. Experiments show that our system outperforms all the existing singleton detection systems. © Springer Nature Singapore Pte Ltd. 2017.

关键词： Convolution

A multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis in the frequency domain

学校读者我要写书评

暂无评论

Advances in Intelligent Systems and Computing

Advances in Intelligent Systems and computing 2014年 215卷 603-616页

作者： Chen, Guang Song, Dandan Liao, Lejian Beijing Eng. Research Center of High Volume Language Inf. Processing and Cloud Computing App. Beijing Lab of Intelligent Information Technology School of Computer Science Beijing Institute of Technology Beijing China

Brain-computer interface (BCI) is a new way for man-machine interaction with wide applications, in which steady-state visual evoked potentials (SSVEP) is a promising option. However, many characteristics of SSVEP show great user variation. So parameter optimization and channel selection for each subject were applied to improve the performance of BCI. These optimizations limit the practical applicability of the SSVEP-based BCI. The use of a canonical correlation analysis (CCA) method for multi-channel SSVEP in the time domain detection showed highly increased detection accuracy, but it is sensitive to the noise when the stimulate frequency is low. In this paper, a method of CCA in the frequency domain is presented for classifying multi-channel SSVEPs. First overlapping average is conducted on the original training signals. Then fast Fourier transform (FFT) is used to transform the signals from time domain to frequency domain to produce the reference data. Finally, according to the correlation coefficients of the new data and the references in the frequency domain, the SSVEP is classified. The experimental results show the enhanced accuracy of our method when applied to low stimulate frequencies. © Springer-Verlag Berlin Heidelberg 2014.

关键词： Brain computer interface

Learning automata-based adaptive web services composition

学校读者我要写书评

暂无评论

Learning automata-based adaptive web services composition

IEEE International Conference on Software Engineering and Service Sciences (ICSESS)

作者： Guoqiang Li Dandan Song Lejian Liao Fuzhen Sun Jianguang Du School of informatics Linyi University Linyi China Language Information Processing & Cloud Computing Applications Beijing Institute of Technology Beijing China Beijing Engineering Research Centre of High Volume Language Information Processing & Cloud Computing Applications School of Computer Science Beijing Institute of Technology Beijing China

ISBN: (纸本)9781479932801

Service-oriented computing is a widely adopted paradigm in real applications. Considering the continuous evolution of services, adaptive service composition has always been a major concern. It is a big challenge to adjust the composition to be optimal in real-time. In this paper, a learning automata-based approach is proposed to attack this problem. It consists of two important components: random environment and a learning automaton. The former can be mapped to the service's execution environment. The latter is responsible for the adaptation achievement using reward and penalty functions, while we take the service composition structures into account to compute the usefulness value of all services. At last, simulation study has shown that our approach is efficient to find the optimal (sub-optimal) composition.

关键词： Learning automata Quality of service Web services Adaptation models Automata Conferences Real-time systems

A Parallel Cross-language Retrieval System for Patent Documents

学校读者我要写书评

暂无评论

A Parallel Cross-Language Retrieval System for Patent Docume...

2015 6th IEEE International Conference on Software Engineering and Service Science(ICSESS 2015)

作者： Xin Shen Heyan Huang Lingzhi Li Yonggang Huang Beijing Engineering Research Center of High Volume Language Information Processing & Cloud Computing Application Beijing Institute of Technology Overhaul Branch Beijing Electric Power Company

In order to help people obtain useful information from patent documents in different languages. This paper proposes a cross-language retrieval system to search Chinese and English patent documents simultaneously. This system consists of query translation module, document retrieval module and user interaction module. Query translation module is used to translate query based on bilingual dictionaries. Document retrieval module consists of monolingual retrieval system using standard vector space model. In order to retrieve in highly parallel, we use the Map Reduce model to calculate the similarity. User interaction module provides users with interactive mechanism used to improve the retrieval accuracy in the system. It contains two parts: the second translation and relevance feedback. The experimental results show that our system has good performance.

关键词： Patent document cross-language information retrieval parallel retrieval Hadoop

Neural Chinese word segmentation as sequence to sequence translation

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Shi, Xuewen Huang, Heyan Jian, Ping Guo, Yuhang Wei, Xiaochi Tang, Yikun Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China

Recently, Chinese word segmentation (CWS) methods using neural networks have made impressive progress. Most of them regard the CWS as a sequence labeling problem which construct models based on local features rather than considering global information of input sequence. In this paper, we cast the CWS as a sequence translation problem and propose a novel sequence-to-sequence CWS model with an attention-based encoder-decoder framework. The model captures the global information from the input and directly outputs the segmented sequence. It can also tackle other NLP tasks with CWS jointly in an end-to-end mode. Experiments on Weibo, PKU and MSRA benchmark datasets show that our approach has achieved competitive performances compared with state-of-the-art methods. Meanwhile, we successfully applied our proposed model to jointly learning CWS and Chinese spelling correction, which demonstrates its applicability of multi-task fusion. Copyright © 2019, The Authors. All rights reserved.

关键词： Natural language processing systems

Concept pointer network for abstractive summarization

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wenbo, Wang Yang, Gao Heyan, Huang Yuxiang, Zhou School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications

A quality abstractive summary should not only copy salient source texts as summaries but should also tend to generate new conceptual words to express concrete details. Inspired by the popular pointer generator sequence-to-sequence model, this paper presents a concept pointer network for improving these aspects of abstractive summarization. The network leverages knowledge-based, context-aware conceptualizations to derive an extended set of candidate concepts. The model then points to the most appropriate choice using both the concept set and original source text. This joint approach generates abstractive summaries with higher-level semantic concepts. The training model is also optimized in a way that adapts to different data, which is based on a novel method of distantly-supervised learning guided by reference summaries and testing set. Overall, the proposed approach provides statistically significant improvements over several state-of-the-art models on both the DUC-2004 and Gigaword datasets. A human evaluation of the model's abstractive abilities also supports the quality of the summaries produced within this *** Codes 68U15, 68T50 Copyright © 2019, The Authors. All rights reserved.

关键词： Knowledge based systems

Tag recommendation by word-level tag sequence modeling

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Shi, Xuewen Huang, Heyan Zhao, Shuyang Jian, Ping Tang, Yi-Kun School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications

Recently, neural network based tag recommendation has gained much attention. Most of these methods regard tag recommendation as multi-label classification. They treat different tags as individual categories with fixed labels, without considering the rich relations among tags. Moreover, using a fixed number of categories cannot tackle dynamic, changing tags with ongoing topics. In this paper, we transform tag recommendation into a word-based text generation problem and introduce a sequence-to-sequence model. For efficiently modeling the semantic dependencies among tags in tag sequence and the strong sequential relations among the tag-words, we propose an essential sequence-to-sequence model, named LSTM-Attention. The model inherits the advantages of recurrent network based encoder for sequential modeling and attention based decoder for learning relations globally. In addition, as a text generation method, the proposed model is able to generate unseen tags, which is more applicable and flexible to real scenarios. Extensive experimental results on two datasets, i.e., Zhihu and Weibo, clearly illustrate the proposed model significantly outperforms other state-of-the-art text classification based methods and well demonstrate its advantage of handling unseen tags. Copyright © 2019, The Authors. All rights reserved.

关键词： Text processing

Personalized next point-of-interest recommendation via latent behavior patterns inference

学校读者我要写书评

暂无评论

arXiv 2018年

作者： He, Jing Li, Xin Liao, Lejian Cheung, William K. Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Application School of Computer Science and Technology Beijing Institute of Technology Beijing10081 China

In this paper, we address the problem of personalized next Point-of-interest (POI) recommendation which has become an important and very challenging task for location-based social networks (LBSNs), but not well studied yet. With the conjecture that, under different contextual scenarios, human exhibits distinct mobility pattern, we attempt here to jointly model the next POI recommendation under the influence of user's latent behavior pattern. We propose to adopt a third-rank tensor to model the successive check-in behaviors. By integrating categorical influence into mobility patterns and aggregating user's spatial preference on a POI, the proposed model deal with the next new POI recommendation problem by nature. By incorporating softmax function to fuse the personalized Markov chain with latent pattern, we furnish a Bayesian Personalized Ranking (BPR) approach and derive the optimization criterion accordingly. Expectation Maximization (EM) is then used to estimate the model parameters. We further develop a personalized model by taking into account personalized mobility patterns under the contextual scenario to improve the recommendation performance. Extensive experiments on two large-scale LBSNs datasets demonstrate the significant improvements of our model over several state-of-the-art methods. Copyright © 2018, The Authors. All rights reserved.

关键词： Maximum principle