检索结果-内蒙古大学图书馆

Converting Continuous-Space language models into N-gram language models with Efficient Bilingual Pruning for Statistical Machine Translation

引用

ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE language INFORMATION PROCESSING 2016年第3期15卷 1–26页

作者： Wang, Rui Utiyama, Masao Goto, Isao Sumita, Eiichiro Zhao, Hai Lu, Bao-Liang Shanghai Jiao Tong Univ Ctr Brain Like Comp & Machine Intelligence Dept Comp Sci & Engn 800 Dongchuan Rd Shanghai 200240 Peoples R China Natl Inst Informat & Commun Technol Multilingual Translat Lab 3-5 Hikaridai Kyoto 6190289 Japan NHK Japan Broadcasting Corp Sci & Technol Res Labs Setagaya Ku 1-10-11 Kinuta Tokyo 1578510 Japan Natl Inst Informat & Commun Technol Kyoto 6190289 Japan Shanghai Jiao Tong Univ Key Lab Shanghai Educ Commiss Intelligent Interac 800 Dongchuan Rd Shanghai 200240 Peoples R China

The language model (LM) is an essential component of Statistical Machine Translation (SMT). In this article, we focus on developing efficient methods for LM construction. Our main contribution is that we propose a Natural N-grams based Converting (NNGC) method for transforming a Continuous-Space language model (CSLM) to a Back-off N-gram language model (BNLM). Furthermore, a Bilingual LM Pruning (BLMP) approach is developed for enhancing LMs in SMT decoding and speeding up CSLM converting. The proposed pruning and converting methods can convert a large LM efficiently by working jointly. That is, a LM can be effectively pruned before it is converted from CSLM without sacrificing performance, and further improved if an additional corpus contains out-of-domain information. For different SMT tasks, our experimental results indicate that the proposed NNGC and BLMP methods outperform the existing counterpart approaches significantly in BLEU and computational cost.

关键词： Machine translation continuous-space language model neural network language model language model pruning

来源：评论

学校读者我要写书评

暂无评论

A Sentence Segmentation Method for Ancient Chinese Texts Based on NNLM 17th

A Sentence Segmentation Method for Ancient Chinese Texts Bas...

引用

17th Chinese Lexical Semantics Workshop (CLSW)

作者： Wang, Boli Shi, Xiaodong Tan, Zhixing Chen, Yidong Wang, Weili Xiamen Univ Dept Cognit Sci Xiamen 361005 Peoples R China Xiamen Univ Collaborat Innovat Ctr Peaceful Dev Cross Strait Xiamen 361005 Peoples R China Xiamen Univ Fujian Prov Key Lab Brain Inspired Comp Xiamen 361005 Peoples R China

ISBN: (纸本)9783319495088;9783319495071

Most of ancient Chinese texts have no punctuations or segmentation of sentences. Recent researches on automatic ancient Chinese sentence segmentation usually resorted to sequence labelling models and utilized small data sets. In this paper, we propose a sentence segmentation method for ancient Chinese texts based on neural network language models. Experiments on large-scale corpora indicate that our method is effective and achieves a comparable result to the traditional CRF model. Implementing sentence length penalty, using larger Simplified Chinese corpora, or dividing corpora by ages can further improve performance of our model.

关键词： Ancient Chinese Sentence segmentation neural network language model

来源：评论

学校读者我要写书评

暂无评论

Evaluation of neural network language models In Handwritten Chinese Text Recognition 13

Evaluation of Neural Network Language Models In Handwritten ...

引用

13th IAPR International Conference on Document Analysis and Recognition (ICDAR)

作者： Wu, Yi-Chao Yin, Fei Liu, Cheng-Lin Chinese Acad Sci NLPR Inst Inst Automat Beijing Peoples R China

ISBN: (纸本)9781479918058

Handwritten Chinese text recognition based on over-segmentation and path search integrating contexts has been demonstrated successful, where language models play an important role. Recently, neural network language models (NNLMs) have shown superiority to back-off N-gram language models (BLMs) in handwriting recognition, but have not been studied in Chinese text recognition system. This paper investigates the effects of NNLMs in handwritten Chinese text recognition and compares the performance with BLMs. We trained character-level language models in 3-, 4- and 5- gram on large scale corpora and applied them in text line recognition system. Experimental results on the CASIA-HWDB database show that NNLM and BLM of the same order perform comparably, and the hybrid model by interpolating NNLM and BLM improves the recognition performance significantly.

关键词： handwritten Chinese text recognition higher order character language model neural network language model hybrid language model

来源：评论

学校读者我要写书评

暂无评论

Real-time Filtering on Interest Profiles in Twitter Stream 16

Real-time Filtering on Interest Profiles in Twitter Stream

引用

16th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL)

作者： Fei, Yue Lv, Chao Feng, Yansong Zhao, Dongyan Peking Univ Inst Comp Sci & Technol Beijing Peoples R China

ISBN: (纸本)9781450342292

The advent of Twitter has led to the ubiquitous information overload problem with a dramatic increase in the amount of tweets a user is exposed to. In this paper, we consider real-time tweet filtering with respect to users' interest profiles in public Twitter stream. While traditional filtering methods mainly focus on judging relevance of a document, we aim to retrieve relevant and novel documents to address the high redundancy of tweets. An unsupervised approach is proposed to model relevance between tweets and different profiles adaptively and a neural network language model is employed to learn semantic representation for tweets. Experiments on TREC 2015 dataset demonstrate the effectiveness of the proposed approach.

关键词： Real-time Filtering neural network language model Adaptive Thresholding

来源：评论

学校读者我要写书评

暂无评论

Bilingual Continuous-Space language model Growing for Statistical Machine Translation

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND language PROCESSING 2015年第7期23卷 1209-1220页

作者： Wang, Rui Zhao, Hai Lu, Bao-Liang Utiyama, Masao Sumita, Eiichiro Shanghai Jiao Tong Univ Dept Comp Sci & Engn Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Key Lab Shanghai Educ Commiss Intelligent Interac Shanghai 200240 Peoples R China Natl Inst Informat & Commun Technol Multilingual Translat Lab Kyoto 6190289 Japan

Larger-gram language models (LMs) perform better in statistical machine translation (SMT). However, the existing approaches have two main drawbacks for constructing larger LMs: 1) it is not convenient to obtain larger corpora in the same domain as the bilingual parallel corpora in SMT;2) most of the previous studies focus on monolingual information from the target corpora only, and redundant-grams have not been fully utilized in SMT. Nowadays, continuous-space language model (CSLM), especially neural network language model (NNLM), has been shown great improvement in the estimation accuracies of the probabilities for predicting the target words. However, most of these CSLM and NNLM approaches still consider monolingual information only or require additional corpus. In this paper, we propose a novel neural network based bilingual LM growing method. Compared to the existing approaches, the proposed method enables us to use bilingual parallel corpus for LM growing in SMT. The results show that our new method outperforms the existing approaches on both SMT performance and computational efficiency significantly.

关键词： Continuous-space language model language model growing (LMG) neural network language model statistical machine translation (SMT)

来源：评论

学校读者我要写书评

暂无评论

PARAPHRASTIC neural network language modelS

PARAPHRASTIC NEURAL NETWORK LANGUAGE MODELS

引用

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Liu, X. Gales, M. J. F. Woodland, P. C. Univ Cambridge Dept Engn Cambridge CB2 1PZ England

ISBN: (纸本)9781479928934

Expressive richness in natural languages presents a significant challenge for statistical language models (LM). As multiple word sequences can represent the same underlying meaning, only modelling the observed surface word sequence can lead to poor context coverage. To handle this issue, paraphrastic LMs were previously proposed to improve the generalization of back-off n-gram LMs. Paraphrastic neural network LMs (NNLM) are investigated in this paper. Using a paraphrastic multi-level feedforward NNLM modelling both word and phrase sequences, significant error rate reductions of 1.3% absolute (8% relative) and 0.9% absolute (5.5% relative) were obtained over the baseline n-gram and NNLM systems respectively on a state-of-the-art conversational telephone speech recognition system trained on 2000 hours of audio and 545 million words of texts.

关键词： neural network language model paraphrase speech recognition

来源：评论

学校读者我要写书评

暂无评论

Empirically combining unnormalized NNLM and back-off N-gram for fast N-best rescoring in speech recognition

引用

EURASIP JOURNAL ON AUDIO SPEECH AND MUSIC PROCESSING 2014年第1期2014卷 1页

作者： Shi, Yongzhe Zhang, Wei-Qiang Cai, Meng Liu, Jia Tsinghua Univ Dept Elect Engn Tsinghua Natl Lab Informat Sci & Technol Beijing 100084 Peoples R China

neural network language models (NNLM) have been proved to be quite powerful for sequence modeling, including feed-forward NNLM (FNNLM), recurrent NNLM (RNNLM), etc. One main issue concerned for NNLM is the heavy computational burden of the output layer, where the output needs to be probabilistically normalized and the normalizing factors require lots of computation. How to fast rescore the N-best list or lattice with NNLM attracts much attention for large-scale applications. In this paper, the statistic characteristics of normalizing factors are investigated on the N-best list. Based on the statistic observations, we propose to approximate the normalizing factors for each hypothesis as a constant proportional to the number of words in the hypothesis. Then, the unnormalized NNLM is investigated and combined with back-off N-gram for fast rescoring, which can be computed very fast without the normalization in the output layer, with the complexity reduced significantly. We apply our proposed method to a well-tuned context-dependent deep neural network hidden Markov model (CD-DNN-HMM) speech recognition system on the English-Switchboard phone-call speech-to-text task, where both FNNLM and RNNLM are trained to demonstrate our method. Experimental results show that unnormalized probability of NNLM is quite complementary to that of back-off N-gram, and combining the unnormalized NNLM and back-off N-gram can further reduce the word error rate with little computational consideration.

关键词： neural network language model N-best rescoring Speech recognition

来源：评论

学校读者我要写书评

暂无评论

Efficient One-Pass Decoding with NNLM for Speech Recognition

引用

IEEE SIGNAL PROCESSING LETTERS 2014年第4期21卷 377-381页

作者： Shi, Yongzhe Zhang, Wei-Qiang Cai, Meng Liu, Jia Tsinghua Univ Tsinghua Natl Lab Informat Sci & Technol Dept Elect & Engn Beijing 100084 Peoples R China

neural network language model (NNLM) has achieved very good results in the field of speech recognition, machine translation, etc. Direct decoding with NNLM is challenging for the overwhelmingly heavy burden in complexity. Most of the previous work focused on rescoring the N-best list and lattice with NNLM in the second pass. In this work, several techniques are explored to directly incorporate the NNLM into the decoder of speech recognition. A novel training algorithm based on variance regularization is proposed to approximate the softmax-normalizing factor as a constant for fast evaluation. Also, the evaluation of NNLM is further speeded up via our advanced storage. Moreover, a simple cache-based strategy is explored to avoid redundant computations during the decoding process. To the authors' knowledge, it is the first time to directly incorporate NNLM into decoding. We evaluate our proposed methods on an English-Switchboard phone-call speech-to-text task. Experimental results show that incorporating the NNLM into the decoder significantly reduces the word error rate (WER) by 1.5% and 1.4% absolutely on the Hub5'00-SWB and RT03S-FSH sets, respectively. Also, the decoding with NNLM is twice as fast as the baseline at the same word error rate.

关键词： neural network language model one-pass decoding speech recognition

来源：评论

学校读者我要写书评

暂无评论

Structured Output Layer neural network language models for Speech Recognition

引用

IEEE TRANSACTIONS ON AUDIO SPEECH AND language PROCESSING 2013年第1期21卷 195-204页

作者： Le, Hai-Son Oparin, Ilya Allauzen, Alexandre Gauvain, Jean-Luc Yvon, Francois Univ Paris 11 F-91405 Orsay France LIMSI CNRS F-91405 Orsay France LIMSI CNRS F-91403 Orsay France

This paper extends a novel neural network language model (NNLM) which relies on word clustering to structure the output vocabulary: Structured OUtput Layer (SOUL) NNLM. This model is able to handle arbitrarily-sized vocabularies, hence dispensing with the need for shortlists that are commonly used in NNLMs. Several softmax layers replace the standard output layer in this model. The output structure depends on the word clustering which is based on the continuous word representation determined by the NNLM. Mandarin and Arabic data are used to evaluate the SOUL NNLM accuracy via speech-to-text experiments. Well tuned speech-to-text systems (with error rates around 10%) serve as the baselines. The SOUL model achieves consistent improvements over a classical shortlist NNLM both in terms of perplexity and recognition accuracy for these two languages that are quite different in terms of their internal structure and recognition vocabulary size. An enhanced training scheme is proposed that allows more data to be used at each training iteration of the neural network.

关键词： Automatic speech recognition neural network language model speech-to-text

来源：评论

学校读者我要写书评

暂无评论

NEW WORD DETECTION AND EMOTIONAL TENDENCY JUDGMENT BASED ON MIXED model 3

NEW WORD DETECTION AND EMOTIONAL TENDENCY JUDGMENT BASED ON ...

引用

3rd IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS)

作者： Sun, Xiao Sun, Chongyuan Ren, Fuji Hefei Univ Technol Sch Comp & Informat Anhui Prov Key Lab Affect Comp & Adv Intelligent Hefei 230009 Peoples R China Univ Tokushima Fac Engn Tokushima 7708506 Japan

ISBN: (纸本)9781479947195

The paper studies a new method for Chinese new word detection and emotional tendency judgment based on mixed model and proposes a new word generation framework. First we construct conditional random fields (CRFs) to recognize the new words, lead-in features based on character combined with the crowd sourcing network dictionary. And then express word as a word vector based on neural network language model (NNLM) to judge the new word emotional tendency. The experimental results show that the method can improve the precision and recall of the new word detection with a good system performance, and it also provides a new way for forecasting the public mood.

关键词： New Word Detection Conditional Random Fields Emotional Tendency Judgment neural network language model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：