检索结果-内蒙古大学图书馆

Syntax-aware multilingual semantic role labeling

学校读者我要写书评

暂无评论

arXiv 2019年

作者： He, Shexia Li, Zuchao Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence Ai Institute Shanghai Jiao Tong University

Recently, semantic role labeling (SRL) has earned a series of success with even higher performance improvements, which can be mainly attributed to syntactic integration and enhanced word representation. However, most of these efforts focus on English, while SRL on multiple languages more than English has received relatively little attention so that is kept underdevelopment. Thus this paper in-tends to fill the gap on multilingual SRL with special focus on the impact of syntax and con-textualized word representation. Unlike exist-ing work, we propose a novel method guided by syntactic rule to prune arguments, which enables us to integrate syntax into multilingual SRL model simply and effectively. We present a unified SRL model designed for multiple languages together with the proposed uniform syntax enhancement. Our model achieves new state-of-the-art results on the CoNLL-2009 benchmarks of all seven languages. Be-sides, we pose a discussion on the syntactic role among different languages and verify the effectiveness of deep enhanced representation for multilingual SRL. Copyright © 2019, The Authors. All rights reserved.

关键词： Syntactics

Global greedy dep.ndency parsing

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Li, Zuchao Zhao, Hai Parnow, Kevin Department of Computer Science and Engineering Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence Ai Institute Shanghai Jiao Tong University

Most syntactic dep.ndency parsing models may fall into one of two categories: transition- and graph-based models. The former models enjoy high inference efficiency with linear time complexity, but they rely on the stacking or re-ranking of partially-built parse trees to build a complete parse tree and are stuck with slower training for the necessity of dynamic oracle training. The latter, graph-based models, may boast better performance but are unfortunately marred by polynomial time inference. In this paper, we propose a novel parsing order objective, resulting in a novel dep.ndency parsing model capable of both global (in sentence scope) feature extraction as in graph models and linear time inference as in transitional models. The proposed global greedy parser only uses two arc-building actions, left and right arcs, for projective parsing. When equipped with two extra non-projective arc-building actions, the proposed parser may also smoothly support non-projective parsing. Using multiple benchmark treebanks, including the Penn Treebank (PTB), the CoNLL-X treebanks, and the Universal dep.ndency Treebanks, we evaluate our parser and demonstrate that the proposed novel parser achieves good performance with faster training and decoding. Copyright © 2019, The Authors. All rights reserved.

关键词： Polynomial approximation

Korean-to-Chinese Machine Translation using Chinese Character as Pivot Clue

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Park, Jeonghyeok Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

Korean-Chinese is a low resource language pair, but Korean and Chinese have a lot in common in terms of vocabulary. Sino-Korean words, which can be converted into corresponding Chinese characters, account for more then fifty of the entire Korean vocabulary. Motivated by this, we propose a simple linguistically motivated solution to improve the performance of Korean-to-Chinese neural machine translation model by using their common vocabulary. We adopt Chinese characters as a translation pivot by converting Sino-Korean words in Korean sentence to Chinese characters and then train machine translation model with the converted Korean sentences as source sentences. The experimental results on Korean-to-Chinese translation demonstrate that the models with the proposed method improve translation quality up to 1.5 BLEU points in comparison to the baseline models. Copyright © 2019, The Authors. All rights reserved.

关键词： Neural machine translation

Modeling named entity embedding distribution into hypersphere

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Zhuosheng Tang, Bingjie Li, Zuchao Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Computer Science Department Brown University RI United States

This work models named entity distribution from a way of visualizing topological structure of embedding space, so that we make an assumption that most, if not all, named entities (NEs) for a language tend to aggregate together to be accommodated by a specific hypersphere in embedding space. Thus we present a novel open definition for NE which alleviates the obvious drawback in previous closed NE definition with a limited NE dictionary. Then, we show two applications with introducing the proposed named entity hypersphere model. First, using a generative adversarial neural network to learn a transformation matrix of two embedding spaces, which results in a convenient determination of named entity distribution in the target language, indicating the potential of fast named entity discovery only using isomorphic relation between embedding spaces. Second, the named entity hypersphere model is directly integrated with various named entity recognition models over sentences to achieve state-of-theart results. Only assuming that embeddings are available, we show a prior knowledge free approach on effective named entity distribution dep.ction. Copyright © 2019, The Authors. All rights reserved.

关键词： Embeddings

LIMIT-BERT : Linguistic informed multi-task BERT

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhou, Junru Zhang, Zhuosheng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

In this paper, we present a Linguistic Informed Multi-Task BERT (LIMIT-BERT) for learning language representations across multiple linguistic tasks by Multi-Task Learning (MTL). LIMIT-BERT includes five key linguistic syntax and semantics tasks: Part-Of-Speech (POS) tags, constituent and dep.ndency syntactic parsing, span and dep.ndency semantic role labeling (SRL). Besides, LIMITBERT adopts linguistics mask strategy: Syntactic and Semantic Phrase Masking which mask all of the tokens corresponding to a syntactic/semantic phrase. Different from recent Multi-Task Deep Neural Networks (MTDNN) (Liu et al., 2019), our LIMIT-BERT is linguistically motivated and learning in a semi-supervised method which provides large amounts of linguistic-task data as same as BERT learning corpus. As a result, LIMITBERT not only improves linguistic tasks performance, but also benefits from a regularization effect and linguistic information that leads to more general representations to help adapt to new tasks and domains. LIMIT-BERT obtains new state-of-the-art or competitive results on both span and dep.ndency semantic parsing on Propbank benchmarks and both dep.ndency and constituent syntactic parsing on Penn Treebank. Copyright © 2019, The Authors. All rights reserved.

关键词： Syntactics

Span model for open information extraction on accurate corpus

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhan, Junlang Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

Open Information Extraction (Open IE) is a challenging task especially due to its brittle data basis. Most of Open IE systems have to be trained on automatically built corpus and evaluated on inaccurate test set. In this work, we first alleviate this difficulty from both sides of training and test sets. For the former, we propose an improved model design to more sufficiently exploit training dataset. For the latter, we present our accurately re-annotated benchmark test set (Re- OIE2016) according to a series of linguistic observation and analysis. Then, we introduce a span model instead of previous adopted sequence labeling formulization for n-ary Open IE. Our newly introduced model achieves new state-of-the-art performance on both benchmark evaluation datasets. Copyright © 2019, The Authors. All rights reserved.

关键词： Benchmarking

Attention Is All You Need for Chinese Word Segmentation

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Duan, Sufeng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

This paper presents a fast and accurate Chinese word segmentation (CWS) model with only unigram feature and greedy decoding algorithm. Our model uses only attention mechanism for network block building. In detail, we adopt a Transformer-based encoder empowered by self-attention mechanism as backbone to take input representation. Then we extend the Transformer encoder with our proposed Gaussian-masked directional multi-head attention, which is a variant of scaled dot-product attention. At last, a bi-affinal attention scorer is to make segmentation decision in a linear time. Our model is evaluated on SIGHAN Bakeoff benchmark dataset. The experimental results show that with the highest segmentation speed, the proposed attention-only model achieves new state-of-the-art or comparable performance against strong baselines in terms of closed test setting. Copyright © 2019, The Authors. All rights reserved.

关键词： Signal encoding

Named entity recognition only from word embeddings

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Luo, Ying Zhao, Hai Zhan, Junlang Department of Computer Science and Engineering Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence Ai Institute Shanghai Jiao Tong University Shanghai China

Deep neural network models have helped named entity (NE) recognition achieve amaz-ing performance without handcrafting features. However, existing systems require large amounts of human annotated training data. Ef-forts have been made to replace human annota-tions with external knowledge (e.g., NE dictio-nary, part-of-speech tags), while it is another challenge to obtain such effective resources. In this work, we propose a fully unsupervised NE recognition model which only needs to take informative clues from pre-trained word embeddings. We first apply Gaussian Hid-den Markov Model and Deep Autoencoding Gaussian Mixture Model on word embeddings for entity span detection and type prediction, and then further design an instance selec-tor based on reinforcement learning to distin-guish positive sentences from noisy sentences and refine these coarse-grained annotations through neural networks. Extensive experiments on CoNLL benchmark datasets demon-strate that our proposed light NE recognition model achieves remarkable performance with-out using any annotated lexicon or corpus. Copyright © 2019, The Authors. All rights reserved.

关键词： Embeddings

Controllable dual skew divergence loss for neural machine translation

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Li, Zuchao Zhao, Hai Wu, Yingting Xiao, Fengshun Jiang, Shu Department of Computer Science and Engineering Shanghai Jiao Tong University China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

In sequence prediction tasks like neural machine translation, training with cross-entropy loss often leads to models that overgeneralize and plunge into local optima. In this paper, we propose an extended loss function called dual skew divergence (DSD) that integrates two symmetric terms on KL divergences with a balanced weight. We empirically discovered that such a balanced weight plays a crucial role in applying the proposed DSD loss into deep models. Thus we eventually develop a controllable DSD loss for general-purpose scenarios. Our experiments indicate that switching to the DSD loss after the convergence of ML training helps models escape local optima and stimulates stable performance improvements. Our evaluations on the WMT 2014 English-German and English-French translation tasks demonstrate that the proposed loss as a general and convenient mean for NMT training indeed brings performance improvement in comparison to strong baselines. Copyright © 2019, The Authors. All rights reserved.

关键词： Neural machine translation