检索结果-内蒙古大学图书馆

Hierarchical contextualized representation for named entity recognition

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Luo, Ying Xiao, Fengshun Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

Named entity recognition (NER) models are typically based on the architecture of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the modeling of single input prevent the full utilization of global information from larger scope, not only in the entire sentence, but also in the entire document (dataset). In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: Sentence-level representation and document-level representation. In sentencelevel, we take different contributions of words in a single sentence into consideration to enhance the sentence representation learned from an indep.ndent BiLSTM via label embedding attention mechanism. In document-level, the key-value memory network is adopted to record the document-aware information for each unique word which is sensitive to similarity of context information. Our two-level hierarchical contextualized representations are fused with each input token embedding and corresponding hidden state of BiLSTM, respectively. The experimental results on three benchmark NER datasets (CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset) show that we establish new state-of-the-art results. Copyright © 2019, The Authors. All rights reserved.

关键词： Embeddings

A Smart Sliding Chinese Pinyin Input Method Editor on Touchscreen

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Zhuosheng Meng, Zhen Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

This paper presents a smart sliding Chinese pinyin Input Method Editor (IME) for touchscreen devices which allows user finger sliding from one key to another on the touchscreen instead of tapping keys one by one, while the target Chinese character sequence will be predicted during the sliding process to help user input Chinese characters efficiently. Moreover, the layout of the virtual keyboard of our IME adapts to user sliding for more efficient inputting. The layout adaption process is utilized with Recurrent Neural Networks (RNN) and deep reinforcement learning. The pinyin-to-character converter is implemented with a sequence-to-sequence (Seq2Seq) model to predict the target Chinese sequence. A sliding simulator is built to automatically produce sliding samples for model training and virtual keyboard test. The key advantage of our proposed IME is that nearly all its built-in tactics can be optimized automatically with deep learning algorithms only following user behavior. Empirical studies verify the effectiveness of the proposed model and show a better user input efficiency. Copyright © 2019, The Authors. All rights reserved.

关键词： Behavioral research

Memorizing all for implicit discourse relation recognition

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Bai, Hongxiao Zhao, Hai Zhao, Junhan Department of Computer Science and Engineering Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence Ai Institute Shanghai Jiao Tong University Computer Graphics Technology Purdue University West LafayetteIN United States

Implicit discourse relation recognition is a challenging task due to the absence of the nec-essary informative clue from explicit connec-tives. The prediction of relations requires a deep understanding of the semantic meanings of sentence pairs. As implicit discourse relation recognizer has to carefully tackle the se-mantic similarity of the given sentence pairs and the severe data sparsity issue exists in the meantime, it is supposed to be beneficial from mastering the entire training data. Thus in this paper, we propose a novel memory mecha-nism to tackle the challenges for further performance improvement. The memory mecha-nism is adequately memorizing information by pairing representations and discourse relations of all training instances, which right fills the slot of the data-hungry issue in the current im-plicit discourse relation recognizer. Our experiments show that our full model with memo-rizing the entire training set reaches new state-of-the-art against strong baselines, which es-pecially for the first time exceeds the milestone of 60% accuracy in the 4-way task. Copyright © 2019, The Authors. All rights reserved.

关键词： Semantics

SCRDet: Towards More Robust Detection for Small, Cluttered and Rotated Objects

学校读者我要写书评

暂无评论

SCRDet: Towards More Robust Detection for Small, Cluttered a...

International Conference on computer Vision (ICCV)

作者： Xue Yang Jirui Yang Junchi Yan Yue Zhang Tengfei Zhang Zhi Guo Xian Sun Kun Fu NIST Institute of Electronics Beijing (Suzhou) China University of Chinese Academy of Sciences Beijing China Department of Computer Science and Engineering Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9781728148038

ISBN: (纸本)9781728148045

Object detection has been a building block in computer vision. Though considerable progress has been made, there still exist challenges for objects with small size, arbitrary direction, and dense distribution. Apart from natural images, such issues are especially pronounced for aerial images of great importance. This paper presents a novel multi-category rotation detector for small, cluttered and rotated objects, namely SCRDet. Specifically, a sampling fusion network is devised which fuses multi-layer feature with effective anchor sampling, to improve the sensitivity to small objects. Meanwhile, the supervised pixel attention network and the channel attention network are jointly explored for small and cluttered object detection by suppressing the noise and highlighting the objects feature. For more accurate rotation estimation, the IoU constant factor is added to the smooth L1 loss to address the boundary problem for the rotating bounding box. Extensive experiments on two remote sensing public datasets DOTA, NWPU VHR-10 as well as natural image datasets COCO, VOC2007 and scene text data ICDAR2015 show the state-of-the-art performance of our detector. The code and models will be available at https://***/DetectionTeamUCAS.

关键词： Detectors Feature extraction Object detection Remote sensing Training Convolution Semantics

Dual co-matching network for multi-choice reading comprehension

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Shuailiang Zhao, Hai Wu, Yuwei Zhang, Zhuosheng Zhou, Xi Zhou, Xiang Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China CloudWalk Technology Shanghai China

Multi-choice reading comprehension is a challenging task to select an answer from a set of candidates when given passage and question. This work proposes dual co-matching network which models the relationship among passage, question and answer bidirectionally. The experimental results on RACE, ROCStories and COIN Shared Task 1 show that our model obtains state-of-the-art results and even the single model outperforms the human performance on RACE dataset. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

DCMN+: Dual Co-matching network for multi-choice reading comprehension

学校读者我要写书评

暂无评论

arXiv 2019年

Multi-choice reading comprehension is a challenging task to select an answer from a set of candidate options when given passage and question. Previous approaches usually only calculate question-aware passage representation and ignore passage-aware question representation when modeling the relationship between passage and question, which obviously cannot take the best of information between passage and question. In this work, we propose dual co-matching network (DCMN) which models the relationship among passage, question and answer options bidirectionally. Besides, inspired by how human solve multi-choice questions, we integrate two reading strategies into our model: (i) passage sentence selection that finds the most salient supporting sentences to answer the question, (ii) answer option interaction that encodes the comparison information between answer options. DCMN integrated with the two strategies (DCMN+) obtains state-of-the-art results on five multi-choice reading comprehension datasets which are from different domains: RACE, SemEval-2018 Task 11, ROCStories, COIN, MCTest. Copyright © 2019, The Authors. All rights reserved.

关键词：

SG-Net: Syntax-guided machine reading comprehension

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Zhuosheng Wu, Yuwei Zhou, Junru Duan, Sufeng Zhao, Hai Wang, Rui Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China College of Zhiyuan Shanghai Jiao Tong University China Kyoto Japan

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling of both passages and questions by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. To serve such a purpose, we propose a novel dual contextual architecture called syntax-guided network (SG-Net), which consists of a BERT context vector and a syntax-guided context vector, to provide more fine-grained representation. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed approach achieves a substantial and significant improvement over the fine-tuned BERT baseline. Copyright © 2019, The Authors. All rights reserved.

关键词： Syntactics

Semantics-aware BERT for language understanding

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhang, Zhuosheng Wu, Yuwei Zhao, Hai Li, Zuchao Zhang, Shuailiang Zhou, Xi Zhou, Xiang Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China College of Zhiyuan Shanghai Jiao Tong University China CloudWalk Technology Shanghai China

The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks. However, the existing language representation models including ELMo, GPT and BERT only exploit plain context-sensitive features such as character or word embeddings. They rarely consider incorporating structured semantic information which can provide rich semantics for language representation. To promote natural language understanding, we propose to incorporate explicit contextual semantics from pre-trained semantic role labeling, and introduce an improved language representation model, Semantics-aware BERT (SemBERT), which is capable of explicitly absorbing contextual semantics over a BERT backbone. SemBERT keeps the convenient usability of its BERT precursor in a light fine-tuning way without substantial task-specific modifications. Compared with BERT, semantics-aware BERT is as simple in concept but more powerful. It obtains new state-of-the-art or substantially improves results on ten reading comprehension and language inference tasks. Copyright © 2019, The Authors. All rights reserved.

关键词： Semantics

Bridging explicit and implicit deep generative models via neural Stein estimators

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wu, Qitian Gao, Rui Zha, Hongyuan Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University University of Texas Austin United States School of Data Science Shenzhen Institute of Artificial Intelligence and Robotics for Society The Chinese University of Hong Kong Shenzhen Hong Kong The Chinese University of Hong Kong Shenzhen China

There are two types of deep generative models: explicit and implicit. The former defines an explicit density form that allows likelihood inference;while the latter targets a flexible transformation from random noise to generated samples. While the two classes of generative models have shown great power in many applications, both of them, when used alone, suffer from respective limitations and drawbacks. To take full advantages of both models and enable mutual compensation, we propose a novel joint training framework that bridges an explicit (unnormalized) density estimator and an implicit sample generator via Stein discrepancy. We show that our method 1) induces novel mutual regularization via kernel Sobolev norm penalization and Moreau-Yosida regularization, and 2) stabilizes the training dynamics. Empirically, we demonstrate that proposed method can facilitate the density estimator to more accurately identify data modes and guide the generator to output higher-quality samples, comparing with training a single counterpart. The new approach also shows promising results when the training samples are contaminated or limited. © 2019, CC BY.

关键词：