检索结果-内蒙古大学图书馆

SG-Net: Syntax guided transformer for language representation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhang, Zhuosheng Wu, Yuwei Zhou, Junru Duan, Sufeng Zhao, Hai Wang, Rui Department of Computer Science and Engineering Shanghai Jiao Tong University China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

Understanding human language is one of the key themes of artificial intelligence. For language representation, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy texts and getting rid of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanisms for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. The proposed SG-Net is applied to typical Transformer encoders. Extensive experiments on popular benchmark tasks, including machine reading comprehension, natural language inference, and neural machine translation show the effectiveness of the proposed SG-Net design. Copyright © 2020, The Authors. All rights reserved.

关键词： Syntactics

Transdisciplinary reflections on social robotics in academia and beyond 3

学校读者我要写书评

暂无评论

Transdisciplinary reflections on social robotics in academia...

3rd International Conference on Robophilosophy 2018 / Transor 2018: Envisioning Robots in Society - Politics, Power, and Public Space

作者： Hannibal, Glenda Lindner, Felix Vienna University of Technology Institute of Visual Computing and Human-Centered Technology Research Division of Human Computer Interaction Argentinierstraße 8 Vienna1040 Austria Department of Computer Science University of Freiburg Germany

Reference language based unsupervised neural machine translation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Zuchao Zhao, Hai Wang, Rui Utiyama, Masao Sumita, Eiichiro Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Kyoto Japan

Exploiting common language as an auxiliary for better translation has a long tradition in machine translation, which lets supervised learning based machine translation enjoy the enhancement delivered by the well-used pivot language, in case that the prerequisite of parallel corpus from source language to target language cannot be fully satisfied. The rising of unsupervised neural machine translation (UNMT) seems completely relieving the parallel corpus curse, though still subject to unsatisfactory performance so far due to vague clues available used for its core back-translation training. Further enriching the idea of pivot translation by freeing the use of parallel corpus other than its specified source and target, we propose a new reference language based UNMT framework, in which the reference language only shares parallel corpus with the source, indicating clear enough signal to help the reconstruction training of UNMT through a proposed reference agreement mechanism. Experimental results show that our methods improve the quality of UNMT over that of a strong baseline in terms of only one auxiliary language, demonstrating the usefulness of the proposed reference language based UNMT with a good start. Copyright © 2020, The Authors. All rights reserved.

关键词： Neural machine translation

Multi-choice Dialogue-Based Reading Comprehension with Knowledge and Key Turns

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Junlong Zhang, Zhuosheng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

Multi-choice machine reading comprehension (MRC) requires models to choose the correct answer from candidate options given a passage and a question. Our research focuses dialogue-based MRC, where the passages are multi-turn dialogues. It suffers from two challenges, the answer selection decision is made without support of latently helpful commonsense, and the multi-turn context may hide considerable irrelevant information. This work thus makes the first attempt to tackle those two challenges by extracting substantially important turns and utilizing external knowledge to enhance the representation of context. In this paper, the relevance of each turn to the question are calculated to choose key turns. Besides, terms related to the context and the question in a knowledge graph are extracted as external knowledge. The original context, question and external knowledge are encoded with the pre-trained language model, then the language representation and key turns are combined together with a will-designed mechanism to predict the answer. Experimental results on a DREAM dataset show that our proposed model achieves great improve- Copyright © 2020, The Authors. All rights reserved.

关键词： Knowledge graph

Learning universal representations from word to sentence

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Yian Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific level of linguistic unit, which cause great inconvenience when being confronted with handling multiple layers of linguistic objects in a unified way. Thus this work introduces and explores the universal representation learning, i.e., embeddings of different levels of linguistic unit in a uniform vector space through a task-independent evaluation. We present our approach of constructing analogy datasets in terms of words, phrases and sentences and experiment with multiple representation models to examine geometric properties of the learned vector space. Then we empirically verify that well pre-trained Transformer models incorporated with appropriate training settings may effectively yield universal representation. Especially, our implementation of fine-tuning ALBERT on NLI and PPDB datasets achieves the highest accuracy on analogy tasks in different language levels. Further experiments on the insurance FAQ task show effectiveness of universal representation models in real-world applications. Copyright © 2020, The Authors. All rights reserved.

关键词： Linguistics

Strategic priorities for transformative progress in advancing biology with proteomics and artificial intelligence

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sun, Yingying Jun, A. Liu, Zhiwei Sun, Rui Qian, Liujia Payne, Samuel H. Bittremieux, Wout Ralser, Markus Li, Chen Chen, Yi Dong, Zhen Perez-Riverol, Yasset Khan, Asif Sander, Chris Aebersold, Ruedi Vizcaíno, Juan Antonio Krieger, Jonathan R. Yao, Jianhua Wen, Han Zhang, Linfeng Zhu, Yunping Xuan, Yue Sun, Benjamin Boyang Qiao, Liang Hermjakob, Henning Tang, Haixu Gao, Huanhuan Deng, Yamin Zhong, Qing Chang, Cheng Bandeira, Nuno Li, Ming Weinan, E. Sun, Siqi Yang, Yuedong Omenn, Gilbert S. Zhang, Yue Xu, Ping Fu, Yan Liu, Xiaowen Overall, Christopher M. Wang, Yu Deutsch, Eric W. Chen, Luonan Cox, Jürgen Demichev, Vadim He, Fuchu Huang, Jiaxing Jin, Huilin Liu, Chao Li, Nan Luan, Zhongzhi Song, Jiangning Yu, Kaicheng Wan, Wanggen Wang, Tai Zhang, Kang Zhang, Le Bell, Peter A. Mann, Matthias Zhang, Bing Guo, Tiannan Affiliated Hangzhou First People’s Hospital State Key Laboratory of Medical Proteomics School of Medicine Westlake University Zhejiang Province Hangzhou China Westlake Center for Intelligent Proteomics Westlake Laboratory of Life Sciences and Biomedicine Zhejiang Province Hangzhou China Biology Department Brigham Young University ProvoUT84602 United States Department of Computer Science University of Antwerp Antwerp2020 Belgium Department of Biochemistry CharitéUniversitätsmedizin Berlin Berlin Germany Biomedicine Discovery Institute Department of Biochemistry and Molecular Biology Monash University MelbourneVICVIC 3800 Australia Wellcome Genome Campus Hinxton CambridgeCB10 1SD United Kingdom Harvard Medical School Ludwig Center at Harvard United States Harvard Medical School Broad Institute Ludwig Center at Harvard Dana-Farber Cancer Institute United States Department of Biology Institute of Molecular Systems Biology ETH Zürich Zürich Switzerland Bruker Ltd. MiltonONL9T 6P4 Canada AI for Life Sciences Lab Tencent Shenzhen518057 China State Key Laboratory of Medical Proteomics AI for Science Institute Beijing100080 China Beijing Institute of Lifeomics Beijing102206 China Thermo Fisher Scientific GmbH Hanna-Kunath Str. 11 Bremen28199 Germany Informatics and Predictive Sciences Research Bristol Myers Squibb United States Department of Chemistry Fudan University Songhu Road 2005 Shanghai200438 China Department of Computer Science Luddy School of Informatics Computing and Engineering Indiana University IN47408 United States ProCan® Children’s Medical Research Institute Faculty of Medicine and Health The University of Sydney WestmeadNSW Australia La Jolla CA United States Central China Institute of Artificial Intelligence University of Waterloo Canada AI for Science Institute Center for Machine Learning Research School of Mathematical Sciences Peking University China Research Institute of Intelligent Complex Systems Fudan U

Artificial intelligence (AI) is transforming scientific research, including proteomics. Advances in mass spectrometry (MS)-based proteomics data quality, diversity, and scale, combined with groundbreaking AI techniques, are unlocking new challenges and opportunities in biological discovery. Here, we highlight key areas where AI is driving innovation, from data analysis to new biological insights. These include developing an AI-friendly ecosystem for proteomics data generation, sharing, and analysis;improving peptide and protein identification and quantification;characterizing protein-protein interactions and protein complexes;advancing spatial and perturbation proteomics;integrating multi-omics data;and ultimately enabling AI-empowered virtual cells. © 2025, CC BY.

关键词： Biotic

Bipartite Flat-Graph Network for Nested Named Entity Recognition

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Luo, Ying Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER), which contains two subgraph modules: a flat NER module for outermost entities and a graph module for all the entities located in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional network (GCN) are adopted to jointly learn flat entities and their inner dependencies. Different from previous models, which only consider the unidirectional delivery of information from innermost layers to outer ones (or outside-to-inside), our model effectively captures the bidirectional interaction between them. We first use the entities recognized by the flat NER module to construct an entity graph, which is fed to the next graph module. The richer representation learned from graph module carries the dependencies of inner entities and can be exploited to improve outermost entity predictions. Experimental results on three standard nested NER datasets demonstrate that our BiFlaG outperforms previous state-of-the-art models. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph neural networks

Semantics-Aware Inferential Network for Natural Language Understanding

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhang, Shuailiang Zhao, Hai Zhou, Junru Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

For natural language understanding tasks, either machine reading comprehension or natural language inference, both semantics-aware and inference are favorable features of the concerned modeling for better understanding performance. Thus we propose a Semantics-Aware Inferential Network (SAIN) to meet such a motivation. Taking explicit contextualized semantics as a complementary input, the inferential module of SAIN enables a series of reasoning steps over semantic clues through an attention mechanism. By stringing these steps, the inferential network effectively learns to perform iterative reasoning which incorporates both explicit semantics and contextualized representations. In terms of well pre-trained language models as front-end encoder, our model achieves significant improvement on 11 tasks including machine reading comprehension and natural language inference. Copyright © 2020, The Authors. All rights reserved.

关键词： Semantics

Dialogue graph modeling for conversational machine reading

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Ouyang, Siru Zhang, Zhuosheng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

Conversational Machine Reading (CMR) aims at answering questions in complicated interactive scenarios. Machine needs to answer questions through interactions with users based on given rule document, user scenario and dialogue history, and even initiatively asks questions for clarification if necessary. Namely, the answer to the task needs a machine in the response of either Yes, No, Irrelevant or to raise a follow-up question for further clarification. To effectively capture multiple objects in such a challenging task, graph modeling is supposed to be adopted, though it is surprising that this does not happen until this work proposes a dialogue graph modeling framework by incorporating two complementary graph models, i.e., explicit discourse graph and implicit discourse graph, which respectively capture explicit and implicit interactions hidden in the rule documents. The proposed model is evaluated on the ShARC benchmark and achieves new state-of-the-art by first exceeding the milestone accuracy score of 80%. The source code of our paper is available at https://***/ozyyshr/DGM. Copyright © 2020, The Authors. All rights reserved.

关键词： Graph theory