检索结果-内蒙古大学图书馆

Capturing Global Structural information in Long Document Question Answering with Compressive Graph Selector Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Nie, Yuxiang Huang, Heyan Wei, Wei Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China Beijing Institute of Technology Southeast Academy of Information Technology China Huazhong University of Science and Technology China

Long document question answering is a challenging task due to its demands for complex reasoning over long text. Previous works usually take long documents as non-structured flat texts or only consider the local structure in long documents. However, these methods usually ignore the global structure of the long document, which is essential for long-range understanding. To tackle this problem, we propose Compressive Graph Selector Network (CGSN) to capture the global structure in a compressive and iterative manner. Specifically, the proposed model consists of three modules: local graph network, global graph network and evidence memory network. Firstly, the local graph network builds the graph structure of the chunked segment in token, sentence, paragraph and segment levels to capture the short-term dependency of the text. Secondly, the global graph network selectively receives the information of each level from the local graph, compresses them into the global graph nodes and applies graph attention into the global graph nodes to build the long-range reasoning over the entire text in an iterative way. Thirdly, the evidence memory network is designed to alleviate the redundancy problem in the evidence selection via saving the selected result in the previous steps. Extensive experiments show that the proposed model outperforms previous methods on two datasets.1 Copyright © 2022, The Authors. All rights reserved.

关键词： Iterative methods

Deep Hierarchical Attention Flow for Visual Commonsense Reasoning 9th

学校读者我要写书评

暂无评论

Deep Hierarchical Attention Flow for Visual Commonsense Reas...

9th CCF International Conference on Natural language processing and Chinese computing, NLPCC 2020

作者： Song, Yuansheng Jian, Ping School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Bejing China

ISBN: (纸本)9783030604493

Visual Commonsense Reasoning (VCR) requires a thoroughly understanding general information connecting language and vision, as well as the background world knowledge. In this paper, we introduce a novel yet powerful deep hierarchical attention flow framework, which takes full advantage of text information in the query and candidate responses to perform reasoning over the image. Moreover, inspired by the success of machine reading comprehension, we also model the correlation among candidate responses to obtain better response representations. Extensive quantitative and qualitative experiments are conducted to evaluate the proposed model. Empirical results on the benchmark VCR1.0 show that the proposed model outperforms existing strong baselines, which demonstrates the effectiveness of our method. © 2020, Springer Nature Switzerland AG.

关键词： Visual languages

Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yizhe Huang, Heyan Gao, Yang Li, Jiawei School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China Southeast Academy of Information Technology Beijing Institute of Technology Fujian Putian351100 China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

The knowledge-grounded dialogue task aims to generate responses that convey information from given knowledge documents. However, it is a challenge for the current sequence-based model to acquire knowledge from complex documents and integrate it to perform correct responses without the aid of an explicit semantic structure. To address these issues, we propose a novel graph structure, Grounded Graph (G2), that models the semantic structure of both dialogue and knowledge to facilitate knowledge selection and integration for knowledge-grounded dialogue generation. We also propose a Grounded Graph Aware Transformer (G2AT) model that fuses multi-forms knowledge (both sequential and graphic) to enhance knowledge-grounded response generation. Our experiments results show that our proposed model outperforms the previous state-of-the-art methods with more than 10% gains in response generation and nearly 20% improvement in factual consistency. Further, our model reveals good generalization ability and robustness. By incorporating semantic structures as prior knowledge in deep neural networks, our model provides an effective way to aid language generation. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

Prediction or comparison: Toward interpretable qualitative reasoning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ren, Mucheng Huang, Heyan Gao, Yang School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

Qualitative relationships illustrate how changing one property (e.g., moving velocity) affects another (e.g., kinetic energy) and constitutes a considerable portion of textual knowledge. Current approaches use either semantic parsers to transform natural language inputs into logical expressions or a "black-box" model to solve them in one step. The former has a limited application range, while the latter lacks interpretability. In this work, we categorize qualitative reasoning tasks into two types: prediction and comparison. In particular, we adopt neural network modules trained in an end-to-end manner to simulate the two reasoning processes. Experiments on two qualitative reasoning question answering datasets, QuaRTz and QuaRel, show our methods' effectiveness and generalization capability, and the intermediate outputs provided by the modules make the reasoning process interpretable. Copyright © 2021, The Authors. All rights reserved.

关键词： Kinetics

Incorporating Option and Out-of-domain Knowledge for Multi-choice Machine Reading Comprehension

学校读者我要写书评

暂无评论

Incorporating Option and Out-of-domain Knowledge for Multi-c...

IEEE International Conference on cloud computing and Intelligence Systems (CCIS)

作者： Yuan Xu Shumin Shi Heyan Huang School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

ISBN: (纸本)9781665441506

Multi-choice Machine Reading Comprehension (MRC) requires the model to select the correct answer from a set of answer candidates given the corresponding passage and question. Previous studies mainly focus on complex matching networks to model the relationship among options, passage and question. However, these models obtain little improvement over the powerful Pre-trained language Models (PLMs). In this paper, we propose a simple method to incorporate option knowledge from PLMs and introduce out-of-domain knowledge by multi-task learning skillfully. Our approach obtains state-of-the-art results on Chinese multi-choice MRC dataset ReCO and also effectively improves the performance on C3.

关键词： Training cloud computing Conferences Computational modeling Multitasking Task analysis Intelligent systems

Dynamic Prefix-Tuning for Generative Template-based Event Extraction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Xiao Huang, Heyan Shi, Ge Wang, Bo School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China Key Laboratory of Intelligent Information Processing and Information Security Ministry of Industry and Information Technology China Southeast Academy of Information Technology Beijing Institute of Technology China Faculty of Information Technology Beijing University of Technology China

We consider event extraction in a generative manner with template-based conditional generation. Although there is a rising trend of casting the task of event extraction as a sequence generation problem with prompts, these generation-based methods have two significant challenges, including using suboptimal prompts and static event type information. In this paper, we propose a generative template-based event extraction method with dynamic prefix (GTEE-DYNPREF) by integrating context information with type-specific prefixes to learn a context-specific prefix for each context. Experimental results show that our model achieves competitive results with the state-of-the-art classification-based model ONEIE on ACE 2005 and achieves the best performances on ERE. Additionally, our model is proven to be portable to new types of events effectively. Copyright © 2022, The Authors. All rights reserved.

关键词： Extraction

Unifying Cross-lingual Summarization and Machine Translation with Compression Rate

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bai, Yu Huang, Heyan Fan, Kai Gao, Yang Zhu, Yiming Zhan, Jiaao Chi, Zewen Chen, Boxing School of Computer Science Beijing Institute of Technology Beijing China Southeast Academy of Information Technology Fujian China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

Cross-Lingual Summarization (CLS) is a task that extracts important information from a source document and summarizes it into a summary in another language. It is a challenging task that requires a system to understand, summarize, and translate at the same time, making it highly related to Monolingual Summarization (MS) and Machine Translation (MT). In practice, the training resources for Machine Translation are far more than that for cross-lingual and monolingual summarization. Thus incorporating the Machine Translation corpus into CLS would be beneficial for its performance. However, the present work only leverages a simple multi-task framework to bring Machine Translation in, lacking deeper exploration. In this paper, we propose a novel task, Cross-lingual Summarization with Compression rate (CSC), to benefit Cross-Lingual Summarization by large-scale Machine Translation corpus. Through introducing compression rate, the information ratio between the source and the target text, we regard the MT task as a special CLS task with a compression rate of 100%. Hence they can be trained as a unified task, sharing knowledge more effectively. However, a huge gap exists between the MT task and the CLS task, where samples with compression rates between 30% and 90% are extremely rare. Hence, to bridge these two tasks smoothly, we propose an effective data augmentation method to produce document-summary pairs with different compression rates. The proposed method not only improves the performance of the CLS task, but also provides controllability to generate summaries in desired lengths. Experiments demonstrate that our method outperforms various strong baselines in three cross-lingual summarization datasets. We released our code and data at https://***/ybai-nlp/CLS_CR. Copyright © 2021, The Authors. All rights reserved.

关键词： Computational linguistics

Cross-lingual abstractive summarization with limited parallel resources

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bai, Yu Gao, Yang Huang, Heyan School of Computer Science and Technology Beijing Institute of Technology Beijing China. Southeast Academy of Information Technology Fujian China. Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

Parallel cross-lingual summarization data is scarce, requiring models to better use the limited available cross-lingual resources. Existing methods to do so often adopt sequenceto- sequence networks with multi-task frameworks. Such approaches apply multiple decoders, each of which is utilized for a specific task. However, these independent decoders share no parameters, hence fail to capture the relationships between the discrete phrases of summaries in different languages, breaking the connections in order to transfer the knowledge of the high-resource languages to lowresource languages. To bridge these connections, we propose a novel Multi-Task framework for Cross-Lingual Abstractive Summarization (MCLAS) in a low-resource setting. Employing one unified decoder to generate the sequential concatenation of monolingual and cross-lingual summaries, MCLAS makes the monolingual summarization task a prerequisite of the cross-lingual summarization (CLS) task. In this way, the shared decoder learns interactions involving alignments and summary patterns across languages, which encourages attaining knowledge transfer. Experiments on two CLS datasets demonstrate that our model significantly outperforms three baseline models in both low-resource and full-dataset scenarios. Moreover, in-depth analysis on the generated summaries and attention heads verifies that interactions are learned well using MCLAS, which benefits the CLS task under limited parallel resources. Copyright © 2021, The Authors. All rights reserved.

关键词： Knowledge management

Comprehensive study: How the context information of different granularity affects dialogue state tracking?

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Yang, Puhai Huang, Heyan Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China Southeast Academy of Information Technology Beijing Institute of Technology Fujian China

Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user's goal. In general, there are two strategies to track a dialogue state: predicting it from scratch and updating it from previous state. The scratch-based strategy obtains each slot value by inquiring all the dialogue history, and the previous-based strategy relies on the current turn dialogue to update the previous dialogue state. However, it is hard for the scratch-based strategy to correctly track short-dependency dialogue state because of noise;meanwhile, the previous-based strategy is not very useful for long-dependency dialogue state tracking. Obviously, it plays different roles for the context information of different granularity to track different kinds of dialogue states. Thus, in this paper, we will study and discuss how the context information of different granularity affects dialogue state tracking. First, we explore how greatly different granularities affect dialogue state tracking. Then, we further discuss how to combine multiple granularities for dialogue state tracking. Finally, we apply the findings about context granularity to few-shot learning scenario. Besides, we have publicly released all codes. Copyright © 2021, The Authors. All rights reserved.

关键词： Speech processing