检索结果-内蒙古大学图书馆

Incorporating Option and Out-of-domain Knowledge for Multi-choice Machine Reading Comprehension 7

学校读者我要写书评

暂无评论

Incorporating Option and Out-of-domain Knowledge for Multi-c...

7th IEEE International Conference on cloud computing and Intelligence Systems, CCIS 2021

作者： Xu, Yuan Shi, Shumin Huang, Heyan Beijing Institute of Technology School of Computer Science and Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China

ISBN: (纸本)9781665441490

Multi-choice Machine Reading Comprehension (MRC) requires the model to select the correct answer from a set of answer candidates given the corresponding passage and question. Previous studies mainly focus on complex matching networks to model the relationship among options, passage and question. However, these models obtain little improvement over the powerful Pre-trained language Models (PLMs). In this paper, we propose a simple method to incorporate option knowledge from PLMs and introduce out-of-domain knowledge by multi-task learning skillfully. Our approach obtains state-of-the-art results on Chinese multi-choice MRC dataset ReCO and also effectively improves the performance on C3. © 2021 IEEE.

关键词： Domain Knowledge

MindLLM: Lightweight Large language Model Pre-Training, Evaluation and Domain Application

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Yang, Yizhe Sun, Huashan Li, Jiawei Liu, Runheng Li, Yinghao Liu, Yuhang Gao, Yang Huang, Heyan School of Computer Science Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing China Beijing Institute of Technology Southeast Academy of Information Technology Fujian Putian China

关键词： Artificial intelligence

Boosting legal case retrieval by query content selection with large language models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Youchao Huang, Heyan Wu, Zhijing School of Computer Science and Technology Beijing Institute of Technology Southeast Academy of Information Technology Beijing Institute of Technology Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China

Legal case retrieval, which aims to retrieve relevant cases to a given query case, benefits judgment justice and attracts increasing attention. Unlike generic retrieval queries, legal case queries are typically long and the definition of relevance is closely related to legal-specific elements. Therefore, legal case queries may suffer from noise and sparsity of salient content, which hinders retrieval models from perceiving correct information in a query. While previous studies have paid attention to improving retrieval models and understanding relevance judgments, we focus on enhancing legal case retrieval by utilizing the salient content in legal case queries. We first annotate the salient content in queries manually and investigate how sparse and dense retrieval models attend to those content. Then we experiment with various query content selection methods utilizing large language models (LLMs) to extract or summarize salient content and incorporate it into the retrieval models. Experimental results show that reformulating long queries using LLMs improves the performance of both sparse and dense models in legal case retrieval. © 2023, CC BY-NC-SA.

关键词： Computational linguistics

Prompt-based Logical Semantics Enhancement for Implicit Discourse Relation Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Chenxu Jian, Ping Huang, Mu School of Computer Science and Technology Beijing Institute of Technology Beijing China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing Institute of Technology Beijing China

Implicit Discourse Relation Recognition (IDRR), which infers discourse relations without the help of explicit connectives, is still a crucial and challenging task for discourse parsing. Recent works tend to exploit the hierarchical structure information from the annotated senses, which demonstrate enhanced discourse relation representations can be obtained by integrating sense hierarchy. Nevertheless, the performance and robustness for IDRR are significantly constrained by the availability of annotated data. Fortunately, there is a wealth of unannotated utterances with explicit connectives, that can be utilized to acquire enriched discourse relation features. In light of such motivation, we propose a Prompt-based Logical Semantics Enhancement (PLSE) method for IDRR. Essentially, our method seamlessly injects knowledge relevant to discourse relation into pre-trained language models through prompt-based connective prediction. Furthermore, considering the prompt-based connective prediction exhibits local dependencies due to the deficiency of masked language model (MLM) in capturing global semantics, we design a novel self-supervised learning objective based on mutual information maximization to derive enhanced representations of logical semantics for IDRR. Experimental results on PDTB 2.0 and CoNLL16 datasets demonstrate that our method achieves outstanding and consistent performance against the current state-of-the-art models. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Unsupervised Style Control for Image Captioning

学校读者我要写书评

暂无评论

Unsupervised Style Control for Image Captioning

作者： Junyu Tian Zhikun Yang Shumin Shi School of Computer Science and Technology Beijing Institute of Technology Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications

We propose a novel unsupervised image captioning *** captioning involves two fields of deep learning,natural language processing and computer *** excessive pursuit of model evaluation results makes the caption style generated by the model too monotonous,which is difficult to meet people's demands for vivid and stylized image ***,we propose an image captioning model that combines text style transfer and image emotion recognition methods,with which the model can better understand images and generate controllable stylized *** proposed method can automatically judge the emotion contained in the image through the image emotion recognition module,better understand the image content,and control the description through the text style transfer method,thereby generating captions that meet people's *** our knowledge,this is the first work to use both image emotion recognition and text style control.

关键词：

Reducing Length Bias in Scoring Neural Machine Translation via a Causal Inference Method 20

学校读者我要写书评

暂无评论

Reducing Length Bias in Scoring Neural Machine Translation v...

20th China National Conference on Computational Linguistics, CCL 2021

作者： Shi, Xuewen Huang, Heyan Jian, Ping Tang, Yi-Kun School of Computer Science and Technology Beijing Institute of Technology Beijing100081 China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing100081 China

ISBN: (纸本)9783030841850

Neural machine translation (NMT) usually employs beam search to expand the searching space and obtain more translation candidates. However, the increase of the beam size often suffers from plenty of short translations, resulting in dramatical decrease in translation quality. In this paper, we handle the length bias problem through a perspective of causal inference. Specifically, we regard the model generated translation score S as a degraded true translation quality affected by some noise, and one of the confounders is the translation length. We apply a Half-Sibling Regression method to remove the length effect on S, and then we can obtain a debiased translation score without length information. The proposed method is model agnostic and unsupervised, which is adaptive to any NMT model and test dataset. We conduct the experiments on three translation tasks with different scales of datasets. Experimental results and further analyses show that our approaches gain comparable performance with the empirical baseline methods. © 2021, Springer Nature Switzerland AG.

关键词： Neural machine translation

Exploiting Knowledge Embedding to Improve the Description for Image Captioning 5th

学校读者我要写书评

暂无评论

Exploiting Knowledge Embedding to Improve the Description fo...

5th China Conference on Knowledge Graph, and Semantic computing, CCKS 2020

作者： Song, Dandan Peng, Cuimei Yang, Huan Liao, Lejian Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications Beijing Key Laboratory of Intelligent Information Technology School of Computer Science and Technology Beijing Institute of Technology Beijing China

ISBN: (纸本)9789811619632

Most existing methods for image captioning are based on the encoder-decoder framework which directly translates visual features into sentences, without exploiting commonsense knowledge available in the form of knowledge graph. Inspired by the success of information retrieval and question answering systems that leverage prior knowledge, we explore a knowledge embedding approach for image captioning. In this paper, we propose a Knowledge Embedding with Attention on Attention (KE-AoA) method for image captioning, which judges whether or how well the objects are related and augments semantic correlations and constraints between them. The KE-AoA method combines knowledge base method (TransE) and text method (Skip-gram), adding external knowledge graph information (triplets) into the language model to guide the learning of word vectors as the regularization term. Then it employs the AoA module to model the relations among different objects. As more inherent relations and commonsense knowledge are learned, the model can generate better image descriptions. The experiments on MSCOCO data sets achieve a significant improvement on the existing methods and validate the effectiveness of our prior knowledge-based approach. © 2021, Springer Nature Singapore Pte Ltd.

关键词： Knowledge graph

SciMRC: Multi-perspective Scientific Machine Reading Comprehension

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Xiao Zheng, Heqi Nie, Yuxiang Huang, Heyan Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China Southeast Academy of Information Technology Beijing Institute of Technology China

Scientific machine reading comprehension (SMRC) aims to understand scientific texts through interactions with humans by given questions. As far as we know, there is only one dataset focused on exploring full-text scientific machine reading comprehension. However, the dataset has ignored the fact that different readers may have different levels of understanding of the text, and only includes single-perspective question-answer pairs, leading to a lack of consideration of different perspectives. To tackle the above problem, we propose a novel multi-perspective SMRC dataset, called SciMRC, which includes perspectives from beginners, students and experts. Our proposed SciMRC is constructed from 741 scientific papers and 6,057 question-answer pairs. Each perspective of beginners, students and experts contains 3,306, 1,800 and 951 QA pairs, respectively. The extensive experiments on SciMRC by utilizing pre-trained models suggest the importance of considering perspectives of SMRC, and demonstrate its challenging nature for machine comprehension. © 2023, CC BY.

关键词：

Momentum Decoding: Open-ended Text Generation As Graph Exploration

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Lan, Tian Su, Yixuan Liu, Shuhang Huang, Heyan Mao, Xian-Ling School of Computer Science and Technology Beijing Institute of Technology China Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications China

Open-ended text generation with autoregressive language models (LMs) is one of the core tasks in natural language processing. However, maximization-based decoding methods (e.g., greedy/beam search) often lead to the degeneration problem, i.e., the generated text is unnatural and contains undesirable repetitions. Existing solutions to this problem either introduce randomness prone to incoherence or require a look-ahead mechanism that demands extra computational overhead. In this study, we formulate open-ended text generation from a new perspective, i.e., we view it as an exploration process within a directed graph. Thereby, we understand the phenomenon of degeneration as circular loops within the directed graph. Based on our formulation, we propose a novel decoding method—momentum decoding—which encourages the LM to greedily explore new nodes outside the current graph. Meanwhile, it also allows the LM to return to the existing nodes with a momentum downgraded by a pre-defined resistance function. We extensively test our approach on three benchmarks from different domains through automatic and human evaluations. The results show that momentum decoding performs comparably with the current state of the art while enjoying notably improved inference speed and computation FLOPs. Furthermore, we conduct a detailed analysis to reveal the merits and inner workings of our approach. © 2022, CC BY.

关键词： Momentum