检索结果-内蒙古大学图书馆

28th International Conference on Neural Information Processing

作者： Dong, Tao Shan, Shimin Liu, Yu Qian, Yue Ma, Anqi Dalian Univ Technol Sch Software Dalian 116620 Peoples R China

ISBN: (纸本)9783030923068;9783030923075

The use of large-scale pre-trained models for text summarization has attracted increasing attention in the computer science community. However, pre-training models with millions of parameters and long training time cause difficulty to deployment. Furthermore, pre-training models focus on understanding language but ignore reproduction of factual details when generating text. In this paper, we propose a method for text summarization that applies knowledge distillation to a pre-trained model called the teacher model. We build a novel sequence-to-sequence model as the student model to learn from the teacher model's knowledge for imitation. Specifically, we propose a variant of the pointer-generator network, which integrates multi-head attention mechanism, coverage mechanism and copy mechanism. We apply the variant to our student model to solve the word repetition and out-of-vocabulary words problem, so that improving the quality of generation. With experiments on Gigaword andWeibo datasets, our model achieves better performance and costs less time beyond the baseline models.

关键词： Natural language processing Abstractive summarization Knowledge distillation pointer-generator

来源：评论

学校读者我要写书评

暂无评论

From Plots to Endings: A Reinforced pointer generator for Story Ending Generation 7th

From Plots to Endings: A Reinforced Pointer Generator for St...

引用

7th CCF International Conference on Natural Language Processing and Chinese Computing, NLPCC

作者： Zhao, Yan Liu, Lu Liu, Chunhua Yang, Ruoyao Yu, Dong Beijing Language & Culture Univ Beijing Peoples R China Beijing Adv Innovat Language Resources BLCU Beijing Peoples R China

ISBN: (纸本)9783319994956;9783319994949

We introduce a new task named Story Ending Generation (SEG), which aims at generating a coherent story ending from a sequence of story plot. We propose a framework consisting of a generator and a Reward Manager for this task. The generator follows the pointer-generator network with coverage mechanism to deal with out-of-vocabulary (OOV) and repetitive words. Moreover, a mixed loss method is introduced to enable the generator to produce story endings of high semantic relevance with story plots. In the Reward Manager, the reward is computed to fine-tune the generator with policy-gradient reinforcement learning (PGRL). We conduct experiments on the recently-introduced ROCStories Corpus. We evaluate our model in both automatic evaluation and human evaluation. Experimental results show that our model exceeds the sequence-to-sequence baseline model by 15.75% and 13.57% in terms of CIDEr and consistency score respectively.

关键词： Story Ending Generation pointer-generator Policy gradient

来源：评论

学校读者我要写书评

暂无评论

Identification and summarisation of events from Twitter using clustering algorithms and deep neural network

NATURAL LANGUAGE PROCESSING

引用

NATURAL LANGUAGE PROCESSING 2024年

作者： Chakma, Kunal Jamatia, Anupam Rudrapal, Dwijen Natl Inst Technol Comp Sci & Engn Dept Agartala Tripura India

The proliferation of social networks has caused an increase in the amount of textual content generated by users. The voluminous nature of such content poses a challenge to users, necessitating the development of technological solutions for automatic summarisation. This paper presents a two-stage framework for generating abstractive summaries from a collection of Twitter texts. In the first stage of the framework, event detection is carried out through clustering, followed by event summarisation in the second stage. Our approach involves generating contextualised vector representations of tweets and applying various clustering techniques to the vectors. The quality of the resulting clusters is evaluated, and the best clusters are selected for the summarisation task based on this evaluation. In contrast to previous studies, we experimented with various clustering techniques as a preprocessing step to obtain better event representations. For the summarisation task, we utilised pre-trained models of three state-of-the-art deep neural network architectures and evaluated their performance on abstractive summarisation of the event clusters. Summaries are generated from clusters that contain (a) unranked tweets, (b) all ranked tweets, and (c) the top 10 ranked tweets. Of these three sets of clusters, we obtained the best ROUGE scores from the top 10 ranked tweets. From the summaries generated from the clusters containing the top ten tweets, we obtained ROUGE-1 F score of 48%, ROUGE-2 F score of 37%, ROUGE-L F score of 44%, and ROUGE-SU F score of 33% which suggests that if relevant tweets are at the top of a cluster, and then better summaries are generated.

关键词： Abstractive summarisation Twitter BERT transformer pointer-generator

来源：评论

学校读者我要写书评

暂无评论

Enhancements on a Pipeline Approach for Abstract Meaning Representation Parsing 23rd

Enhancements on a Pipeline Approach for Abstract Meaning Rep...

引用

23rd International Conference on Engineering Applications of Neural Networks (EAAAI/EANN)

作者： Frasie, Alexandru Vezan, Nicoleta-Teodora Marian, Georgiana Macicasan, Florin Lemnaru, Camelia Tech Univ Cluj Napoca Romania Dept Comp Sci Cluj Napoca Romania

ISBN: (纸本)9783031082238;9783031082221

'Abstract Meaning Representation (AMR) parsing attempts to extract a structured representation of a sentence's meaning. This paper enhances an existing processing pipeline for AMR parsing, inspired by state-of-the-art solutions in dependency parsing. It enhances the existing Concept Identification module by using pointer-generator Networks. A further considerable improvement of this module is brought by the use of embeddings. An alternative approach is provided through Transformers, an architecture which needs large data-sets to accurately predict concepts. For predicting the relations between concepts, the proposed pipeline combines the two Heads Selection and now trainable Arcs Labelling tasks into a joint Relation Identification module, which enhances the overall performance of edge prediction. The improvements made to this AMR parser have resulted in a completely trainable model that can be improved further with end-to-end training.

关键词： Natural Language Processing Abstract meaning representation Concept Identification LSTM pointer-generator Transformers Relation Identification

来源：评论

学校读者我要写书评

暂无评论

ConfNet2Seq Full Length Answer Generation from Spoken Questions 23rd

ConfNet2Seq Full Length Answer Generation from Spoken Questi...

引用

23rd Annual International Conference on Text, Speech, and Dialogue (TSD)

作者： Pal, Vaishali Shrivastava, Manish Besacier, Laurent Int Inst Informat Technol Hyderabad LTRC Hyderabad India Univ Grenoble Alpes LIG Grenoble France

ISBN: (纸本)9783030583231;9783030583224

Conversational and task-oriented dialogue systems aim to interact with the user using natural responses through multi-modal interfaces, such as text or speech. These desired responses are in the form of full-length natural answers generated over facts retrieved from a knowledge source. While the task of generating natural answers to questions from an answer span has been widely studied, there has been little research on natural sentence generation over spoken content. We propose a novel system to generate full length natural language answers from spoken questions and factoid answers. The spoken sequence is compactly represented as a confusion network extracted from a pre-trained Automatic Speech Recognizer. This is the first attempt towards generating full-length natural answers from a graph input (confusion network) to the best of our knowledge. We release a large-scale dataset of 259,788 samples of spoken questions, their factoid answers and corresponding full-length textual answers. Following our proposed approach, we achieve comparable performance with best ASR hypothesis.

关键词： Confusion network pointer-generator Copy attention Natural answer generation Question answering

来源：评论

学校读者我要写书评

暂无评论

Automated Smartification of Notices to Airmen 7

Automated Smartification of Notices to Airmen

引用

7th Swiss Conference on Data Science (SDS)

作者： Bravin, Marc Pfaffli, Daniel Mazumder, Sita Pouly, Marc Univ Lucerne Appl Sci & Arts Sch IT Rotkreuz Switzerland

ISBN: (纸本)9781728171777

Notices to airmen (NOTAMs) are text messages targeted to pilots to make them aware of short term events and obstacles. Typical examples are military actions in airspaces or closed runways. NOTAMs are written in a strongly compressed format and vocabulary, making the information hard to access for non-trained readers. Smart NOTAM is a commercial service offered by Skyguide that filters irrelevant messages for a specific route or period. Additionally, it converts the messages into a standardized phraseology and removes irrelevant information. Having access to thousands of NOTAMs together with their smartified version, a transformer model was trained and evaluated to automate the currently manual smartification process. NOTAMs contain a large number of factual details such as coordinates or radio frequencies, making the decoding process vulnerable to the inaccurate generation of such details. Therefore, the model was extended by a pointer-network that allows copying words directly from the source text. It achieves a BLEU score above 86% on unseen test data. Visual inspection of fails reveals many ground truth messages written in other languages than English or containing factual details added from other sources.

关键词： notice to airmen machine learning natural language processing sequence to sequence model transformer pointer-generator

来源：评论

学校读者我要写书评

暂无评论

Improving the Responses Diversity of Persona-based Neural Conversation System: A Conditional Variational Autoencoders-based Approach

Improving the Responses Diversity of Persona-based Neural Co...

引用

Chinese Automation Congress (CAC)

作者： Shi, Tianyuan Yu, Wei Yang, Huichun Song, Yongduan Chongqing Univ Sch Automat Chongqing Peoples R China

ISBN: (纸本)9781728176871

Given a conversational context based on persona information to a chatbot, how to generate diverse and relevant responses? Many existing works could effectively capture useful information from external knowledge, but they rarely consider the diversity of responses. In this paper, we consider both the persona information and the responses' diversity, upon which we develop a Seq2Seq-based model with hierarchical RNN context encoder, Conditional Variational Autoencoder (CVAE) and pointer generator. Hierarchical RNN allows those context utterances to be used more effectively. CVAE is used to capture the relationship between question and several appropriate answers. With pointer generator, an output token in a response could either be generated or copied from persona facts. The model is trained and tested on Persona-Chat dataset. Finally, automatic evaluation shows that compared with the baseline model, this developed model is able to better integrate the persona information and generate more diverse responses.

关键词： Seq2Seq CVAE Hierarchical RNN pointer-generator Diversity Persona-based

来源：评论

学校读者我要写书评

暂无评论

Rare words in text summarization

Natural Language Processing Journal

引用

Natural Language Processing Journal 2023年 3卷

作者： Danila Morozovskii Sheela Ramanna Department of Applied Computer Science University of Winnipeg Winnipeg Manitoba R3B 2E9 Canada

Automatic text summarization is a difficult task, which involves a good understanding of an input text to produce fluent, brief and vast summary. The usage of text summarization models can vary from legal document summarization to news summarization. The model should be able to understand where important information is located to produce a good summary. However, infrequently used or rare words might limit model’s understanding of an input text, as the model might ignore such words or put less attention on them, especially, when the trained model is tested on the dataset, where the frequency distribution of used words is different. Another issue is that the model accepts only a limited amount of tokens (words) of an input text, which might contain redundant information or not including important information as it is located further in the text. To address the problem of rare words, we have proposed a modification to the attention mechanism of the transformer model with pointer-generator layer, where attention mechanism receives frequency information for each word, which helps to boost rare words. Additionally, our proposed supervised learning model uses the hybrid approach incorporating both extractive and abstractive elements, to include more important information for the abstractive model in a news summarization task. We have designed experiments involving a combination of six different hybrid models with varying input text sizes (measured as tokens) to test our proposed model. Four well-known datasets specific to news articles were used in this work: CNN/DM, XSum, Gigaword and DUC 2004 Task 1. Our results were compared using the well-known ROUGE metric. Our best model achieved R-1 score of 38.22, R-2 score of 15.07 and R-L score of 35.79, outperforming three existing models by several ROUGE points.

关键词： Abstractive summarization Extractive summarization pointer-generator Transformer Rare words

来源：评论

学校读者我要写书评

暂无评论

Hie-Transformer: A Hierarchical Hybrid Transformer for Abstractive Article Summarization 26th

Hie-Transformer: A Hierarchical Hybrid Transformer for Abstr...

引用

26th International Conference on Neural Information Processing (ICONIP) of the Asia-Pacific-Neural-Network-Society (APNNS)

作者： Zhang, Xuewen Meng, Kui Liu, Gongshen Shanghai Jiao Tong Univ Shanghai 200240 Peoples R China

ISBN: (纸本)9783030367183;9783030367176

A Abstractive summarization methods based on neural network models can generate more human-written and higher qualities summaries than extractive methods. However, there are three main problems for these abstractive models: inability to deal with long article inputs, out-of-vocabulary (OOV) words and repetition words in generated summaries. To tackle these problems, we proposes a hierarchical hybrid Transformer model for abstractive article summarization in this work. First, the proposed model is based on a hierarchical Transformer with selective mechanism. The Transformer has outperformed traditional sequence-to-sequence models in many natural language processing (NLP) tasks and the hierarchical structure can handle the very long article inputs. Second, the pointer-generator mechanism is applied to combine generating novel words with copying words from article inputs, which can reduce the probability of the OOV words. Additionally, we use the coverage mechanism to reduce the repetitions in summaries. The proposed model is applied to CNN-Daily Mail summarization task. The evaluation results and analyses can demonstrate that our proposed model has a competitively performance compared with the baselines.

关键词： Abstractive summarization Hierarchical transformer Selective mechanism pointer-generator Coverage mechanism

来源：评论

学校读者我要写书评

暂无评论

LEARNING COMMENT GENERATION BY LEVERAGING USER-GENERATED DATA 44

LEARNING COMMENT GENERATION BY LEVERAGING USER-GENERATED DAT...

引用

44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Lin, Zhaojiang Winata, Genta Indra Fung, Pascale Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Ctr Artificial Intelligence Res CAiRE Clear Water Bay Hong Kong Peoples R China

ISBN: (纸本)9781479981311

Existing models on open-domain comment generation are difficult to train, and they produce repetitive and uninteresting responses. The problem is due to multiple and contradictory responses from a single article, and by the rigidity of retrieval methods. To solve this problem, we propose a combined approach to retrieval and generation methods. We propose an attentive scorer to retrieve informative and relevant comments by leveraging user-generated data. Then, we use such comments, together with the article, as input for a sequence-to-sequence model with copy mechanism. We show the robustness of our model and how it can alleviate the aforementioned issue by using a large scale comment generation dataset. The result shows that the proposed generative model significantly outperforms strong baseline such as Seq2Seq with attention and Information Retrieval models by around 27 and 30 BLEU-1 points respectively.

关键词： comment generation natural language generation copy attention pointer-generator user-generated data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：