检索结果-内蒙古大学图书馆

Incorporating word attention with convolutional neural networks for abstractive summarization

WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS 2020年第1期23卷 267-287页

作者： Yuan, Chengzhe Bao, Zhifeng Sanderson, Mark Tang, Yong South China Normal Univ Sch Comp Sci Guangzhou Peoples R China RMIT Univ Sch Sci Comp Sci & Informat Technol Melbourne Vic Australia

Neural sequence-to-sequence (seq2seq) models have been widely used in abstractive summarization tasks. One of the challenges of this task is redundant contents in the input document often confuses the models and leads to poor performance. An efficient way to solve this problem is to select salient information from the input document. In this paper, we propose an approach that incorporates word attention with multilayer convolutional neural networks (CNNs) to extend a standard seq2seq model for abstractive summarization. First, by concentrating on a subset of source words during encoding an input sentence, word attention is able to extract informative keywords in the input, which gives us the ability to interpret generated summaries. Second, these keywords are further distilled by multilayer CNNs to capture the coarse-grained contextual features of the input sentence. Thus, the combined word attention and multilayer CNNs modules provide a better-learned representation of the input document, which helps the model generate interpretable, coherent and informative summaries in an abstractive summarization task. We evaluate the effectiveness of our model on the English Gigaword, DUC2004 and Chinese summarization dataset LCSTS. Experimental results show the effectiveness of our approach.

关键词： Abstractive summarization Word attention Convolutional neural networks sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

LongT5-Mulla: LongT5 With Multi-Level Local Attention for a Longer sequence

引用

IEEE ACCESS 2023年 11卷 138433-138444页

作者： Zhou, Le Shanghai Jiao Tong Univ Dept Comp Sci & Engn Shanghai 200240 Peoples R China

Efficient Transformer models typically employ local and global attention methods, or utilize hierarchical or recurrent architectures, to process long text inputs in natural language processing tasks. However, these models face challenges in terms of sacrificing either efficiency, accuracy, or compatibility to develop their application in longer sequences. To maintain both the accuracy of global attention and the efficiency of local attention, while keeping a good compatibility to be easily applied to an existing pre-trained model, in this paper, we propose multi-level local attention (Mulla attention), which is a hierarchical local attention that acts on both the input sequence and multiple pooling sequences of different granularity simultaneously, thus performing long-range modeling while maintaining linear or log-linear complexity. We apply Mulla attention to LongT5 and implement our LongT5-Mulla sequence-to-sequence model, without introducing new parameters except for positional embeddings. Experiments show that our model can surpass all baseline models, including two original variants of LongT5, in the 8 similar to 16k-input long text summarization task on the Multi-News, arXiv and WCEP-10 datasets, with improvements of at least +0.22, +0.01, +0.52 percentage points (pp) averaged Rouge scores respectively, while at the meantime being able to effectively process longer sequences that have 16 similar to 48k tokens with at least 52.6% lower memory consumption than LongT5-tglobal, and +0.56 similar to 1.62 pp averaged Rouge scores higher than LongT5-local. These results demonstrate that our proposed LongT5-Mulla model can effectively process long sequences and extend the maximum input length for long text tasks from 16k to 48k while maintaining accuracy and efficiency.

关键词： Efficient transformer long-range modeling natural language processing sequence-to-sequence model text summarization

来源：评论

学校读者我要写书评

暂无评论

The Analysis of Music Emotion and Visualization Fusing Long Short-Term Memory Networks Under the Internet of Things

引用

IEEE ACCESS 2023年 11卷 141192-141204页

作者： Cao, Yujing Park, Jinwan Chung Ang Univ Grad Sch Adv Imaging Sci Multimedia & Film Dept Video Art Media Art Direct New Media Prod Seoul 06911 South Korea Chung Ang Univ Grad Sch Adv Imaging Sci Multimedia & Film Dept Image Sci & Arts Seoul 06911 South Korea

This paper aims to provide an automatic analysis of music emotion, enhancing users' ability to perceive and comprehend the emotional nuances conveyed in music intuitively. It first reviews the limitations of traditional music emotion analysis methods, and highlights the transformative potential of Internet of Things (IoT) technology in addressing these limitations. Subsequently, the paper employs the Long Short-Term Memory (LSTM) network to model time series data in music, integrating it with the sequence-to-sequence (STS) framework to construct an advanced music emotion analysis model. Finally, several machine learning algorithms are adopted to train and evaluate the system. The findings underscore the efficacy of the music emotion analysis model based on the LSTM network fused with STS, demonstrating notable success in the task of music emotion prediction. Specifically, the fusion model achieves an average absolute error of Arousal value of 0.921, a root mean square error (RMSE) of 0.534, and an R square of 0.498. The average absolute error in the Valence value is 0.902, the RMSE is 0.575, and the R square is 0.478. This paper holds significant implications for a deeper understanding of music emotion and provides guidance for music recommendation and creation.

关键词： Music emotion long short-term memory network sequence-to-sequence model music emotion prediction

来源：评论

学校读者我要写书评

暂无评论

Summary-aware attention for social media short text abstractive summarization

引用

NEUROCOMPUTING 2021年 425卷 290-299页

作者： Wang, Qianlong Ren, Jiangtao Sun Yat Sen Univ Sch Data & Comp Sci Guangdong Prov Key Lab Computat Sci Guangzhou Guangdong Peoples R China

Neural sequence-to-sequence (Seq2Seq) models are currently dominant approaches in social media short text abstractive summarization task. However, the computation of source attention weight in Seq2Seq models does not fully consider the information of the previously generated summary, which harms the coherence of the summary. In addition, given the noisy source text, it is difficult for Seq2Seq models to learn an accurate source context. To address these two shortcomings, we propose summary-aware attention for social media short text abstractive summarization. In this work, we use the source hidden states and attended summary vectors to compute summary-aware attention weight. The attended summary vectors contribute to generate a coherent summary. Moreover, under the guidance of the attended summary vectors, the attention mechanism increases the weight of the related content and reduces the weight of noise. Following the previous work, we evaluate our model on a popular Chinese social media dataset. Both automatic and human evaluation results demonstrate that our model achieves significant improvements compared with strong baselines.

关键词： Abstractive summarization Attention mechanism sequence-to-sequence model Social media short text

来源：评论

学校读者我要写书评

暂无评论

Improving Seq2Seq TTS Frontends With Transcribed Speech Audio

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2023年 31卷 1940-1952页

作者： Sun, Siqi Richmond, Korin Tang, Hao Univ Edinburgh UKRI Ctr Doctoral Training Nat Language Proc Edinburgh EH8 9LW Scotland Univ Edinburgh Ctr Speech Technol Res Edinburgh EH8 9LW Scotland Univ Edinburgh Sch Informat Edinburgh EH8 9LW Scotland

Due to the data inefficiency and low speech quality of grapheme-based end-to-end text-to-speech (TTS), having a separate high-performance TTS linguistic frontend is still commonly regarded as necessary. However, a TTS frontend is itself difficult to build and maintain, since it requires abundant linguistic knowledge for its construction. In this article, we start by bootstrapping an integrated sequence-to-sequence (Seq2Seq) TTS frontend using a pre-existing pipeline-based frontend and large amounts of unlabelled normalized text, achieving promising memorization and generalisation abilities. To overcome the performance limitation imposed by the pipeline-based frontend, this work proposes a Forced Alignment (FA) method to decode the pronunciations from transcribed speech audio and then use them to update the Seq2Seq frontend. Our experiments demonstrate the effectiveness of our proposed FA method, which can significantly improve the word token accuracy from 52.6% to 91.2% for out-of-dictionary words. In addition, it can also correct the pronunciation of homographs from transcribed speech audio and potentially improve the homograph disambiguation performance of the Seq2Seq frontend.

关键词： Training Pipelines Phonetics Dictionaries Acoustics Neural networks Training data Text-to-Speech synthesis sequence-to-sequence model linguistic frontend pronunciation learning grapheme-to-phoneme

来源：评论

学校读者我要写书评

暂无评论

Rhythm-Aware sequence-to-sequence Learning for Labanotation Generation With Gesture-Sensitive Graph Convolutional Encoding

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2022年 24卷 1488-1502页

作者： Li, Min Miao, Zhenjiang Zhang, Xiao-Ping Xu, Wanru Ma, Cong Xie, Ningwei Beijing Jiaotong Univ Inst Informat Sci Beijing 100044 Peoples R China Ryerson Univ Dept Elect Comp & Biomed Engn Toronto ON M5B 2K3 Canada

Labanotation is a professional dance notation system widely used in dance education and choreography preservation. Automatically generatingLabanotation dance scores from motion capture data can save a huge amount of manual time and effort. Recently, the sequence-to-sequence (seq2seq) model is applied to the automatic Labanotation generation. This model is based on an encoder-decoder structure, which encodes the input motion sequence to a fixed-length vector and then decodes it to generate the target sequence. However, the encoding of spatial skeleton structure of motion data is not considered in the existing work. Besides, it is challenging to align between the input motion data and the output Laban symbol sequences due to the severe imbalance of sequence lengths. Therefore, in this paper, we present a new seq2seq model for more effective Labanotation generation. In the encoder, we propose a new gesture-sensitive graph convolutional network with learned adaptive joint weights and non-physical connections to learn both spatial and temporal patterns from motion data sequences. In the decoder, we exploit motion rhythm information and propose a novel rhythm-aware attention mechanism to learn a good alignment between motion sequences and Laban symbol sequences, so that we can focus on relevant parts of the input motion sequence without searching in the whole sequence when predicting a target Laban symbol. Extensive experiments on two real-world datasets show that the proposed method achieves a better performance compared with the state-of-the-art approaches on the task of automatic Labanotation generation.

关键词： Motion segmentation Data models Skeleton Hidden Markov models Feature extraction Decoding Rhythm Attention mechanism graph convolutional network labanotation generation sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

Cross-domain machine transfer learning for capacity aging trajectory prediction of lithium-ion batteries

引用

JOURNAL OF POWER SOURCES 2024年 624卷

作者： Zhang, Caiping Du, Jingcai Li, Shuowei Zhang, Linjing Zhang, Weige Beijing Jiaotong Univ Natl Act Distribut Network Technol Res Ctr NANTEC Beijing 100044 Peoples R China

Machine transfer learning-based methods have emerged as a promising solution to capacity aging trajectory prediction for different types of batteries. However, the methods lack transferring interpretability, coupled with obvious variations in data across diverse batteries, presenting challenges to early lifespan prediction. A voltage- capacity (V-Q) curve reconstruction-based transfer learning method across source and target domains is proposed to predict the aging trajectories for various batteries. Initially, a generalized mathematical model is established for the V-Q curves of three types of batteries after a two-step transformation. Based on the model, a partial loss function for parameter optimization within deep learning is constructed. Subsequently, a Global Attention- sequence-to-sequence model is developed to reconstruct the V-Q curves in the target domain. The reconstructed data distribution in the target domain exhibits high similarity to that in the source domain, significantly reducing the detrimental impact of domain differences on transfer learning. Ultimately, a predictive model for aging trajectories, applicable to different types of batteries, is formulated with the transfer method. The model's accuracy is validated across three types of batteries. It is demonstrated that the mean absolute percentage error of aging trajectory for Lithium Cobalt Oxide battery and Ternary Lithium-ion battery are 1.09 % and 1.71 %, respectively.

关键词： Lithium-ion batteries EOL prediction Aging trajectory prediction Cross domains sequence-to-sequence model Transfer learning

来源：评论

学校读者我要写书评

暂无评论

TAGNet: a tiny answer-guided network for conversational question generation

引用

INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS 2023年第5期14卷 1921-1932页

作者： Wang, Zekun Zhu, Haichao Liu, Ming Qin, Bing Harbin Inst Technol Res Ctr Social Comp & Informat Retrieval Harbin Peoples R China Peng Cheng Lab Shenzhen Peoples R China

Conversational Question Generation (CQG) aims to generate conversational questions with the given passage and conversa-tion history. Previous work of CQG presumes a contiguous span as the answer and generates a question targeting it. However, this limits the application scenarios because answers in practical conversations are usually abstractive free-form text instead of extractive spans. In addition, most state-of-the-art CQG systems are based on pretrained language models consisting of hundreds of millions of parameters, bringing challenges to real-life applications due to latency and capacity constraints. To elegantly address these problems, in this work, we introduce the Tiny Answer-Guided Network (TAGNET) based on the lightweight module (Bi-LSTM) for CQG. We explicitly take the target answers as input, which interacts with the passages and conversation history in the encoder and guides the question generation through the gated attention mechanism in the decoder. Besides, we distill the knowledge from larger pretrained language models into our smaller network to make the trade-off between performance and efficiency. Experimental results show that our TAGNET achieves a comparable perfor-mance with large pretrained language models (retaining 95.9% of teacher performance) while using 5.7x fewer parameters and 10.4x faster inference latency. TAGNET outperforms the previous best-performing model with similar parameter size by a large margin, and further analysis shows that TAGNET generates more answer-specific conversational questions.

关键词： Conversational Question Generation sequence-to-sequence model Knowledge Distillation model Compression

来源：评论

学校读者我要写书评

暂无评论

Multistep speed prediction on traffic networks: A deep learning approach considering spatio-temporal dependencies

引用

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES 2019年第Aug.期105卷 297-322页

作者： Zhang, Zhengchao Li, Meng Lin, Xi Wang, Yinhai He, Fang Tsinghua Univ Dept Civil Engn Beijing 100084 Peoples R China Tsinghua Univ Tsinghua Daimler Joint Res Ctr Sustainable Transp Beijing 100084 Peoples R China Univ Washington Dept Civil & Environm Engn Seattle WA 98195 USA Tsinghua Univ Dept Ind Engn Beijing 100084 Peoples R China

Multistep traffic forecasting on road networks is a crucial task in successful intelligent transportation system applications. To capture the complex non-stationary temporal dynamics and spatial correlations in multistep traffic-condition prediction, we propose a novel deep learning framework named attention graph convolutional sequence-to-sequence model (AGC-Seq2Seq). In the proposed deep learning framework, spatial and temporal dependencies are modeled through the Seq2Seq model and graph convolution network separately, and the attention mechanism along with a newly designed training method based on the Seq2Seq architecture is proposed to overcome the difficulty in multistep prediction and further capture the temporal heterogeneity of traffic pattern. We conduct numerical tests to compare AGC-Seq2Seq with other benchmark models using two real-world datasets. The results indicate that our model yields the best prediction performance in terms of various prediction error measures. Furthermore, the variations of spatio-temporal correlations of traffic conditions under different perdition steps and road segments are revealed.

关键词： Traffic forecasting Deep learning Attention mechanism Graph convolution Multistep prediction sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

An Attentive LSTM based approach for adverse drug reactions prediction

引用

APPLIED INTELLIGENCE 2023年第5期53卷 4875-4889页

作者： Qian, Jiahui Qiu, Xihe Tan, Xiaoyu Li, Qiong Chen, Jue Jiang, Xiaoyan Shanghai Univ Engn Sci Sch Elect & Elect Engn 333 Longteng Rd Shanghai 201620 Peoples R China Ant Grp 566 Xixi Rd Hangzhou 310013 Zhejiang Peoples R China Beijing Jiaotong Univ Sch Comp Sci & Technol 3 ShangYuanCun Beijing 100044 Peoples R China

Adverse drug reactions (ADRs), which are harmful physical reactions of patients to drug treatments, are inherent to the nature of drugs;the reactions can occur with any drug and are becoming a leading cause of patient morbidity and mortality during medical procedures. ADRs can be hazardous and even fatal to patients. In traditional methods, ADRs are detected through clinical trials. To obtain a comprehensive collection of ADRs, sufficient experimental samples and time are required before a drug comes to the market, which is not a realistic possibility. Moreover, even if extensive clinical trials are performed, many undetected ADRs might still be discovered after a drug is released to the market. ADRs can lead to disastrous consequences for humanity, which obviates a dramatically increased need for precise predictions of potential ADRs as early as possible. In this paper, we propose an encoder-decoder framework based on attention mechanism and the long short-term memory (LSTM) model to predict potential ADRs. We regard the prediction of ADRs as a sequence-to-sequence problem and improve the encoder-decoder framework based on the attention mechanism to learn the interrelationships between ADRs. Unlike other classical methods utilizing molecular drug structures, our model is based solely on ADRs, which is an independent but parallel approach compared to traditional methods. We capitalize on the mask method to generate the target data and use the 5-fold cross-validation method to cyclically verify the performance of our proposed model. Based on the Top-k accuracy test results, our model outperforms the baseline models in potential ADRs predictions.

关键词： Adverse drug reactions Attention mechanism Long short-term memory sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：