检索结果-内蒙古大学图书馆

Recurrent neural network-based semantic variational autoencoder for sequence-to-sequence learning

INFORMATION SCIENCES 2019年 490卷 59-73页

作者： Jang, Myeongjun Seo, Seungwan Kang, Pilsung Sch Ind Management Engn 145 Anam Ro Seoul 136713 South Korea

sequence-to-sequence (Seq2seq) models have played an important role in the recent success of various natural language processing methods, such as machine translation, text summarization, and speech recognition. However, current Seq2seq models have trouble preserving global latent information from a long sequence of words. Variational autoencoder (VAE) alleviates this problem by learning a continuous semantic space of the input sentence. However, it does not solve the problem completely. In this paper, we propose a new recurrent neural network (RNN)-based Seq2seq model, RNN semantic variational autoencoder (RNN-SVAE), to better capture the global latent information of a sequence of words. To suitably reflect the meanings of words in a sentence regardless of their position within the sentence, we utilized two approaches: (1) constructing a document information vector based on the attention information between the final state of the encoder and every prior hidden state, and (2) extracting the semantic vector based on the self-attention mechanism. Then, the mean and standard deviation of the continuous semantic space are learned by using this vector to take advantage of the variational method. By using the document information vector and the self-attention mechanism to find the semantic space of the sentence, it becomes possible to better capture the global latent feature of the sentence. Experimental results of three natural language tasks (i.e., language modeling, missing word imputation, paraphrase identification) confirm that the proposed RNN-SVAE yields higher performance than two benchmark models. (C) 2019 Elsevier Inc. All rights reserved.

关键词： sequence-to-sequence learning Recurrent neural network Auto-encoder Variational method Document information vector Self attention mechanism Natural language processing

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence learning for Prediction of Soil Temperature and Moisture

引用

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS 2022年 19卷 1页

作者： Li, Xiaojie Tang, Jian Yin, Chengxiang Chinese Acad Sci Northeast Inst Geog & Agroecol Changchun 130102 Peoples R China Syracuse Univ Dept Elect Engn & Comp Sci Syracuse NY 13244 USA

Soil temperature and moisture play a significant influence on vegetation and climate. Measurements of soil temperature and moisture can now be obtained through soil temperature and moisture sensors, but the measurements can only be done for a specified period of time in the past. Predicting soil temperature and moisture for the future is of significant importance because it can provide guidance on making plant plans. With deep learning algorithms, the characteristics of soil temperature and moisture in a certain upcoming period can be predicted from data measured in the past. In this letter, we develop a sequence-to-sequence learning model for multistep ahead prediction, which features a long short-term memory (LSTM)-based encoder-decoder structure for modeling long-term temporal correlations as well as an autoencoder for modeling spatial correlations (i.e., considering data from neighboring locations). For performance evaluation, we use real data collected by the temperature and moisture sensors on soils at the Heihe River Basin in the west of China. Our experimental results show that: 1) the proposed model significantly outperforms two widely used time series analysis algorithms, autoregressive integrated moving average (ARIMA) and support vector regression (SVR), and 2) the proposed spatial encoder-decoder LSTM modeling method is indeed effective in that the root-mean-squared error (RMSE) and mean absolute error (MAE) are only 0.22 degrees C and 0.17 degrees C for 24-h (144 steps ahead) prediction of soil temperature and the values are only 0.28% and 0.23% for soil moisture.

关键词： Predictive models Moisture Temperature measurement Temperature sensors Time series analysis Data models Temperature distribution Beijing Normal University Net (BNUNET) encoder-decoder long short-term memory (LSTM) sequence-to-sequence learning soil temperature and moisture spatial encoder-decoder LSTM

来源：评论

学校读者我要写书评

暂无评论

DUALFORMER: A UNIFIED BIDIRECTIONAL sequence-to-sequence learning

DUALFORMER: A UNIFIED BIDIRECTIONAL SEQUENCE-TO-SEQUENCE LEA...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Chien, Jen-Tzung Chang, Wei-Hsiang Natl Chiao Tung Univ Dept Elect & Comp Engn Hsinchu Taiwan

ISBN: (纸本)9781728176055

This paper presents a new dual domain mapping based on a unified bidirectional sequence-to-sequence (seq2seq) learning. Traditionally, dual learning in domain mapping was constructed with intrinsic connection where the conditional generative models in two directions were mutually leveraged and combined. The additional feedback from the other generation direction was used to regularize sequential learning in original direction of domain mapping. Domain matching between source sequence and target sequence was accordingly improved. However, the reconstructions for knowledge in two domains were ignored. The dual information based on separate models in two training directions was not sufficiently discovered. To cope with this weakness, this study proposes a closed-loop seq2seq learning where domain mapping and domain knowledge are jointly learned. In particular, a new feature-level dual learning is incorporated to build a dualformer where feature integration and feature reconstruction are further performed to bridge dual tasks. Experiments demonstrate the merit of the proposed dualformer for machine translation based on the multi-objective seq2seq learning.

关键词： sequence-to-sequence learning dual learning transformer domain mapping machine translation

来源：评论

学校读者我要写书评

暂无评论

Generating Natural Answers on Knowledge Bases and Text by sequence-to-sequence learning 27th

Generating Natural Answers on Knowledge Bases and Text by Se...

引用

27th International Conference on Artificial Neural Networks (ICANN)

作者： Ye, Zhihao Cai, Ruichu Liao, Zhaohui Hao, Zhifeng Li, Jinfen Guangdong Univ Technol Sch Comp Sci & Technol Guangzhou Guangdong Peoples R China

ISBN: (纸本)9783030014186

Generative question answering systems aim at generating more contentful responses and more natural answers. Existing generative question answering systems applied to knowledge grounded conversation generate natural answers either with a knowledge base or with raw text. Nevertheless, performance of their methods is often affected by the incompleteness of the KB or text facts. In this paper, we propose an end-to-end generative question answering model. We make use of unstructured text and structured KBs to establish an universal schema as a large external facts library. Each words of a natural answer are dynamically predicted from the common vocabulary and retrieved from the corresponding external facts. And our model can generate natural answer containing arbitrary number of knowledge entities through selecting from multiple relevant external facts by the dynamic knowledge enquirer. Finally, empirical study shows that our model is efficient and outperforms baseline methods significantly in terms of automatic evaluation and human evaluation.

关键词： Natural answers Universal schema sequence-to-sequence learning

来源：评论

学校读者我要写书评

暂无评论

Voice Conversion Using sequence-to-sequence learning of Context Posterior Probabilities 18

Voice Conversion Using Sequence-to-Sequence Learning of Cont...

引用

18th Annual Conference of the International-Speech-Communication-Association (INTERSPEECH 2017)

作者： Miyoshi, Hiroyuki Saito, Yuki Takamichi, Shinnosuke Saruwatari, Hiroshi Univ Tokyo Grad Sch Informat Sci & Technol Bunkyo Ku 7-3-1 Hongo Tokyo 1138656 Japan

ISBN: (纸本)9781510848764

Voice conversion (VC) using sequence-to-sequence learning of context posterior probabilities is proposed. Conventional VC using shared context posterior probabilities predicts target speech parameters from the context posterior probabilities estimated from the source speech parameters. Although conventional VC can be built from non-parallel data, it is difficult to convert speaker individuality such as phonetic property and speaking rate contained in the posterior probabilities because the source posterior probabilities are directly used for predicting target speech parameters. In this work, we assume that the training data partly include parallel speech data and propose sequence-to-sequence learning between the source and target posterior probabilities. The conversion models perform non-linear and variable-length transformation from the source probability sequence to the target one. Further, we propose a joint training algorithm for the modules. In contrast to conventional VC, which separately trains the speech recognition that estimates posterior probabilities and the speech synthesis that predicts target speech parameters, our proposed method jointly trains these modules along with the proposed probability conversion modules. Experimental results demonstrate that our approach outperforms the conventional VC.

关键词： voice conversion context posterior probabilities sequence-to-sequence learning

来源：评论

学校读者我要写书评

暂无评论

DIALOG STATE TRACKING WITH ATTENTION-BASED sequence-to-sequence learning

DIALOG STATE TRACKING WITH ATTENTION-BASED SEQUENCE-TO-SEQUE...

引用

IEEE Workshop on Spoken Language Technology (SLT)

作者： Hori, Takaaki Wang, Hai Hori, Chiori Watanabe, Shinji Harsham, Bret Le Roux, Jonathan Hershey, John R. Koji, Yusuke Jing, Yi Zhu, Zhaocheng Aikawa, Takeyuki Mitsubishi Elect Res Labs Cambridge MA 02139 USA Toyota Technol Inst Chicago IL USA Mitsubishi Electr Corp Informat Technol R&D Ctr Tokyo Tokyo Japan Peking Univ Beijing Peoples R China

ISBN: (纸本)9781509049035

We present an advanced dialog state tracking system designed for the 5th Dialog State Tracking Challenge (DSTC5). The main task of DSTC5 is to track the dialog state in a human-human dialog. For each utterance, the tracker emits a frame of slot-value pairs considering the full history of the dialog up to the current turn. Our system includes an encoder-decoder architecture with an attention mechanism to map an input word sequence to a set of semantic labels, i.e., slot-value pairs. This handles the problem of the unknown alignment between the utterances and the labels. By combining the attention-based tracker with rule-based trackers elaborated for English and Chinese, the F-score for the development set improved from 0.475 to 0.507 compared to the rule-only trackers. Moreover, we achieved 0.517 F-score by refining the combination strategy based on the topic and slot level performance of each tracker. In this paper, we also validate the efficacy of each technique and report the test set results submitted to the challenge.

关键词： Dialog state tracking attention model sequence-to-sequence learning encoder-decoder long short-term memory

来源：评论

学校读者我要写书评

暂无评论

Multistep ahead atmospheric optical turbulence forecasting for free-space optical communication using empirical mode decomposition and LSTM-based sequence-to-sequence learning

引用

FRONTIERS IN PHYSICS 2023年 11卷

作者： Li, Yalin Zhang, Hongqun Li, Lang Shi, Lu Huang, Yan Fu, Shiyao Chinese Acad Sci Aerosp Informat Res Inst Beijing Peoples R China Beijing Inst Technol Sch Opt & Photon Beijing Peoples R China Minist Ind & Informat Technol Peoples Republ China Key Lab Informat Photon Technol Beijing Peoples R China Minist Educ Peoples Republ China Key Lab Photoelect Imaging Technol & Syst Beijing Peoples R China Northwestern Polytech Univ Sch Elect & Informat Xian Peoples R China

Although free-space optical communication (FSOC) is a promising means of high data rate satellite-to-ground communication, beam distortion caused by atmospheric optical turbulence remains a major challenge for its engineering applications. Accurate prediction of atmospheric optical turbulence to optimize communication plans and equipment parameters, such as adaptive optics (AO), is an effective means to address this problem. In this research, a hybrid multi-step prediction model for atmospheric optical turbulence, EMD-Seq2Seq-LSTM, is proposed by combining empirical mode decomposition (EMD), sequence-to-sequence (Seq2Seq), and long short-term memory (LSTM) network. First, using empirical mode decomposition to decompose the non-linear and non-stationary atmospheric optical turbulence dataset into a set of stationary components for which internal feature information can be easily extracted significantly reduces the training difficulty and improves the forecast accuracy of the model. Second, sequence-to-sequence is combined with LSTM networks to build a prediction model that can eliminate time delay and make full use of long-term information and then use the model to predict each component separately. Finally, the prediction results of each component are combined to obtain the final atmospheric turbulence forecasting results. To validate the performance of the proposed method, three comparative models, including WRF, LSTM, and sequence-to-sequence-LSTM, are demonstrated in this study. The forecasting results reveal that the proposed model outperforms all other models both qualitatively and quantitatively and thus can be a powerful method for atmospheric optical turbulence forecasting.

关键词： atmospheric optical turbulence forecasting free-space optical communication empirical mode decomposition LSTM sequence-to-sequence learning

来源：评论

学校读者我要写书评

暂无评论

learning Higher Representations from Bioacoustics: A sequence-to-sequence Deep learning Approach for Bird Sound Classification 27th

Learning Higher Representations from Bioacoustics: A Sequenc...

引用

27th International Conference on Neural Information Processing

作者： Qiao, Yu Qian, Kun Zhao, Ziping Tianjin Normal Univ Coll Comp & Informat Engn Tianjin Peoples R China Univ Tokyo Educ Physiol Lab Tokyo Japan

ISBN: (纸本)9783030638221;9783030638238

In the past two decades, a plethora of efforts have been given to the field of automatic classification of bird sounds, which can facilitate a long-term, non-human, and low-energy consumption ubiquitous computing system for monitoring the nature reserve. Nevertheless, human hand-crafted features need numerous domain knowledge, and inevitably make the designing progress time-consuming and expensive. To this line, we propose a sequence-to-sequence deep learning approach for extracting the higher representations automatically from bird sounds without any human expert knowledge. First, we transform the birds sound audio into spectrograms. Subsequently, higher representations were learnt by an autoencoder-based encoder-decoder paradigm combined with the deep recurrent neural networks. Finally, two typical machine learning models are selected to predict the classes, i.e., support vector machines and multi-layer perceptrons. Experimental results demonstrate the effectiveness of the method proposed, which can reach an unweighted average recall (UAR) at 66.8% in recognising 86 species of birds.

关键词： sequence-to-sequence learning Bird sound classification Bioacoustics Deep learning Internet of Things

来源：评论

学校读者我要写书评

暂无评论

Speech2Vec: A sequence-to-sequence Framework for learning Word Embeddings from Speech 19

Speech2Vec: A Sequence-to-Sequence Framework for Learning Wo...

引用

19th Annual Conference of the International-Speech-Communication-Association (INTERSPEECH 2018)

作者： Chung, Yu-An Glass, James MIT Comp Sci & Artificial Intelligence Lab 77 Massachusetts Ave Cambridge MA 02139 USA

ISBN: (纸本)9781510872219

In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar. The proposed model can be viewed as a speech version of Word2Vec [1]. Its design is based on a RNN Encoder-Decoder framework, and borrows the methodology of skipgrams or continuous bag-of-words for training. learning word embeddings directly from speech enables Speech2Vec to make use of the semantic information carried by speech that does not exist in plain text. The learned word embeddings are evaluated and analyzed on 13 widely used word similarity benchmarks, and outperform word embeddings learned by Word2Vec from the transcriptions.

关键词： word embeddings recurrent neural networks sequence-to-sequence learning skipgrams bag-of-words

来源：评论

学校读者我要写书评

暂无评论

Multivariate probabilistic prediction of dam displacement behaviour using extended Seq2Seq learning and adaptive kernel density estimation

引用

ADVANCED ENGINEERING INFORMATICS 2025年 65卷

作者： Li, Minghao Ren, Qiubing Li, Mingchao Qi, Zhiyong Tan, Dawen Wang, Hai Tianjin Univ State Key Lab Hydraul Engn Intelligent Construct & Tianjin 300350 Peoples R China China Yangtze Power Co Ltd CYPC Wuhan 430000 Peoples R China Minist Water Resources Sci & Technol Promot Ctr Beijing 100032 Peoples R China

Displacement prediction is an essential component of concrete dam health monitoring. However, considering the inherent uncertainties that exist within the structural responses to ever-fluctuating environmental factors like water levels and temperatures, which directly affect dam displacement behaviour, it is clear that mere point predictions are insufficient to accurately quantify the resultant uncertainty. To address this issue, we propose a multivariate probabilistic prediction framework that integrates the extended matrix long short-term memory-embedded attention sequence-to-sequence (Att-S2S-mLSTM) model with the density-based adaptive kernel density estimation (DAKDE). Specifically, the designed mLSTM incorporates a matrix memory and a covariance update rule to effectively simulate the complex nonlinear relationship between environmental factors and dam displacements. The Att-S2S architecture dynamically models sequential monitoring data of varying lengths, where the attention mechanism further augments its capacity to capture long-range dependencies, consequently generating displacement predictions with minimized errors. The DAKDE method then exploits the local density of prediction residuals obtained from Att-S2S-mLSTM to adaptively construct probability density functions. In this case, reliable prediction intervals are derived at various confidence levels, which are regarded as probabilistic predictions and serve as uncertainty quantification associated with displacement predictions. The large amount of monitoring data collected from a real-world concrete dam is employed to verify the effectiveness and superiority of the proposed framework. Moreover, a set of comparisons made among our proposed model and other advanced methods demonstrate that our methodology is more accurate and robust than other baselines in making both point and probabilistic predictions for dam displacement behaviour.

关键词： Dam displacement prediction Uncertainty quantification mLSTM sequence-to-sequence learning Kernel density estimation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：