检索结果-内蒙古大学图书馆

36th Annual ACM Symposium on Applied Computing (SAC)

作者： Mallick, Ritam Susan, Seba Agrawal, Vaibhaw Garg, Rizul Rawal, Prateek Delhi Technol Univ Rohini Delhi 12 India

ISBN: (纸本)9781450381048

Neural Machine Translation(1) model is a sequence-to-sequence converter based on neural networks. Existing models use recurrent neural networks to construct both the encoder and decoder modules. In alternative research, the recurrent networks were substituted by convolutional neural networks for capturing the syntactic structure in the input sentence and decreasing the processing time. We incorporate the goodness of both approaches by proposing a convolutional-recurrent encoder for capturing the context information as well as the sequential information from the source sentence. Word embedding and position embedding of the source sentence is performed prior to the convolutional encoding layer which is basically a n-gram feature extractor capturing phrase-level context information. The rectified output of the convolutional encoding layer is added to the original embedding vector, and the sum is normalized by layer normalization. The normalized output is given as a sequential input to the recurrent encoding layer that captures the temporal information in the sequence. For the decoder, we use the attention-based recurrent neural network. Translation task on the German-English dataset verifies the efficacy of the proposed approach from the higher BLEU scores achieved as compared to the state of the art.

关键词： Neural machine translation convolutional recurrent encoder decoder context sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

Prediction of mine water quality by the Seq2Seq model based on attention mechanism

引用

Heliyon 2024年第18期10卷 e37916页

作者： Wang, Xiaolong Li, Yang CHN Shendong Coal Group Co. LTD. Shenmu 719300 China Summit Technologies Co. LTD Xian 710000 China

In recent years, as China's industrialization level has advanced, the issue of environmental pollution, particularly mine water pollution, has become increasingly severe. Water quality prediction is a fundamental aspect of water resource protection and a critical approach to addressing the water resource crisis. For improvement in water quality prediction, this research first analyzes the characteristics of mine water quality changes and provides a brief overview of water quality prediction. Subsequently, the Long Short-Term Memory and sequence to sequence (Seq2Seq) models, derived from Artificial Neural Networks, are introduced. The Seq2Seq water quality prediction model is implemented, incorporating the attention mechanism. Experimental validation confirms the effectiveness of the proposed model. The results demonstrate that the attention mechanism-based Seq2Seq model accurately predicts parameters such as pH value, Dissolved Oxygen, ammonia nitrogen, and Chemical Oxygen Demand, exhibiting a high degree of consistency with actual results. They play a vital role in assessing the health of the water and its ability to support aquatic life. The change of these indicators can reflect the degree and type of water pollution. Moreover, the Seq2Seq + attention model stands out with the lowest predicted Root Mean Square Error of 0.309. Notably, in comparison to the traditional Seq2Seq model, the incorporation of attention mechanisms in the Seq2Seq model results in a substantial 2.94 reduction in Mean Absolute Error. This research on the Seq2Seq water quality prediction model with attention mechanism provides valuable insights and references for future endeavors in water quality prediction. © 2024 The Authors

关键词： Artificial neural network Attention mechanism sequence-to-sequence model Water quality prediction

来源：评论

学校读者我要写书评

暂无评论

Transformer-Based model for Electrical Load Forecasting

引用

ENERGIES 2022年第14期15卷 4993页

作者： L'Heureux, Alexandra Grolinger, Katarina Capretz, Miriam A. M. Univ Western Ontario Dept Elect & Comp Engn London ON N6A 5B9 Canada

Amongst energy-related CO2 emissions, electricity is the largest single contributor, and with the proliferation of electric vehicles and other developments, energy use is expected to increase. Load forecasting is essential for combating these issues as it balances demand and production and contributes to energy management. Current state-of-the-art solutions such as recurrent neural networks (RNNs) and sequence-to-sequence algorithms (Seq2Seq) are highly accurate, but most studies examine them on a single data stream. On the other hand, in natural language processing (NLP), transformer architecture has become the dominant technique, outperforming RNN and Seq2Seq algorithms while also allowing parallelization. Consequently, this paper proposes a transformer-based architecture for load forecasting by modifying the NLP transformer workflow, adding N-space transformation, and designing a novel technique for handling contextual features. Moreover, in contrast to most load forecasting studies, we evaluate the proposed solution on different data streams under various forecasting horizons and input window lengths in order to ensure result reproducibility. Results show that the proposed approach successfully handles time series with contextual data and outperforms the state-of-the-art Seq2Seq models.

关键词： electrical load forecasting deep learning transformer architecture machine learning sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

MTrajRec: Map-Constrained Trajectory Recovery via Seq2Seq Multi-task Learning 21

MTrajRec: Map-Constrained Trajectory Recovery via Seq2Seq Mu...

引用

27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)

作者： Ren, Huimin Ruan, Sijie Li, Yanhua Bao, Jie Meng, Chuishi Li, Ruiyuan Zheng, Yu Worcester Polytech Inst Worcester MA 01609 USA Xidian Univ Xian Shaanxi Peoples R China Chongqing Univ Chongqing Peoples R China JD Intelligent Cities Res Beijing Peoples R China

ISBN: (纸本)9781450383325

With the increasing adoption of GPS modules, there are a wide range of urban applications based on trajectory data analysis, such as vehicle navigation, travel time estimation, and driver behavior analysis. The effectiveness of urban applications relies greatly on the high sampling rates of trajectories precisely matched to the map. However, a large number of trajectories are collected under a low sampling rate in real-world practice, due to certain communication loss and energy constraints. To enhance the trajectory data and support the urban applications more effectively, many trajectory recovery methods are proposed to infer the trajectories in free space. In addition, the recovered trajectory still needs to be mapped to the road network, before it can be used in the applications. However, the two-stage pipeline, which first infers high-sampling-rate trajectories and then performs the map matching, is inaccurate and inefficient. In this paper, we propose a Map-constrained Trajectory Recovery framework, MTrajRec, to recover the fine-grained points in trajectories and map match them on the road network in an end-to-end manner. MTrajRec implements a multi-task sequence-to-sequence learning architecture to predict road segment and moving ratio simultaneously. Constraint mask, attention mechanism, and attribute module are proposed to overcome the limits of coarse grid representation and improve the performance. Extensive experiments based on large-scale real-world trajectory data confirm the effectiveness and efficiency of our approach.

关键词： Trajectory recovery Road network Multi-task learning sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet 22

Triple M: A Practical Text-to-speech Synthesis System With M...

引用

Interspeech Conference

作者： Lin, Shilun Xie, Fenglong Meng, Li Li, Xinhui Lu, Li Tencent Beijing Peoples R China

ISBN: (纸本)9781713836902

In this work, a robust and efficient text-to-speech (TTS) synthesis system named Triple M is proposed for large-scale online application. The key components of Triple M are: 1) A sequence-to-sequence model adopts a novel multi-guidance attention to transfer complementary advantages from guiding attention mechanisms to the basic attention mechanism without in-domain performance loss and online service modification. Compared with single attention mechanism, multi-guidance attention not only brings better naturalness to long sentence synthesis, but also reduces the word error rate by 26.8%. 2) A new efficient multi-band multi-time vocoder framework, which reduces the computational complexity from 2.8 to 1.0 GFLOP and speeds up LPCNet by 2.75x on a single CPU.

关键词： Speech synthesis sequence-to-sequence model attention transfer learning vocoder LPCNet

来源：评论

学校读者我要写书评

暂无评论

Category Recognition in E-Commerce using sequence-to-sequence Hierarchical Classification 21

Category Recognition in E-Commerce using Sequence-to-Sequenc...

引用

14th ACM International Conference on Web Search and Data Mining (WSDM)

作者： Hasson, Idan Novgorodov, Slava Fuchs, Gilad Acriche, Yoni eBay Res San Jose CA 95125 USA

ISBN: (纸本)9781450382977

E-commerce platforms often use a predefined structured hierarchy of product categories. Apart from helping buyers sort between different product types, listing categorization is also critical for multiple downstream tasks, including the platform's main listing search. Traditionally, when creating a new listing, sellers need to assign the product they sell to a single category. However, the high diversity of product types in the platform, along with the hierarchy's low level of granularity result in tens of thousands of different possible categories that sellers need to pick from. This, in turn, creates a unique classification challenge, especially for sellers with a large number of listings. Moreover, the expected cost of making a category classification error is high, as it can impact the likelihood that their listing will get discovered by relevant buyers, and eventually sold. To help with the challenge of category recognition we present CATRECOMM - an interactive real-time system that is generalized to provide category recommendations in different e-commerce scenarios. We present results from using the system for two main sub-tasks - listing and search-query category recognition, and demonstrate an end-to-end scenario of the former one. The system uses a convolutional sequence-to-sequence approach, and to the best of our knowledge, is the first to use this approach for category recognition. We define a new metric for evaluating this model which captures the hierarchical characteristics of the data and supports displaying multiple classification results. Finally, our experimental results show the effectiveness and efficiency on real-world data.

关键词： sequence-to-sequence model hierarchical classification e-commerce machine learning

来源：评论

学校读者我要写书评

暂无评论

Conformer Parrotron: a Faster and Stronger End-to-end Speech Conversion and Recognition model for Atypical Speech 22

Conformer Parrotron: a Faster and Stronger End-to-end Speech...

引用

Interspeech Conference

作者： Chen, Zhehuai Ramabhadran, Bhuvana Biadsy, Fadi Zhang, Xia Chen, Youzheng Jiang, Liyang Chu, Fang Doshi, Rohan Moreno, Pedro J. Google LLC Mountain View CA 94043 USA

ISBN: (纸本)9781713836902

Parrotron is an end-to-end personalizable model that enables many-to-one voice conversion (VC) and automated speech recognition (ASR) simultaneously for atypical speech. In this work, we present the next-generation Parrotron model with improvements in overall accuracy, training and inference speeds. The proposed architecture builds on the recent Conformer encoder comprising of convolution and attention layer based blocks used in ASR. We introduce architectural modifications that subsamples encoder activations to achieve speed-ups in training and inference. In order to jointly improve ASR and voice conversion quality, we show that this requires a corresponding upsampling after the Conformer encoder blocks. We provide an in-depth analysis on how the proposed approach can maximize the efficiency of a speech-to-speech conversion model in the context of atypical speech. Experiments on both many-to-one and one-to-one dysarthric speech conversion tasks show that we can achieve up to 7X speedup and 35% relative reduction in WER over the previous best Transformer Parrotron.

关键词： voice conversion speech impairments sequence-to-sequence model speech recognition

来源：评论

学校读者我要写书评

暂无评论

SeqAD: An Unsupervised and Sequential Autoencoder Ensembles based Anomaly Detection Framework for KPI 29

SeqAD: An Unsupervised and Sequential Autoencoder Ensembles ...

引用

29th IEEE/ACM International Symposium on Quality of Service (IWQOS)

作者： Zhao, Na Han, Biao Cai, Yang Su, Jinshu Natl Univ Def Technol Coll Comp Changsha Peoples R China Changsha Normal Univ Coll Informat Sci & Technol Changsha Peoples R China

ISBN: (纸本)9781665414944

Key Performance Indicator (KPI), a kind of time-series data, its anomalies are the most intuitive characteristics when failures occurred in IT systems. KPI anomaly detection is increasingly critical to provide reliable and stable services for IT systems. Unsupervised learning is a promising method because of lacking labels and the unbalance in KPI samples. However, existing unsupervised KPI anomaly detection methods suffer from high false alarm rates. They handle KPI sequence as non-sequential data and ignore the time information, which is an essential KPI character. To this end, in this paper, we propose an unsupervised and sequential autoencoder ensembles based anomaly detection framework called SeqAD. SeqAD inherits the advantages both from the sequence-to-sequence model and autoencoder ensembles. SeqAD reduces the KPI over-fitting problem effectively by introducing autoencoder ensembles. In order to better capture the time information of KPI, we propose a random step connection based recurrent neural network (RSC-RNN) to train the KPI sequence, which can provide random connections to construct autoencoders with different structures and retain time information to the most extent. Extensive experiments are conducted on two public KPI data-sets from real-world deployed systems to evaluate the efficiency and robustness of our proposed SeqAD framework. Results show that SeqAD is able to smoothly capture most of the characteristics in all KPI data-sets, as well as to achieve a high F1 score between 0.93 and 0.98, which is better than the state-of-art unsupervised KPI anomaly detection methods.

关键词： KPI anomaly detection autoencoder ensembles sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

TRANSFORMER-BASED DIRECT SPEECH-TO-SPEECH TRANSLATION WITH TRANSCODER

TRANSFORMER-BASED DIRECT SPEECH-TO-SPEECH TRANSLATION WITH T...

引用

IEEE Spoken Language Technology Workshop (SLT)

作者： Kano, Takatomo Sakti, Sakriani Nakamura, Satoshi Nara Inst Sci & Technol Ikoma Japan RIKEN Ctr Adv Intelligence Project AIP Wako Saitama Japan

ISBN: (纸本)9781728170664

Traditional speech translation systems use a cascade manner that concatenates speech recognition (ASR), machine translation (MT), and text-to-speech (TTS) synthesis to translate speech from one language to another language in a step-by-step manner. Unfortunately, since those components are trained separately, MT often struggles to handle ASR errors, resulting in unnatural translation results. Recently, one work attempted to construct direct speech translation in a single model. The model used a multi-task scheme that learns to predict not only the target speech spectrograms directly but also the source and target phoneme transcription as auxiliary tasks. However, that work was only evaluated Spanish-English language pairs with similar syntax and word order. With syntactically distant language pairs, speech translation requires distant word order, and thus direct speech frame-to-frame alignments become difficult. Another direction was to construct a single deep-learning framework while keeping the step-by-step translation process. However, such studies focused only on speech-to-text translation. Furthermore, all of these works were based on a recurrent neural network (RNN) model. In this work, we propose a step-by-step scheme to a complete end-to-end speech-to-speech translation and propose a Transformer-based speech translation using Transcoder. We compare our proposed and multi-task model using syntactically similar and distant language pairs.

关键词： speech-to-speech translation Transcoder Transformer sequence-to-sequence model multitask learning

来源：评论

学校读者我要写书评

暂无评论

Improving Rumor Detection by Image Captioning and Multi-Cell Bi-RNN With Self-Attention in Social Networks

引用

INTERNATIONAL JOURNAL OF DATA WAREHOUSING AND MINING 2022年第1期18卷 1页

作者： Wang, Jenq-Haur Huang, Chin-Wei Norouzi, Mehdi Natl Taipei Univ Technol Taipei Taiwan Univ Cincinnati Cincinnati OH USA

User-generated contents in social media are not verified before being posted. They could bring many problems if they were misused. Among various types of rumors, the authors focus on the type in which there's mismatch between images and their surrounding texts. They can be detected by multimodal feature fusion in RNNs with attention mechanism, but the relations between images and texts are not well-addressed. In this paper, the authors propose to improve rumor detection by image captioning and RNNs with self-attention. Firstly, they utilize the idea of image captioning to translate images into the corresponding text descriptions. Secondly, these caption words are represented by word embedding models and aggregated with surrounding texts using early fusion. Finally, multi-cell bidirectional RNNs with self-attention are used to learn important features to identify rumors. From the experimental results, the best F-measure of 0.882 can be obtained, which shows the potential of our proposed approach to rumor detection. Further investigation is needed for data in larger scale.

关键词： Bi-Directional Gated Recurrent Unit Multimodal Feature Fusion Rumor Detection Image Captioning Self-Attention Mechanism sequence-to-sequence model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：