检索结果-内蒙古大学图书馆

sequence-to-sequence Traffic Missing Data Imputation via Self-Supervised Contrastive Learning

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2025年

作者： Zhou, Wenfeng Shen, Guojiang Zhang, Yimei Deng, Zhaolin Kong, Xiangjie Xia, Feng Zhejiang Univ Technol Coll Comp Sci & Technol Hangzhou 310023 Peoples R China RMIT Univ Sch Comp Technol Melbourne Vic 3000 Australia

Missing traffic data is a common phenomenon in practice, which has serious implications for the development of downstream data-driven traffic research. Although numerous methods have been proposed to impute missing values, two limitations still exist: lack of complete ground truth and reliance on carefully constructed model structures. In this paper, we propose a novel deep neural imputation network framework named STIseq2seq, based on self-supervised contrastive learning. It follows a sequence-to-sequence architecture with attention mechanism to address the traffic data imputation problem. Specifically, the model consists of two key components: encoder and decoder, both composed of ST-block. Initially, we utilize dynamic graph convolution (DGC) and gated recurrent unit (GRU) to construct the fundamental unit, ST-block, for learning spatiotemporal dependencies. Then, a bidirectional ST-block is employed to model incomplete input data in the encoder;while a forward ST-block combined with an attention mechanism is used to accurately impute missing data in the decoder. Lastly, we devise a training method based on self-supervised contrastive learning, achieved through randomly shuffling the masking matrix, to enhance training effectiveness. Extensive experiments conducted on two available real-world traffic datasets demonstrate that our proposed STIseq2seq has superior and stable performance compared to multiple baseline models under different missing data scenarios with missing rates ranging from 20% to 80%. Visualization analysis of the imputation results further provides insights into the advantages of STIseq2seq.

关键词： Data models Imputation Spatiotemporal phenomena Contrastive learning Training Deep learning Decoding Attention mechanisms Maintenance engineering Generative adversarial networks Missing traffic data data imputation sequence-to-sequence spatiotemporal dependency self-supervised contrastive learning

来源：评论

学校读者我要写书评

暂无评论

A deep sequence-to-sequence model for power swing blocking of distance protection in power transmission lines

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 139卷

作者： Birgani, Amin Mehdipour Shams, Mohammadreza Jannati, Mohsen Aloghareh, Farhad Hatami Univ Isfahan Dept Elect Engn Shahreza Campus Esfahan Iran Univ Isfahan Dept Comp Engn Shahreza Campus Esfahan Iran

As the primary protection method for transmission lines, distance relays are prone to malfunction during power swings. In fact, the inability of distance relays to differentiate between power swings and short-circuit faults imposes a significant risk to power system stability that can result in blackouts. In recent years, there has been increasing interest in leveraging machine learning techniques to identify various types of faults and power swings in electrical systems. However, previous works mainly focus on fault classification, which is mostly done after along period from the moment of fault initiation. This is the reason for requiring extensive post-fault data for diagnosis. To address this challenge, this study proposes a predictive protection strategy utilizing deep learning methodologies, specifically a sequence-to-sequence model, to monitor electrical power systems continuously. The objective is to effectively detect power swings from short-circuit faults with minimal reliance on post-fault data and accurately identify short-circuit faults during power swings. In the proposed approach, features are extracted from grid current signals using the Hilbert transform and empirical mode decomposition algorithms. These features are then fed into the sequence-to-sequence model, which issues block/unblock commands upon confirming the presence of a power swing or fault during the power swing. Results from various simulations conducted on an IEEE 39-bus grid in DIgSILENT and MATLAB environments demonstrate that the proposed scheme outperforms baseline methods in the detection of short-circuit faults, power swings, and short-circuit faults occurring during the power swings. The timely and correct operation of the proposed protection scheme contributes to the stability of transmission lines and power systems.

关键词： Distance relay Power swing Hilbert transform Empirical mode decomposition sequence-to-sequence Long short-term memory

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Acoustic Modeling for Voice Conversion

引用

IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING 2019年第3期27卷 631-644页

作者： Zhang, Jing-Xuan Ling, Zhen-Hua Liu, Li-Juan Jiang, Yuan Dai, Li-Rong Univ Sci & Technol China Natl Engn Lab Speech & Language Informat Proc Hefei 230027 Anhui Peoples R China iFLYTEK Co Ltd Hefei 230088 Anhui Peoples R China

In this paper, a neural network named sequence-to-sequence ConvErsion NeTwork (SCENT) is presented for acoustic modeling in voice conversion. At training stage, a SCENT model is estimated by aligning the feature sequences of source and target speakers implicitly using attention mechanism. At the conversion stage, acoustic features and durations of source utterances are converted simultaneously using the unified acoustic model. Mel-scale spectrograms are adopted as acoustic features, which contain both excitation and vocal tract descriptions of speech signals. The bottleneck features extracted from source speech using an automatic speech recognition model are appended as an auxiliary input. A WaveNet vocoder conditioned on Mel-spectrograms is built to reconstruct waveforms from the outputs of the SCENT model. It is worth noting that our proposed method can achieve appropriate duration conversion, which is difficult in conventional methods. Experimental results show that our proposed method obtained better objective and subjective performance than the baseline methods using Gaussian mixture models and deep neural networks as acoustic models. This proposed method also outperformed our previous work, which achieved the top rank in Voice Conversion Challenge 2018. Ablation tests further confirmed the effectiveness of several components in our proposed method.

关键词： Voice conversion sequence-to-sequence attention Mel-spectrogram

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Stacked Gate Recurrent Unit Networks for Approximating the Forward Problem of Partial Differential Equations

引用

IEEE ACCESS 2024年 12卷 61795-61809页

作者： Zhang, Zhaoyang Wang, Qingwang Kunming Univ Sci & Technol Fac Informat Engn & Automat Kunming 650500 Peoples R China

We proposed an optimisation algorithm based on the sequence-to-sequence (Seq2Seq) stacking of the gate recurrent unit (GRU) model to characterise and approximate the forward problem of partial differential equations (PDEs). Unlike traditional methods based on mesh differential approximation and no parameters, this is a meshless approach based on parametric semi-supervised learning. Specifically, the algorithm employs the ability of deep feedback neural networks to approximate continuous dynamical systems, which is enhanced by stacked GRU modules to capture the evolution of the PDEs over time and thus enrich the representations of the sequences in the hidden space. The loss function of the model incorporates partial physical knowledge as an a priori condition to guide the optimisation direction, i.e., transforming the numerical iterative problem into a non-convex optimisation problem. In addition, each round of training of the model incorporates data resampling to prevent it from overfitting. We evaluated the ability of the proposed algorithm to solve mathematical physics equations for a variety of differential operators and constraints, including the heat, wave, Burgers, Schrodinger, diffusion, and Kovasnay flow equations. The experimental results confirmed the outstanding prediction precision and generalisation capability of the proposed algorithm.

关键词： Forward problem of partial differential equations deep learning gate recurrent unit sequence-to-sequence

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence models for workload interference prediction on batch processing datacenters

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2020年 110卷 155-166页

作者： Buchaca, David Marcual, Joan LLuis Berral, Josep Carrera, David BSC C Jordi Girona 1-3 Barcelona 08034 Spain UPC BarcelonaTECH Barcelona Spain

Co-scheduling of jobs in data centers is a challenging scenario where jobs can compete for resources, leading to severe slowdowns or failed executions. Efficient job placement on environments where resources are shared requires awareness on how jobs interfere during execution, to go far beyond ineffective resource overbooking techniques. Current techniques, most of which already involve machine learning and job modeling, are based on workload behavior summarization over time, rather than focusing on effective job requirements at each instant of the execution. In this work, we propose a methodology for modeling co-scheduling of jobs on data centers, based on their behavior towards resources and execution time and using sequence-to-sequence models based on recurrent neural networks. The goal is to forecast co-executed jobs footprint on resources throughout their execution time, from the profile shown by the individual jobs, in order to enhance resource manager and scheduler placement decisions. The methods presented herein are validated by using High Performance Computing benchmarks based on different frameworks (such as Hadoop and Spark) and applications (CPU bound, IO bound, machine learning, SQL queries...). Experiments show that the model can correctly identify the resource usage trends from previously seen and even unseen co-scheduled jobs. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Resource management sequence-to-sequence Workload interference Deep learning Workload placement

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Speech Recognition with Time-Depth Separable Convolutions 20

Sequence-to-Sequence Speech Recognition with Time-Depth Sepa...

引用

Interspeech Conference

作者： Hannun, Awni Lee, Ann Xu, Qiantong Collobert, Ronan Facebook AI Res Menlo Pk CA 94025 USA

We propose a fully convolutional sequence-to-sequence encoder architecture with a simple and efficient decoder. Our model improves WER on LibriSpeech while being an order of magnitude more efficient than a strong RNN baseline. Key to our approach is a time-depth separable convolution block which dramatically reduces the number of parameters in the model while keeping the receptive field large. We also give a stable and efficient beam search inference procedure which allows us to effectively integrate a language model. Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.

关键词： speech recognition sequence-to-sequence neural networks

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Models for Extracting Information from Registration and Legal Documents 15th

Sequence-to-Sequence Models for Extracting Information from ...

引用

15th IAPR International Workshop on Document Analysis Systems (DAS)

作者： Pires, Ramon de Souza, Fabio C. Rosa, Guilherme Lotufo, Roberto A. Nogueira, Rodrigo NeuralMind Inteligencia Artificial Sao Paulo SP Brazil Univ Estadual Campinas Inst Comp Campinas SP Brazil Univ Estadual Campinas Sch Elect & Comp Engn Campinas SP Brazil

ISBN: (纸本)9783031065552;9783031065545

A typical information extraction pipeline consists of token- or span-level classification models coupled with a series of pre- and post-processing scripts. In a production pipeline, requirements often change, with classes being added and removed, which leads to nontrivial modifications to the source code and the possible introduction of bugs. In this work, we evaluate sequence-to-sequence models as an alternative to token-level classification methods for information extraction of legal and registration documents. We finetune models that jointly extract the information and generate the output already in a structured format. Post-processing steps are learned during training, thus eliminating the need for rule-based methods and simplifying the pipeline. Furthermore, we propose a novel method to align the output with the input text, thus facilitating system inspection and auditing. Our experiments on four real-world datasets show that the proposed method is an alternative to classical pipelines. The source code is available at https://***/neuralmind-ai/information-extraction-t5.

关键词： Information extraction sequence-to-sequence Legal texts

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Learning for Deep Gaussian Process Based Speech Synthesis Using Self-Attention GP Layer 22

Sequence-to-Sequence Learning for Deep Gaussian Process Base...

引用

Interspeech Conference

作者： Nakamura, Taiki Koriyama, Tomoki Saruwatari, Hiroshi Univ Tokyo Grad Sch Informat Sci & Technol Tokyo Japan

ISBN: (纸本)9781713836902

This paper presents a speech synthesis method based on deep Gaussian process (DGP) and sequence-to-sequence (Seq2Seq) learning toward high-quality end-to-end speech synthesis. Feed-forward and recurrent models using DGP are known to produce more natural synthetic speech than deep neural networks (DNNs) because of Bayesian learning and kernel regression. However, such DGP models consist of a pipeline architecture of independent models, acoustic and duration models, and require a high level of expertise in text processing. The proposed model is based on Seq2Seq learning, which enables a unified training of acoustic and duration models. The encoder and decoder layers are represented by Gaussian process regressions (GPRs) and the parameters are trained as a Bayesian model. We also propose a self-attention mechanism with Gaussian processes to effectively model character-level input in the encoder. The subjective evaluation results show that the proposed Seq2Seq-SA-DGP can synthesize more natural speech than DNNs with self-attention and recurrent structures. Besides, Seq2Seq-SA-DGP reduces the smoothing problems of recurrent structures and is effective when a simple input for an end-to-end system is given.

关键词： speech synthesis deep Gaussian process sequence-to-sequence Bayesian deep model sequential modeling

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence AUTOMATIC SPEECH RECOGNITION WITH WORD EMBEDDING REGULARIZATION AND FUSED DECODING

SEQUENCE-TO-SEQUENCE AUTOMATIC SPEECH RECOGNITION WITH WORD ...

引用

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Liu, Alexander H. Sung, Tzu-Wei Chuang, Shun-Po Lee, Hung-yi Lee, Lin-shah Natl Taiwan Univ Coll Elect Engn & Comp Sci Taipei Taiwan Univ Calif San Diego Dept Comp Sci & Engn San Diego CA 92103 USA

ISBN: (纸本)9781509066315

In this paper, we investigate the benefit that off-the-shelf word embedding can bring to the sequence-to-sequence (seq-to-seq) automatic speech recognition (ASR). We first introduced the word embedding regularization by maximizing the cosine similarity between a transformed decoder feature and the target word embedding. Based on the regularized decoder, we further proposed the fused decoding mechanism. This allows the decoder to consider the semantic consistency during decoding by absorbing the information carried by the transformed decoder feature, which is learned to be close to the target word embedding. Initial results on LibriSpeech demonstrated that pre-trained word embedding can significantly lower ASR recognition error with a negligible cost, and the choice of word embedding algorithms among Skip-gram, CBOW and BERT is important.

关键词： automatic speech recognition sequence-to-sequence word embedding regularization decoding

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence SINGING SYNTHESIS USING THE FEED-FORWARD TRANSFORMER

SEQUENCE-TO-SEQUENCE SINGING SYNTHESIS USING THE FEED-FORWAR...

引用

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Blaauw, Merlijn Bonada, Jordi Univ Pompeu Fabra Mus Technol Grp Barcelona Spain

ISBN: (纸本)9781509066315

We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model.

关键词： Singing synthesis sequence-to-sequence self-attention feed-forward transformer

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：