检索结果-内蒙古大学图书馆

sequence-to-sequence Traffic Missing Data Imputation via Self-Supervised Contrastive Learning

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 2025年

作者： Zhou, Wenfeng Shen, Guojiang Zhang, Yimei Deng, Zhaolin Kong, Xiangjie Xia, Feng Zhejiang Univ Technol Coll Comp Sci & Technol Hangzhou 310023 Peoples R China RMIT Univ Sch Comp Technol Melbourne Vic 3000 Australia

Missing traffic data is a common phenomenon in practice, which has serious implications for the development of downstream data-driven traffic research. Although numerous methods have been proposed to impute missing values, two limitations still exist: lack of complete ground truth and reliance on carefully constructed model structures. In this paper, we propose a novel deep neural imputation network framework named STIseq2seq, based on self-supervised contrastive learning. It follows a sequence-to-sequence architecture with attention mechanism to address the traffic data imputation problem. Specifically, the model consists of two key components: encoder and decoder, both composed of ST-block. Initially, we utilize dynamic graph convolution (DGC) and gated recurrent unit (GRU) to construct the fundamental unit, ST-block, for learning spatiotemporal dependencies. Then, a bidirectional ST-block is employed to model incomplete input data in the encoder;while a forward ST-block combined with an attention mechanism is used to accurately impute missing data in the decoder. Lastly, we devise a training method based on self-supervised contrastive learning, achieved through randomly shuffling the masking matrix, to enhance training effectiveness. Extensive experiments conducted on two available real-world traffic datasets demonstrate that our proposed STIseq2seq has superior and stable performance compared to multiple baseline models under different missing data scenarios with missing rates ranging from 20% to 80%. Visualization analysis of the imputation results further provides insights into the advantages of STIseq2seq.

关键词： Data models Imputation Spatiotemporal phenomena Contrastive learning Training Deep learning Decoding Attention mechanisms Maintenance engineering Generative adversarial networks Missing traffic data data imputation sequence-to-sequence spatiotemporal dependency self-supervised contrastive learning

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Stacked Gate Recurrent Unit Networks for Approximating the Forward Problem of Partial Differential Equations

引用

IEEE ACCESS 2024年 12卷 61795-61809页

作者： Zhang, Zhaoyang Wang, Qingwang Kunming Univ Sci & Technol Fac Informat Engn & Automat Kunming 650500 Peoples R China

We proposed an optimisation algorithm based on the sequence-to-sequence (Seq2Seq) stacking of the gate recurrent unit (GRU) model to characterise and approximate the forward problem of partial differential equations (PDEs). Unlike traditional methods based on mesh differential approximation and no parameters, this is a meshless approach based on parametric semi-supervised learning. Specifically, the algorithm employs the ability of deep feedback neural networks to approximate continuous dynamical systems, which is enhanced by stacked GRU modules to capture the evolution of the PDEs over time and thus enrich the representations of the sequences in the hidden space. The loss function of the model incorporates partial physical knowledge as an a priori condition to guide the optimisation direction, i.e., transforming the numerical iterative problem into a non-convex optimisation problem. In addition, each round of training of the model incorporates data resampling to prevent it from overfitting. We evaluated the ability of the proposed algorithm to solve mathematical physics equations for a variety of differential operators and constraints, including the heat, wave, Burgers, Schrodinger, diffusion, and Kovasnay flow equations. The experimental results confirmed the outstanding prediction precision and generalisation capability of the proposed algorithm.

关键词： Forward problem of partial differential equations deep learning gate recurrent unit sequence-to-sequence

来源：评论

学校读者我要写书评

暂无评论

A deep sequence-to-sequence model for power swing blocking of distance protection in power transmission lines

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 139卷

作者： Birgani, Amin Mehdipour Shams, Mohammadreza Jannati, Mohsen Aloghareh, Farhad Hatami Univ Isfahan Dept Elect Engn Shahreza Campus Esfahan Iran Univ Isfahan Dept Comp Engn Shahreza Campus Esfahan Iran

As the primary protection method for transmission lines, distance relays are prone to malfunction during power swings. In fact, the inability of distance relays to differentiate between power swings and short-circuit faults imposes a significant risk to power system stability that can result in blackouts. In recent years, there has been increasing interest in leveraging machine learning techniques to identify various types of faults and power swings in electrical systems. However, previous works mainly focus on fault classification, which is mostly done after along period from the moment of fault initiation. This is the reason for requiring extensive post-fault data for diagnosis. To address this challenge, this study proposes a predictive protection strategy utilizing deep learning methodologies, specifically a sequence-to-sequence model, to monitor electrical power systems continuously. The objective is to effectively detect power swings from short-circuit faults with minimal reliance on post-fault data and accurately identify short-circuit faults during power swings. In the proposed approach, features are extracted from grid current signals using the Hilbert transform and empirical mode decomposition algorithms. These features are then fed into the sequence-to-sequence model, which issues block/unblock commands upon confirming the presence of a power swing or fault during the power swing. Results from various simulations conducted on an IEEE 39-bus grid in DIgSILENT and MATLAB environments demonstrate that the proposed scheme outperforms baseline methods in the detection of short-circuit faults, power swings, and short-circuit faults occurring during the power swings. The timely and correct operation of the proposed protection scheme contributes to the stability of transmission lines and power systems.

关键词： Distance relay Power swing Hilbert transform Empirical mode decomposition sequence-to-sequence Long short-term memory

来源：评论

学校读者我要写书评

暂无评论

An Emotional Dialogue System Using Conditional Generative Adversarial Networks with a sequence-to-sequence Transformer Encoder

引用

ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING 2024年第12期23卷 1-23页

作者： Huang, Wen-chieh Hsueh, Yu-ling Natl Chung Cheng Univ Dept Comp Sci & Informat Engn Minhsiung Chiayi Taiwan Natl Chung Cheng Univ Adv Inst Mfg High Tech Innovat Minhsiung Chiayi Taiwan

Understanding the expression of emotion and generating appropriate responses are key steps toward constructing emotional, conversational agents. In this article, we propose a framework for single-turn emotional conversation generation, and there are three main components in our model, namely, a sequence-to-sequence model with stacked encoders, a conditional variational autoencoder, and conditional generative adversarial networks. For the sequence-to-sequence model with stacked encoders, we designed a two-layer encoder by combining Transformer with gated recurrent units-based neural networks. Because of the flexibility of the sequence-to-sequence model, we adopted a conditional variational autoencoder in our framework, which uses latent variables to learn a distribution over potential responses and generates diverse responses. Furthermore, we regard a conditional variational autoencoder-based, sequence-to-sequence model as the generative model, and the training of the generative model is assisted by both a content discriminator and an emotion classifier, which assists our model in promoting content information and emotion expression. We use automated evaluation and human evaluation to evaluate our model and baselines on the NII Test Collections for IR Systems short-text conversation task Chinese emotional conversation generation Subtask dataset [44], and the experimental results demonstrate that our proposed framework can generate semantically reasonable and emotionally appropriate responses.

关键词： Chinese conversation generation emotional intelligence natural lan- guage processing deep learning gated recurrent units sequence-to-sequence attention mechanism trans- former model conditional adversarial generative nets conditional variational autoencoder

来源：评论

学校读者我要写书评

暂无评论

Deep Non-Rigid Structure-From-Motion: A sequence-to-sequence Translation Perspective

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024年第12期46卷 10814-10828页

作者： Deng, Hui Zhang, Tong Dai, Yuchao Shi, Jiawei Zhong, Yiran Li, Hongdong Northwestern Polytech Univ Sch Elect & Informat Xian Peoples R China Shaanxi Key Lab Informat Acquisit & Proc Xian 710129 Shaanxi Peoples R China Ecole Polytech Fed Lausanne Sch Comp & Commun Sci CH-1015 Lausanne Switzerland Shanghai AI Lab Shanghai 200020 Peoples R China Australian Natl Univ Sch Comp Canberra ACT 2601 Australia

Directly regressing the non-rigid shape and camera pose from the individual 2D frame is ill-suited to the Non-Rigid Structure-from-Motion (NRSfM) problem. This frame-by-frame 3D reconstruction pipeline overlooks the inherent spatial-temporal nature of NRSfM, i.e., reconstructing the 3D sequence from the input 2D sequence. In this paper, we propose to solve deep sparse NRSfM from a sequence-to-sequence translation perspective, where the input 2D keypoints sequence is taken as a whole to reconstruct the corresponding 3D keypoints sequence in a self-supervised manner. First, we apply a shape-motion predictor on the input sequence to obtain an initial sequence of shapes and corresponding motions. Then, we propose the Context Layer, which enables the deep learning framework to effectively impose overall constraints on sequences based on the structural characteristics of non-rigid sequences. The Context Layer constructs modules for imposing the self-expressiveness regularity on non-rigid sequences with multi-head attention (MHA) as the core, together with the use of temporal encoding, both of which act simultaneously to constitute constraints on non-rigid sequences in the deep framework. Experimental results across different datasets such as Human3.6M, CMU Mocap, and InterHand prove the superiority of our framework. The code will be made publicly available.

关键词： Non-rigid structure-from-motion (NRSfM) self-attention sequence-to-sequence temporal encoding self-expressiveness

来源：评论

学校读者我要写书评

暂无评论

Efficient prediction uncertainty quantification in dam behavior monitoring with attention-based sequence-to-sequence learning

引用

APPLIED SOFT COMPUTING 2024年 167卷

作者： Li, Minghao Ren, Qiubing Li, Mingchao Chen, Yun Ji, Xiaocui Liu, Hao Tianjin Univ State Key Lab Hydraul Engn Intelligent Construct & Tianjin 300350 Peoples R China China Three Gorges Univ Coll Hydraul & Environm Engn Yichang 443002 Peoples R China PowerChina Zhongnan Engn Corp Ltd Changsha 410014 Peoples R China

Displacement is an intuitive monitoring indicator of dam structural behavior. Conventional deterministic modeling methods frequently disregard the inherent uncertainty associated with monitoring data. This study aims to quantify such uncertainties within dam displacement predictions by augmenting a sequence-to-sequence structure with the novel recurrent unit, termed as tiny gated unit (TGU), attention mechanism, as well as quantile regression, resulting in the Att-S2STQ model. Specifically, TGU adopts only one gate to identify nonlinearity in data sequences, while Bahdanau attention mechanism dynamically assigns weights to different parts of the input sequence. The inclusion of quantile regression allows Att-S2STQ to produce prediction intervals (PIs), from which the probability density functions (PDFs) and cumulative distribution functions (CDFs) are further derived via kernel density estimation. PIs, PDFs, and CDFs jointly serve to reveal the prediction uncertainty. The effectiveness of the model is illustrated through comparative experiments using real-world dam monitoring datasets. Results indicate that the proposed model outperforms traditional methods in point, interval, and probability predictions, while also having a simpler structure and faster training. The superiority of both accuracy and efficiency makes it a valuable tool for dam management, aiding in data-driven decision-making and enhancing operational safety.

关键词： Dam displacement prediction Uncertainty quantification sequence-to-sequence Attention mechanism Quantile regression

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence LSTM-based Dynamic System Identification of Piezo-electric Actuators

Sequence-to-sequence LSTM-based Dynamic System Identificatio...

引用

American Control Conference (ACC)

作者： Yin, Ruocheng Ren, Juan Iowa State Univ Dept Mech Engn Ames IA 50011 USA

ISBN: (纸本)9798350328066

During the past few year, recurrent neural network (RNN) has been proposed to model the nonlinear dynamics of various dynamic systems, such as nano positioning systems (e.g, piezo electric actuators (PEAs)). Although high modeling accuracy has been demonstrated using RNNs, it has been found that the conventional RNNs (such as vanilla RNN) are susceptible to gradient vanishing or exploding issue and hence difficult to train. Deep RNNs, such as Long short-term memory (LSTM), have been proposed to address these issues. However, due to the conventional training data construction, the training is susceptible to overfitting and the computation is extensive. In this paper, we propose a new type of LSTM in the application of PEA system identification: a sequence-to-sequence learning approach (namely, LSTMseq2seq). The structure of LSTMseq2seq and its training data construction are presented in detail. The efficacy of LSTMseq2seq in terms of modeling accuracy and computation speed is demonstrated by applying it for PEA system identification and comparing its performance with that of vanilla RNN.

关键词： sequence-to-sequence LSTM PEA System Identification

来源：评论

学校读者我要写书评

暂无评论

sequence-to-sequence Models for Extracting Information from Registration and Legal Documents 15th

Sequence-to-Sequence Models for Extracting Information from ...

引用

15th IAPR International Workshop on Document Analysis Systems (DAS)

作者： Pires, Ramon de Souza, Fabio C. Rosa, Guilherme Lotufo, Roberto A. Nogueira, Rodrigo NeuralMind Inteligencia Artificial Sao Paulo SP Brazil Univ Estadual Campinas Inst Comp Campinas SP Brazil Univ Estadual Campinas Sch Elect & Comp Engn Campinas SP Brazil

ISBN: (纸本)9783031065552;9783031065545

A typical information extraction pipeline consists of token- or span-level classification models coupled with a series of pre- and post-processing scripts. In a production pipeline, requirements often change, with classes being added and removed, which leads to nontrivial modifications to the source code and the possible introduction of bugs. In this work, we evaluate sequence-to-sequence models as an alternative to token-level classification methods for information extraction of legal and registration documents. We finetune models that jointly extract the information and generate the output already in a structured format. Post-processing steps are learned during training, thus eliminating the need for rule-based methods and simplifying the pipeline. Furthermore, we propose a novel method to align the output with the input text, thus facilitating system inspection and auditing. Our experiments on four real-world datasets show that the proposed method is an alternative to classical pipelines. The source code is available at https://***/neuralmind-ai/information-extraction-t5.

关键词： Information extraction sequence-to-sequence Legal texts

来源：评论

学校读者我要写书评

暂无评论

Graph augmented sequence-to-sequence model for neural question generation

引用

APPLIED INTELLIGENCE 2023年第11期53卷 14628-14644页

作者： Ma, Hui Wang, Jian Lin, Hongfei Xu, Bo Dalian Univ Technol Sch Comp Sci & Technol Dalian 116024 Peoples R China

Neural question generation (NQG) aims to generate a question from a given passage with neural networks. NQG has attracted more attention in recent years, due to its wide applications in reading comprehension, question answering, and dialogue systems. Existing works on NQG mainly use the sequence-to-sequence (Seq2Seq) or graph-to-sequence (Graph2Seq) framework. The former ignores rich structure information of the passage, while the latter is insufficient in modeling semantic information. Moreover, the target answer plays an important role in the task, because without the answer the generated question has great randomness. To effectively utilize answer information and capture both structure and semantic information of the passage, we propose a graph augmented sequence-to-sequence (GA-Seq2Seq) model. Firstly, we design an answer-aware passage representation module to integrate the answer information into the passage. Then, to discover both the structure and semantic information of the passage, we present a graph augmented passage encoder which consists of a graph encoder and a sequence encoder. Finally, we leverage an attention-based long short-term memory decoder to generate the question. Experimental results on the SQuAD and MS MARCO datasets show that our proposed model outperforms the existing state-of-the-art baselines in terms of automatic and human evaluations. The implementation is available at .

关键词： Question generation sequence-to-sequence Graph neural network Recurrent neural network Answer information

来源：评论

学校读者我要写书评

暂无评论

A Hybrid sequence -to -sequence Using Attention Mechanism and Convolutional Neural Network for Multistep Electricity Load Forecasting 19

A Hybrid Sequence -to -Sequence Using Attention Mechanism an...

引用

IEEE 19th Conference on Industrial Electronics and Applications (ICIEA)

作者： Sun, Shuo Wang, Xinli Yin, Xiaohong Wang, Lei Cheng, Xuexiao Li, Yafeng Shandong Univ Sch Control Sci & Engn Jinan Peoples R China Qingdao Univ Sci & Technol Coll Automat & Elect Engn Qingdao Peoples R China Haihui Automobile Mfg Co Ltd Rizhao Peoples R China Zhejiang Business Technol Inst Ningbo Peoples R China

ISBN: (纸本)9798350360875;9798350360868

In the realm of smart grids, exploring the characteristics of historical electricity load data of buildings and accurately predicting future electricity loads are of substantial practical significance for demand-side response and the stable operation of power systems. This paper delves into electricity load data, examining the multifaceted time series characteristics inherent in its sequence, and introduces a novel hybrid sequence-to -sequence using Attention mechanism and Convolutional neural network (AC-Seq2Seq). Within this framework, the Seq2Seq module is leveraged to capture short-term dependence, while temporal feature is discerned via the OneHot module. Concurrently, constant periodicity is identified by the CNN module, which, alongside temporal feature, facilitate the extraction of dynamic periodicity through the Attention Mechanism module. Through single-step and multi-step prediction experiments, the model demonstrates an enhanced capability in extracting diverse data characteristics and mitigating error accumulation during multi-step forecasts, significantly elevating prediction accuracy.

关键词： Keywords electricity load forecasting dynamic periodicity attention mechanism sequence-to-sequence

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：