检索结果-内蒙古大学图书馆

transformer-encoder-GRU (T-E-GRU) for Chinese Sentiment Analysis on Chinese Comment Text

NEURAL PROCESSING LETTERS 2023年第2期55卷 1847-1867页

作者： Zhang, Binlong Zhou, Wei Northwest Univ Xian Peoples R China

Chinese sentiment analysis (CSA) has always been one of the challenges in natural language processing due to its complexity and uncertainty. transformer has been successfully utilized in the understanding of semantics. However, it captures the sequence features in the text through position encoding, which is naturally insufficient compared with the recurrent model. To address this problem, we propose T-E-GRU. T-E-GRU combines the powerful global feature extraction of transformer encoder and the natural sequence feature extraction of GRU for CSA. The experimental evaluations are conducted on three real Chinese datasets, the experimental results show that T-E-GRU has unique advantages over recurrent model, recurrent model with attention and BERT-based model.

关键词： Chinese comments Chinese sentiment analysis Recurrent model transformer-encoder T-E-GRU

来源：评论

学校读者我要写书评

暂无评论

Evaluation of Operating State for Smart Electricity Meters Based on transformer-encoder-BiLSTM

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2023年第3期19卷 2409-2420页

作者： Zhao, Zhongyong Chen, Yu Liu, Jiangnan Cheng, Yingying Tang, Chao Yao, Chenguo Southwest Univ Coll Engn & Technol Chongqing 400716 Peoples R China State Grid Chongqing Elect Power Co Mkt Serv Ctr Metrol Ctr Chongqing 401120 Peoples R China Chongqing Univ State Key Lab Power Transmiss Equipment & Syst Sec Chongqing 400044 Peoples R China

The reliable operating state of smart electricity meters is significant in industrial applications. Faulty meters or meters in a poor measurement state will seriously impact both customers and stakeholders. However, the maintenance personnel now still use the manually periodic sampling inspection from the batch of smart electricity meters to evaluate the state of the entire batch, which has limitations of blindness, poor real time, inadequate, and insufficient examination. With the population of smart meters, it is feasible to evaluate the health condition of these devices with big data and artificial intelligence technology. One significant contribution of this article is first proposing an operating state evaluation method of smart electricity meters based on the transformer-encoder and bidirectional long-term and short-term memory. The evaluation indicators and preprocess of meters' data are carefully selected. A deep neural network is constructed and trained, the experimental verification is carried out, and the performance of the proposed method is compared with that of other traditional methods. The results show that the average classification accuracy of the proposed neural network model is 99.5%. Besides, compared with conventional machine learning and deep learning models, the proposed model is suitable for the operation state evaluation of smart electricity meters. From the experimental result, the potential benefit of the proposed method is that it could improve the accuracy and robustness of state evaluation.(1)

关键词： Bidirectional long-term and short-term memory (BiLSTM) deep learning neural network smart electricity meters state evaluation transformer-encoder

来源：评论

学校读者我要写书评

暂无评论

A Hybrid Approach for Driving Behavior Recognition: Integration of CNN and transformer-encoder with EEG data 98

A Hybrid Approach for Driving Behavior Recognition: Integrat...

引用

98th IEEE Vehicular Technology Conference (VTC-Fall)

作者： Wang, Yunlong Liu, Tianqi Qin, Yanjun Shen, Siyuan Tao, Xiaoming Tsinghua Univ Dept Elect Engn Beijing Peoples R China East China Normal Univ Sch Comp Sci & Technol Shanghai Peoples R China

ISBN: (纸本)9798350329285

Human factors are considered as one of the main causes affecting road traffic safety. Therefore, it is highly necessary to establish a driving behavior model for predicting driver behaviors and states, which can be used for risk monitoring. As an objective indicator that can accurately measure human cognitive and emotional states, electroencephalography (EEG) has attracted widespread attention from researchers due to its suitability and high temporal resolution for driver state and behavior measurement. Our study proposes a novel EEG-based driving behavior recognition algorithm that combines CNN and transformer-encoder modules. We deploy CNN to extract spatial and temporal features from EEG signals, capturing local dependencies between different time points. Subsequently, the encoder module of transformer is utilized to handle long-term dependencies, enhancing the correlation between different time steps and improving classification accuracy. Experimental results demonstrate that our proposed model outperforms the base-line model on a self-constructed driving behavior classification dataset, providing a foundation for further in-depth research into real-world road traffic safety.

关键词： Driving behavior recognition EEG signal CNN transformer-encoder

来源：评论

学校读者我要写书评

暂无评论

基于时空transformer-encoder的跨社交网络用户匹配方法

引用

计算机应用研究 2024年第12期41卷 3742-3748页

作者：张洋马强西南科技大学信息工程学院四川绵阳621010

针对目前基于签到时空数据的跨社交网络用户匹配方法未充分利用时空信息之间的耦合关系,导致时空数据特征提取困难,匹配准确率下降的问题,提出了一种基于时空transformer-encoder的跨社交网络用户匹配方法。该方法通过网格映射将签到时... 详细信息

针对目前基于签到时空数据的跨社交网络用户匹配方法未充分利用时空信息之间的耦合关系,导致时空数据特征提取困难,匹配准确率下降的问题,提出了一种基于时空transformer-encoder的跨社交网络用户匹配方法。该方法通过网格映射将签到时空信息转换为序列数据,生成签到序列;利用序列嵌入层将离散的签到序列映射到连续高维空间;然后借助多头注意力机制和卷积神经网络提取高维签到特征,并利用卷积神经网络实现优化多头注意力模块权重变换和特征融合;最后利用前馈神经网络实现分类,输出用户匹配得分。在两组真实社交网络用户数据集上进行大量用户匹配实验,与现有方法相比,准确率提升了0.40~10.53百分点,F_(1)值提升了0.43~9.5百分点。这验证了所提方法能够有效提取用户签到耦合特征,并提高用户匹配的性能。

关键词：跨社交网络用户匹配 transformer-encoder 卷积神经网络

来源：评论

学校读者我要写书评

暂无评论

BAT-transformer: Prediction of Bus Arrival Time with transformer encoder for Smart Public Transportation System

引用

APPLIED SCIENCES-BASEL 2024年第20期14卷 9488页

作者： Jeong, Suhyun Oh, Changsong Jeong, Jongpil Sungkyunkwan Univ Dept Smart Factory Convergence 2066 Seobu Ro Suwon 16419 South Korea ATEC Mobil AI Res Lab 289 Pangyo Ro Seongnam 13488 South Korea

In urban public transportation systems, the accuracy of bus arrival time prediction is crucial to reduce passenger waiting time, increase satisfaction, and ensure efficient transportation operations. However, traditional bus information systems (BISs) rely on neural network models, which have limited prediction accuracy, and some public transportation systems have non-fixed or irregular arrival times, making it difficult to directly apply traditional prediction models. Therefore, we used a transformer encoder model to effectively learn the long-term dependencies of time series data, and a multi-headed attentional mechanism to reduce the root mean square error (RMSE) and lower the mean absolute percentage error (MAPE) compared to other models to improve prediction performance. The model was trained on real bus-operation data collected from a public data portal covering the Gangnam-gu area of Seoul, Korea, and data preprocessing included missing value handling, normalization and one-hot encoding, and resampling techniques. A linear projection process, learnable location-encoding technique, and a fully connected layer were applied to the transformer-encoder model to capture the time series data more precisely. Therefore, we propose BAT-transformer, a method that applies a linear projection process, learnable location-encoding technique, and a fully connected layer using bus data. It is expected to help optimize public transportation systems and show its applicability in various urban environments.

关键词： bus information system resampling transformer-encoder bus information terminal time series

来源：评论

学校读者我要写书评

暂无评论

Network Intrusion Detection Using transformer and BiGRU-DNN in Edge Computing

引用

JOURNAL OF INFORMATION PROCESSING SYSTEMS 2024年第4期20卷 458-476页

作者： Sun, Huijuan Henan Finance Univ Coll Comp & Informat Technol Zhengzhou Peoples R China Henan Finance Univ Sch Comp & Artificial Intelligence Zhengzhou Henan Peoples R China

To address the issue of class imbalance in network traffic data, which affects the network intrusion detection performance, a combined framework using transformers is proposed. First, Tomek Links, SMOTE, and WGAN are used to preprocess the data to solve the class-imbalance problem. Second, the transformer is used to encode traffic data to extract the correlation between network traffic. Finally, a hybrid deep learning network model combining a bidirectional gated current unit and deep neural network is proposed, which is used to extract longdependence features. A DNN is used to extract deep level features, and softmax is used to complete classification. Experiments were conducted on the NSLKDD, UNSWNB15, and CICIDS2017 datasets, and the detection accuracy rates of the proposed model were 99.72%, 84.86%, and 99.89% on three datasets, respectively. Compared with other relatively new deep-learning network models, it effectively improved the intrusion detection performance, thereby improving the communication security of network data.

关键词： Bi-directional Gated Recurrent Unit Class Imbalance Deep Neural Network Edge Computing Network Intrusion Detection transformer-encoder transformer-encoder

来源：评论

学校读者我要写书评

暂无评论

Modelling monthly rainfall of India through transformer-based deep learning architecture

引用

MODELING EARTH SYSTEMS AND ENVIRONMENT 2024年第3期10卷 3119-3136页

作者： Nayak, G. H. Harish Alam, Wasi Singh, K. N. Avinash, G. Ray, Mrinmoy Kumar, Rajeev Ranjan ICAR Indian Agr Res Inst Grad Sch New Delhi 110012 India ICAR Indian Agr Stat Res Inst New Delhi 110012 India

In the realm of Earth systems modelling, the forecasting of rainfall holds crucial significance. The accurate prediction of monthly rainfall in India is paramount due to its pivotal role in determining the country's agricultural productivity. Due to this phenomenon's highly nonlinear dynamic nature, linear models are deemed inadequate. Parametric non-linear models also face limitations due to stringent assumptions. Consequently, there has been a notable surge in the adoption of machine learning approaches in recent times, owing to their data-driven nature. However, it is acknowledged that machine learning algorithms lack automatic feature extraction capabilities. This limitation has propelled the popularity of deep learning models, particularly in the domain of rainfall forecasting. Nevertheless, conventional deep learning architectures typically engage in the sequential processing of input data, a task that can prove challenging and time-consuming, especially when dealing with lengthy sequences. To address this concern, the present article proposes a rainfall modelling algorithm founded on a transformer-based deep learning architecture. The primary distinguishing feature of this approach lies in its capacity to parallelize sequential input data through an attention mechanism. This attribute facilitates expedited processing and training of larger datasets. The predictive performance of the transformer-based architecture was assessed using monthly rainfall data spanning 41 years, from 1980 to 2021, in India. Comparative evaluations were conducted with conventional recurrent neural networks, long short-term memory, and gated recurrent unit architectures. Experimental findings reveal that the transformer architecture outperforms other conventional deep learning architectures based on root mean square error and mean absolute percentage error. Furthermore, the accuracy of each architecture's predictions underwent testing using the Diebold-Mariano test. The conclusive fin

关键词： Attention mechanism encoder-decoder Gated recurrent units (GRUs) Long short-term memory (LSTM) Recurrent neural network (RNN) transformer-encoder

来源：评论

学校读者我要写书评

暂无评论

Estimating Extreme 3D Image Rotations using Cascaded Attention

Estimating Extreme 3D Image Rotations using Cascaded Attenti...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Dekel, Shay Keller, Yosi Cadik, Martin Bar Ilan Univ Ramat Gan Israel Brno Univ Technol FIT Brno Czech Republic

ISBN: (纸本)9798350353013;9798350353006

Estimating large, extreme inter-image rotations is critical for numerous computer vision domains involving images related by limited or non-overlapping fields of view. In this work, we propose an attention-based approach with a pipeline of novel algorithmic components. First, as rotation estimation pertains to image pairs, we introduce an inter-image distillation scheme using Decoders to improve embeddings. Second, whereas contemporary methods compute a 4D correlation volume (4DCV) encoding inter-image relationships, we propose an encoder-based cross-attention approach between activation maps to compute an enhanced equivalent of the 4DCV. Finally, we present a cascaded Decoder-based technique for alternately refining the cross-attention and the rotation query. Our approach outperforms current state-of-the-art methods on extreme rotation estimation. We make our code publicly available(1).

关键词： 3D reconstruction activation maps cascaded Decoder Decoder decoder-decoder module encoder-based cross-attention Extreme 3D image rotation transformer transformer-encoder

来源：评论

学校读者我要写书评

暂无评论

结合动态行为和静态特征的APT攻击检测方法

引用

计算机工程与应用 2023年第18期59卷 249-259页

作者：梁鹤李鑫尹南南李超中国人民公安大学信息网络安全学院北京100038 公安部第一研究所北京100048

针对APT攻击网络流量难以获得,模拟的数据与现实又很难匹配的问题,提出了一种基于动态行为和静态特征结合的APT攻击检测方法。采用Noriben沙箱提取待测软件的进程行为、文件行为、注册表行为和网络行为构建动态行为特征集,基于Transform... 详细信息

针对APT攻击网络流量难以获得,模拟的数据与现实又很难匹配的问题,提出了一种基于动态行为和静态特征结合的APT攻击检测方法。采用Noriben沙箱提取待测软件的进程行为、文件行为、注册表行为和网络行为构建动态行为特征集,基于transformer-encoder算法识别APT恶意软件的准确率达到了95.8%。对识别出的APT恶意软件进行组织分类,提取软件调用的DLL(dynamic link library)和API(application programming interface),并组合成DLL:API的特征形式,将1D-CNN(one dimensional convolutional neural networks)算法应用于APT恶意软件组织分类的准确率达到了98.7%,比之前的方法提高了5个百分点。与热门的深度学习算法和机器学习算法的实验效果做对比,数据表明,提出的方法相比其他方法,准确率有较大提升。

关键词：高级持续性威胁(APT)攻击动态行为静态特征 transformer-encoder 1D-CNN

来源：评论

学校读者我要写书评

暂无评论

基于注意力时间卷积网络的农产品期货分解集成预测

引用

南京信息工程大学学报 2024年第3期16卷 311-320页

作者：张大斌黄均杰凌立文林锐斌华南农业大学数学与信息学院广州510642

针对农产品期货时间序列数据受多方面因素影响,非线性、非平稳数据特征难以提取而导致预测准确性不高的问题,基于“分解-集成”的预测思想,本文提出一种基于自适应噪声完备经验模态分解(CEEMDAN)与transformer-encoder-TCN的农产品期货... 详细信息

针对农产品期货时间序列数据受多方面因素影响,非线性、非平稳数据特征难以提取而导致预测准确性不高的问题,基于“分解-集成”的预测思想,本文提出一种基于自适应噪声完备经验模态分解(CEEMDAN)与transformer-encoder-TCN的农产品期货预测方法.首先,使用CEEMDAN将时间序列分解为多尺度多频率的本征模态分量(IMF)与残差,降低了序列建模复杂度;其次,使用融合多阶段自注意力单元transformer-encoder的时间卷积网络(TCN)对各个分量子序列进行特征提取与预测,优化了序列显著特征建模权重;最后,将各个子序列预测值线性相加集成得到最终预测结果.以南华期货公司农产品指数中的大豆期货指数为研究对象,采用时序交叉验证与参数迁移的方式进行模型重训练,消融和对比实验结果表明,提出的新模型在RMSE、MAE和DS三个评价指标上具有良好的效果,验证了该模型对农产品期货预测的有效性.

关键词：农产品期货自适应噪声完备经验模态分解自注意力机制 transformer-encoder 时间卷积网络

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：