检索结果-内蒙古大学图书馆

A Spatiotemporal Deep Learning Approach for unsupervised anomaly detection in Cloud Systems

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023年第4期34卷 1705-1719页

作者： He, Zilong Chen, Pengfei Li, Xiaoyun Wang, Yongfeng Yu, Guangba Chen, Cailin Li, Xinrui Zheng, Zibin Sun Yat Sen Univ Sch Data Sci & Comp Guangzhou 510006 Peoples R China

anomaly detection is a critical task for maintaining the performance of a cloud system. Using data-driven methods to address this issue is the mainstream in recent years. However, due to the lack of labeled data for training in practice, it is necessary to enable an anomaly detection model trained on contaminated data in an unsupervised way. Besides, with the increasing complexity of cloud systems, effectively organizing data collected from a wide range of components of a system and modeling spatiotemporal dependence among them become a challenge. In this article, we propose TopoMAD, a stochastic seq2seq model which can robustly model spatial and temporal dependence among contaminated data. We include system topological information to organize metrics from different components and apply sliding windows over metrics collected continuously to capture the temporal dependence. We extract spatial features with the help of graph neural networks and temporal features with long short-term memory networks. Moreover, we develop our model based on variational auto-encoder, enabling it to work well robustly even when trained on contaminated data. Our approach is validated on the run-time performance data collected from two representative cloud systems, namely, a big data batch processing system and a microservice-based transaction processing system. The experimental results show that TopoMAD outperforms some state-of-the-art methods on these two data sets.

关键词： Cloud computing Spatiotemporal phenomena neural networks unsupervised anomaly detection variational auto-encoder (VAE)

来源：评论

学校读者我要写书评

暂无评论

Correlation-based feature partition regression method for unsupervised anomaly detection

引用

APPLIED INTELLIGENCE 2022年第13期52卷 15074-15090页

作者： Liu, Zhiyu Gao, Xin Jia, Xin Xue, Bing Fu, Shiyuan Li, Kangsheng Huang, Xu Huang, Zijian Beijing Univ Posts & Telecommun Sch Artificial Intelligence Beijing 100876 Peoples R China

anomaly detection problem has been extensively studied in a variety of application domains, where the data tags are difficult to obtain. Most unsupervised algorithms rely on some notions such as distance and density to detect anomalies. However, the performance of such algorithms is easier to decrease as the dimension of the datasets increases. Some studies which use features as pseudo-labels for prediction detect anomalies according to the deviation value of the prediction model. Even so, the improvement of model performance is still restricted to ignoring the correlation between feature attributes. In this paper, we propose a correlation-based feature partition regression prediction method called CFPR, which can alleviate the adverse effects of dataset dimensions and irrelevant attributes on model performance to a certain extent. According to the correlation between the features, the high-dimensional datasets will be divided into multiple feature subspaces. In each subspace, the feature with the highest correlation coefficient will be conducted as a pseudo-label. After that, we use the remaining features as the prediction attributes to train a supervised regression prediction model. We can calculate the anomaly score of each sample in the subspace according to the difference between the regression prediction value and the true value of the pseudo-label. Furthermore, we define a weighting strategy based on the level of correlation in the subspace integration stage to obtain the final anomaly score ranking table. Extensive experiments on twenty-eight UCI public datasets show that the CFPR performs better than several state-of-art anomaly algorithms at the AUC metric.

关键词： unsupervised anomaly detection Pseudo labeling Feature correlation partition Weighted regression predict

来源：评论

学校读者我要写书评

暂无评论

Variational auto encoder fused with Gaussian process for unsupervised anomaly detection

引用

JOURNAL OF SUPERCOMPUTING 2024年第6期80卷 7662-7690页

作者： Guan, Yaonan Xu, Yunwen Xi, Yugeng Li, Dewei Dept Automat Key Lab Syst Control & Informat Proc Dongchuan Rd Shanghai 200240 Peoples R China

The unsupervised anomaly detection in high-dimensional and complex settings poses a formidable challenge. To tackle the challenges associated with the recognition of high-dimensional data, this paper proposes a feedback channel between the Variational Auto Encoder and the Gaussian process to enhance its data feature extraction capabilities. In order to alleviate the impact of common mislabeling issues within unsupervised contexts, the proposed model introduces a method for the selection of representative data. Subsequently, these representative data points are employed as anchors for the construction of a full-data Gaussian process model covering the entire dataset. To further disentangle the intertwined normal and abnormal data, the proposed model employs a Bayesian framework to guide the displacement of each data point within the latent space toward its position with the highest confidence. This strategic approach effectively discriminates between normal and abnormal data. This innovative method significantly enhances data consolidation within the latent space, thus preserving robustness, particularly when processing high-complexity data. Also, the numerical experiments not only validate the method's effectiveness but also showcase the model's robustness. These experiments highlight that even a small number of incorrect labels do not significantly impact the accuracy of anomaly detection.

关键词： Machine learning unsupervised anomaly detection Gaussian process Variational auto encoder Feature conversion

来源：评论

学校读者我要写书评

暂无评论

An Adversarial Time-Frequency Reconstruction Network for unsupervised anomaly detection

引用

NEURAL NETWORKS 2023年 168卷 44-56页

作者： Fan, Jin Wang, Zehao Wu, Huifeng Sun, Danfeng Wu, Jia Lu, Xin Hangzhou Dianzi Univ Dept Comp Sci & Technol Hangzhou Peoples R China Zhejiang Prov Key Lab Ind Internet Discrete Ind Hangzhou Peoples R China Macquarie Univ Dept Comp Sydney Australia Leeds Trinity Univ Business Sch Leeds England

Detecting anomalies in massive volumes of multivariate time series data, particularly in the IoT domain, is critical for maintaining stable systems. Existing anomaly detection models based on reconstruction techniques face challenges in distinguishing normal and abnormal samples from unlabeled data, leading to performance degradation. Moreover, accurately reconstructing abnormal values and pinpointing anomalies remains a limitation. To address these issues, we introduce the Adversarial Time-Frequency Reconstruction Network for unsupervised anomaly detection (ATF-UAD). ATF-UAD consists of a time reconstructor, a frequency reconstructor and a dual-view adversarial learning mechanism. The time reconstructor utilizes a parity sampling mechanism to weaken the dependency between neighboring points. Then attention mechanisms and graph convolutional networks (GCNs) are used to update the feature information for each point, which combines points with close feature relationships and dilutes the influence of abnormal points on normal points. The frequency reconstructor transforms the input sequence into the frequency domain using a Fourier transform and extracts the relationship between frequencies to reconstruct anomalous frequency bands. The dual-view adversarial learning mechanism aims to maximize the normal values in the reconstructed sequences and highlight anomalies and aid in their localization within the data. Through dual-view adversarial learning, ATFUAD minimizes reconstructed value errors and maximizes the identification of residual outliers. We conducted extensive experiments on nine datasets from different domains, and ATF-UAD showed an average improvement of 6.94% in terms of F1 score compared to the state-of-the-art method.(c) 2023 Elsevier Ltd. All rights reserved.

关键词： anomaly detection unsupervised anomaly detection Neural networks

来源：评论

学校读者我要写书评

暂无评论

Constructing a Meta-Learner for unsupervised anomaly detection

引用

IEEE ACCESS 2023年 11卷 45815-45825页

作者： Gutowska, Malgorzata Little, Suzanne Mccarren, Andrew Dublin City Univ Sch Comp Dublin 9 Ireland

unsupervised anomaly detection (AD) is critical for a wide range of practical applications, from network security to health and medical tools. Due to the diversity of problems, no single algorithm has been found to be superior for all AD tasks. Choosing an algorithm, otherwise known as the Algorithm Selection Problem (ASP), has been extensively examined in supervised classification problems, through the use of meta-learning and AutoML, however, it has received little attention in unsupervised AD tasks. This research proposes a new meta-learning approach that identifies an appropriate unsupervised AD algorithm given a set of meta-features generated from the unlabelled input dataset. The performance of the proposed meta-learner is superior to the current state of the art solution. In addition, a mixed model statistical analysis has been conducted to examine the impact of the meta-learner components: the meta-model, meta-features, and the base set of AD algorithms, on the overall performance of the meta-learner. The analysis was conducted using more than 10,000 datasets, which is significantly larger than previous studies. Results indicate that a relatively small number of meta-features can be used to identify an appropriate AD algorithm, but the choice of a meta-model in the meta-learner has a considerable impact.

关键词： Meta-learning Classification algorithms anomaly detection Machine learning algorithms Measurement Benchmark testing unsupervised learning unsupervised anomaly detection algorithm selection problem model selection meta-learning meta-features

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection in time-series: An extensive evaluation and analysis of state-of-the-art methods

引用

EXPERT SYSTEMS WITH APPLICATIONS 2024年 256卷

作者： Mejri, Nesryne Lopez-Fuentes, Laura Roy, Kankana Chernakov, Pavel Ghorbel, Enjie Aouada, Djamila Univ Luxembourg Interdisciplinary Ctr Secur Reliabil & Trust SnT 6 Rue Richard Coudenhove Kalergi L-1359 Luxembourg Luxembourg Univ Manouba Natl Sch Comp Sci Cristal Lab Campus Univ Manouba Manouba 2010 Tunisia

unsupervised anomaly detection in time-series has been extensively investigated in the literature. Notwithstanding the relevance of this topic in numerous application fields, a comprehensive and extensive evaluation of recent state-of-the-art techniques taking into account real-world constraints is still needed. Some efforts have been made to compare existing unsupervised time-series anomaly detection methods rigorously. However, only standard performance metrics, namely precision, recall, and F1-score are usually considered. Essential aspects for assessing their practical relevance are therefore neglected. This paper proposes an in-depth evaluation study of recent unsupervised anomaly detection techniques in time-series. Instead of relying solely on standard performance metrics, additional yet informative metrics and protocols are taken into account. In particular, (i) more elaborate performance metrics specifically tailored for time-series are used;(ii) the model size and the model stability are studied;(iii) an analysis of the tested approaches with respect to the anomaly type is provided;and (iv) a clear and unique protocol is followed for all experiments. Overall, this extensive analysis aims to assess the maturity of state-of-the-art time-series anomaly detection, give insights regarding their applicability under real-world setups and provide to the community a more complete evaluation protocol.

关键词： Time-series unsupervised anomaly detection Evaluation study

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection in pressurized water reactor digital twins using autoencoder neural networks

引用

NUCLEAR ENGINEERING AND DESIGN 2023年 413卷

作者： Cancemi, S. A. Lo Frano, R. Santus, C. Inoue, T. Univ Pisa Dept Civil & Ind Engn Pisa Italy JAMSTEC Inst Marine Earth Explorat & Engn Yokosuka Japan

Deep learning (DL), that is becoming quite popular for prediction and analysis of complex patterns in large amounts of data is used to investigate the safety behaviour of the nuclear plant items. This is achieved by using multiple layers of artificial neural networks to process and transform input data, allowing for the creation of highly accurate predictive models. Particularly to the aim the unsupervised machine learning approach and the digital twin concept in form of pressurized water reactor 2-loop simulator are used. This innovative methodology is based on neural network algorithm that makes capable to predict failures of plant structure, system, and components earlier than the activation of safety and emergency systems. Moreover, to match the objective of the study several scenarios of loss of cooling accident (LOCA) of different break size were simulated. To make the acquisition platform realistic, Gaussian noise was added to the input signals. The neural network has been fed by synthetic dataset provide by PCTRAN simulator and the efficiency in event identification was studied. Further, due to the very limited studies on the unsupervised anomaly detection by means of autoencoder neural networks applied for plant monitoring and surveillance, the methodology has been validated with experimental data from resonant test rig designed for fatigue testing of tubular components. The obtained results demonstrate the reliability and the efficiency of the methodology in detecting anomalous events prior the activation of safety system. Particularly, if the difference between the expected readings and the collected data goes beyond the predetermined threshold, then the anomalous event is identified, e.g., the model detected anomalies up to 38 min before the reactor scram intervention.

关键词： Neural Network Autoencoder Predictive Maintenance Safety NPP unsupervised anomaly detection LOCA

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection Applied to Φ-OTDR

引用

SENSORS 2022年第17期22卷 6515页

作者： Almudevar, Antonio Sevillano, Pascual Vicente, Luis Preciado-Garbayo, Javier Ortega, Alfonso Univ Zaragoza Aragon Inst Engn Res I3A ViVoLab Zaragoza 50009 Spain Univ Zaragoza Aragon Inst Engn Res I3A Appl Phys Dept Zaragoza 50009 Spain Univ Zaragoza Aragon Photon Labs APL Zaragoza 50009 Spain Univ Zaragoza Elect Engn & Commun Dept Zaragoza 50009 Spain

Distributed acoustic sensors (DASs) based on direct-detection Phi-OTDR use the light-matter interaction between light pulses and optical fiber to detect mechanical events in the fiber environment. The signals received in Phi-OTDR come from the coherent interference of the portion of the fiber illuminated by the light pulse. Its high sensitivity to minute phase changes in the fiber results in a severe reduction in the signal to noise ratio in the intensity trace that demands processing techniques be able to isolate events. For this purpose, this paper proposes a method based on unsupervised anomaly detection techniques which make use of concepts from the field of deep learning and allow the removal of much of the noise from the Phi-OTDR signals. The fact that this method is unsupervised means that no human-labeled data are needed for training and only event-free data are used for this purpose. Moreover, this method has been implemented and its performance has been tested with real data showing promising results.

关键词： distributed acoustic sensors Phi-OTDR deep learning autoencoder unsupervised anomaly detection

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection of Retail Stores Using Predictive Analysis Library on SAP HANA XS Advanced

引用

Procedia Computer Science 2021年 181卷 882-889页

作者： João Pedro Oliveira Rui Dinis Sousa University of Minho Alameda da Universidade 4804-533 Guimarães Portugal University of Minho ALGORITMI Center Alameda da Universidade 4804-533 Guimarães Portugal

The retail industry is quite exposed to fraudulent situations. Daily, thousands of transactions are processed, which may include some frauds difficult to detect, mainly when the perpetrators are the own employees at the retail stores. Large retailers with several stores across different locations may have considerable difficulty in detecting frauds involving their cashiers since they have to take into account different contexts of operation. To reduce fraud losses, retailers get an overview of the transactions in each store to filter the ones that look suspicious deviating from what would be normal. Data mining algorithms can be useful to detect anomalies, differentiating the normal from the abnormal. This study adopted the k-Means clustering algorithm for anomaly detection on a sample of 90 stores in a large food retail chain, revealing the existence of some outliers in the data. The anomaly detection process was fully implemented in SAP HANA XS Advanced using the Predictive Analysis Library (PAL). In the end, it was possible to identify the stores with abnormal behavior and conclude for the usefulness and ease of use of such a library, despite some lack of documentation to use it.

关键词： unsupervised anomaly detection k-Means clustering retail PAL SAP HANA XSA

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection algorithms on real-world data: how many do we need?

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2024年第1期25卷 5199-5232页

作者： Roel Bouman Zaharah Bukhsh Tom Heskes Institute for Computing and Information Sciences Radboud University EC Nijmegen The Netherlands Information Systems Industrial Engineering and Innovation Sciences Eindhoven University of Technology AE Eindhoven The Netherlands

In this study we evaluate 33 unsupervised anomaly detection algorithms on 52 real-world multivariate tabular data sets, performing the largest comparison of unsupervised anomaly detection algorithms to date. On this collection of data sets, the EIF (Extended Isolation Forest) algorithm significantly outperforms the most other algorithms. Visualizing and then clustering the relative performance of the considered algorithms on all data sets, we identify two clear clusters: one with "local" data sets, and another with "global" data sets. "Local" anomalies occupy a region with low density when compared to nearby samples, while "global" occupy an overall low density region in the feature space. On the local data sets the kNN (k-nearest neighbor) algorithm comes out on top. On the global data sets, the EIF (extended isolation forest) algorithm performs the best. Also taking into consideration the algorithms' computational complexity, a toolbox with these two unsupervised anomaly detection algorithms suffices for finding anomalies in this representative collection of multivariate data sets. By providing access to code and data sets, our study can be easily reproduced and extended with more algorithms and/or data sets.

关键词： unsupervised anomaly detection anomaly analysis algorithm comparison outlier detection outlier analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：