检索结果-内蒙古大学图书馆

unsupervised anomaly detection using HDG-Clustering algorithm

14th International Conference on Neural Information Processing (ICONIP 2007)

作者： Tsai, Cheng-Fa Yen, Chia-Chen Natl Pingtung Univ Sci & Technol Dept Management Informat Syst Pingtung 91201 Taiwan

ISBN: (纸本)9783540691594

As intrusion posing a serious security threat in network environments, many network intrusion detection schemes have been proposed in recent years. Most such methods employ signature-based or data-mining based techniques that rely on labeled training data, but cannot detect new types of attacks. anomaly detection techniques can be adopted to solve this problem with purely normal data. However, extracting these data is a very costly task. Unlike the approaches that rely on labeled data or purely normal data, unsupervised anomaly detection can discover "unseen" attacks by unlabeled data. This investigation presents a new mixed clustering algorithm named HDG-Clustering for unsupervised anomaly detection. The proposed algorithm is evaluated using the 1999 KDD Cup data set. Experimental results indicate that the proposed approach outperforms several existing techniques.

关键词： data clustering data mining intrusion detection anomaly detection unsupervised anomaly detection

来源：评论

学校读者我要写书评

暂无评论

unsupervised anomaly detection Using Variational Auto-Encoder based Feature Extraction

Unsupervised Anomaly Detection Using Variational Auto-Encode...

引用

IEEE International Conference on Prognostics and Health Management (ICPHM)

作者： Yao, Rong Liu, Chongdang Zhang, Linxuan Peng, Peng Tsinghua Univ Dept Automat Beijing Peoples R China

ISBN: (纸本)9781538683576

anomaly detection is a key task in Prognostics and Health Management (PHM) system. Specially, in most practical applications, the lack of labels often exists which makes the unsupervised anomaly detection very meaningful. Furthermore, unsupervised anomaly detection is also considered as a challenging task due to the diversity and information-lack of data. Variational Auto-Encoder (VAE) is a stochastic generative model which is designed to reconstruct input data as close as possible. In this paper, VAE is applied to extract valuable features for the unsupervised anomaly detection tasks. Comparison experiments are conducted on KDD CUP 99 dataset and MNIST dataset. Results show that features obtained by VAE can make unsupervised anomaly detection approaches perform better. Auto-Encoder (AE) and Kernel Principle Component Analysis (KPCA) were applied as comparisons. The result demonstrates that VAE gets best performance among them.

关键词： unsupervised anomaly detection VAE feature extraction

来源：评论

学校读者我要写书评

暂无评论

Latent feature reconstruction for unsupervised anomaly detection

引用

APPLIED INTELLIGENCE 2023年第20期53卷 23628-23640页

作者： Lin, Jinghuang He, Yifan Xu, Weixia Guan, Jihong Zhang, Ji Zhou, Shuigeng Fudan Univ Sch Comp Sci Shanghai 200438 Peoples R China Fudan Univ Shanghai Key Lab Intelligent Informat Proc Shanghai 200438 Peoples R China Tongji Univ Dept Comp Sci & Technol Shanghai 201804 Peoples R China Nanhu Headquarters Zhejiang Lab Hangzhou 311121 Peoples R China

Anomalies (or outliers) indicate a minority of data items that are quite different from the majority (inliers) of a dataset in a certain aspect. unsupervised anomaly detection (UAD) is an important but not yet extensively studied research topic. Recent deep learning based methods exploit the reconstruction gap between inliers and outliers to discriminate them. However, it is observed that the reconstruction gap often decreases rapidly as the training process goes. And there is no reasonable way to set the training stop point. To support effective UAD, we propose a new UAD framework by introducing a Latent Feature Reconstruction (LFR) layer that can be applied to recent UAD methods. The LFR layer acts as a regularizer to constrain the latent features in a low-rank subspace from which inliers can be reconstructed well while outliers cannot. We develop two new UAD methods by implementing the proposed framework with autoencoder architecture and geometric transformation scheme. Experiments on five benchmarks show that our proposed methods can achieve state-of-the-art performance in most cases.

关键词： unsupervised anomaly detection Latent feature reconstruction Autoencoder Geometric transformation

来源：评论

学校读者我要写书评

暂无评论

Adaptive Memory Networks With Self-Supervised Learning for unsupervised anomaly detection

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2023年第12期35卷 12068-12080页

作者： Zhang, Yuxin Wang, Jindong Chen, Yiqiang Yu, Han Qin, Tao Global Energy Interconnect Dev & Cooperat Org Beijing 100031 Peoples R China Chinese Acad Sci Inst Comp Technol Beijing Key Lab Mobile Comp & Pervas Device Beijing 100045 Peoples R China Univ Chinese Acad Sci Beijing 101408 Peoples R China Microsoft Res Beijing 100080 Peoples R China Peng Cheng Lab PCL Shenzhen 518066 Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore 639798 Singapore

unsupervised anomaly detection aims to build models to effectively detect unseen anomalies by only training on the normal data. Although previous reconstruction-based methods have made fruitful progress, their generalization ability is limited due to two critical challenges. First, the training dataset only contains normal patterns, which limits the model generalization ability. Second, the feature representations learned by existing models often lack representativeness which hampers the ability to preserve the diversity of normal patterns. In this paper, we propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges and enhance the generalization ability in unsupervised anomaly detection. Based on the convolutional autoencoder structure, AMSL incorporates a self-supervised learning module to learn general normal patterns and an adaptive memory fusion module to learn rich feature representations. Experiments on four public multivariate time series datasets demonstrate that AMSL significantly improves the performance compared to other state-of-the-art methods. Specifically, on the largest CAP sleep stage detection dataset with 900 million samples, AMSL outperforms the second-best baseline by 4%+ in both accuracy and F1 score. Apart from the enhanced generalization ability, AMSL is also more robust against input noise.

关键词： unsupervised anomaly detection time series self-supervised learning memory network

来源：评论

学校读者我要写书评

暂无评论

Dual-Constraint Autoencoder and Adaptive Weighted Similarity Spatial Attention for unsupervised anomaly detection

引用

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2024年第7期20卷 9393-9403页

作者： Zhang, Ruifan Wang, Hao Feng, Mingyao Liu, Yikun Yang, Gongping Shandong Univ Sch Software Jinan 250101 Peoples R China

Image reconstruction-based methods with autoencoder have been widely used for unsupervised anomaly detection. By training the reconstruction on normal samples, autoencoder is supposed to produce higher reconstruction error for anomalous samples, which is used as an indicator for detecting anomalies. However, since autoencoder adopts the bottleneck layer to reconstruct data, it is hard to control its generalization capability. When the generalization capability is high, anomalous features can be confused with normal features, resulting in accurate reconstruction of anomalous regions as well. In this article, we propose a dual-constraint autoencoder to alleviate the problem of feature confusion through the dual constraint of adversarial learning and global memory bank. Given a sample pair consisting of a normal sample and a synthetic anomaly sample generated from it as input, the proposed autoencoder first forces the encoding of the synthetic anomaly image to be close to the normal encoding by adversarial learning. Then, the synthetic anomaly encoding is rerepresented with items in memory bank, which records normal features that facilitate image restoration, and the new feature is fed to the decoder for image inpainting. Under such constraints, the generalization capability of the autoencoder is suppressed and will not reconstruct the anomaly region well. After that, we feed the synthetic anomaly image together with the repaired image into the proposed adaptive weighted similarity spatial attention-based U-Net and produce an accurate anomaly map with the help of our proposed adaptive weighted similarity spatial attention. Results on the a comprehensive real-world dataset for unsupervised anomaly detection (MVTec AD) dataset and the beanTech anomaly detection dataset dataset show that our framework achieves state-of-the-art performance.

关键词： Image reconstruction Feature extraction anomaly detection Decoding Noise reduction Training Encoding Adaptive weighted similarity attention adversarial learning dual-constraint autoencoder (DC-AE) synthetic anomaly unsupervised anomaly detection

来源：评论

学校读者我要写书评

暂无评论

Robust unsupervised anomaly detection via multi-time scale DCGANs with forgetting mechanism for industrial multivariate time series

引用

NEUROCOMPUTING 2021年 423卷 444-462页

作者： Liang, Haoran Song, Lei Wang, Jianxing Guo, Lili Li, Xuzhi Liang, Ji Chinese Acad Sci Technol & Engn Ctr Space Utilizat Key Lab Space Utilizat Beijing 100094 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China Tsinghua Univ Sch Software Beijing 100084 Peoples R China

Detecting anomalies in time series is a vital technique in a wide variety of industrial application in which sensors monitor expensive machinery. The complexity of this task increases when heterogeneous sensors provide information of different attributes, scales and characteristics from the same machine. Actually, the challenges of anomaly detection for industrial time series are to design effective pre-processing, feature extraction and overcome the lack of abnormal samples. Recent deep learning models have shown prominent abilities on raw multivariate time series, alleviating these previous works. In this work, a novel framework named multi-time scale deep convolutional generative adversarial network (MTS-DCGAN) is proposed to deal with anomaly detection of industrial time series. Firstly, multivariate time series are transformed into the multi-channel signature matrices via sliding window based cross correlation computation, and therein forgetting mechanism is introduced to effectively avoid false alarms due to excessive influence of old sequences. Then the framework conducts an unsupervised adversarial training of multi-channel signature matrices and capture their hidden features via deep convolution structure. Besides, a novel threshold setting strategy is proposed to optimize anomaly detection performance under imbalance of normal and abnormal data. Finally, the proposed framework is assessed against the experiments on four datasets. Within this study, experimental results show our framework outperforms the comparison algorithms in terms of model performance and robustness, providing an effective anomaly detection method for industrial multivariate time series. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Multivariate time series unsupervised anomaly detection Multi-time scale Deep convolutional generative adversarial networks Threshold setting strategy

来源：评论

学校读者我要写书评

暂无评论

Negative Selection Algorithm for unsupervised anomaly detection

引用

APPLIED SCIENCES-BASEL 2024年第23期14卷

作者： Bereta, Michal Cracow Univ Technol Dept Comp Sci PL-31155 Krakow Poland

In this work, we present a modification of the well-known Negative Selection Algorithm (NSA), inspired by the process of T-cell generation in the immune system. The approach employs spherical detectors and was initially developed in the context of semi-supervised anomaly detection. The novelty of this work lies in proposing an adapted version of the NSA for unsupervised anomaly detection. The goal is to develop a method that can be applied to datasets that may not only represent self-data but also contain a small percentage of anomalies, which must be detected without prior knowledge of their locations. The proposed unsupervised algorithm leverages neighborhood sampling and ensemble methods to enhance its performance. We conducted comparative tests with 11 other algorithms across 17 datasets with varying characteristics. The results demonstrate that the proposed algorithm is competitive. The proposed algorithm performs well across multiple metrics, including accuracy, AUC, precision, recall, F1 score, Cohen's kappa, and Matthews correlation coefficient. It consistently ranks among the top algorithms for recall, indicating its effectiveness in scenarios where detecting all existing anomalies is critical, even at the expense of some increase in false positives. Further research is possible and may focus on exploring normalization procedures, improving threshold automation, and extending the method for more detailed anomaly confidence assessments.

关键词： anomaly detection negative selection algorithm unsupervised anomaly detection artificial immune systems outlier detection unsupervised negative selection algorithm

来源：评论

学校读者我要写书评

暂无评论

Exploiting Structural Consistency of Chest Anatomy for unsupervised anomaly detection in Radiography Images

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2024年第9期46卷 6070-6081页

作者： Xiang, Tiange Zhang, Yixiao Lu, Yongyi Yuille, Alan Zhang, Chaoyi Cai, Weidong Zhou, Zongwei Univ Sydney Sch Comp Sci Camperdown NSW 2006 Australia Johns Hopkins Univ Dept Comp Sci Baltimore MD 21218 USA

Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. Exploiting this structured information could potentially ease the detection of anomalies from radiography images. To this end, we propose a Simple Space-Aware Memory Matrix for In-painting and Detecting anomalies from radiography images (abbreviated as SimSID). We formulate anomaly detection as an image reconstruction task, consisting of a space-aware memory matrix and an in-painting block in the feature space. During the training, SimSID can taxonomize the ingrained anatomical structures into recurrent visual patterns, and in the inference, it can identify anomalies (unseen/modified visual patterns) from the test image. Our SimSID surpasses the state of the arts in unsupervised anomaly detection by +8.0%, +5.0%, and +9.9% AUC scores on ZhangLab, COVIDx, and CheXpert benchmark datasets, respectively.

关键词： unsupervised anomaly detection radiography image analysis image in-painting unsupervised anomaly detection radiography image analysis image in-painting

来源：评论

学校读者我要写书评

暂无评论

Seismic erratic noise attenuation using unsupervised anomaly detection

引用

GEOPHYSICAL PROSPECTING 2021年第7期69卷 1473-1486页

作者： Jeong, Woodon Almubarak, Mohammed S. Tsingas, Constantinos Saudi Aramco EXPEC Adv Res Ctr Dhahran Saudi Arabia

This study introduces a new attribute to identify seismic erratic noise, i.e. outlier, in the context of unsupervised anomaly detection and is defined as local outlier probabilities. The local outlier probabilities calculate scores of degrees of isolation, i.e. outlier-ness, for each object in a data set, which represents how far an object is deviated from its surrounding objects. Since the local outlier probabilities combines a density-based outlier detection method with a statistically oriented scheme, its scoring system provides regularized outlier-ness, which is an outlier probability, to be used for making a binary decision to do inclusion or exclusion of an object;such a decision only requires a simple and straightforward threshold on a probability. Based on the binary decision that flags outliers versus non-outliers, local outlier probabilities-denoising workflows are developed by combining multiple steps to complete an application of the local outlier probabilities to attenuate seismic erratic noise. Higher stability and improved robustness in the detection and rejection of seismic erratic noise have been achieved by implementing moving windows and decision tree-based processes. To avoid loss of useful signal energy, signal enhancement applications are additionally suggested. Numerical experiments on synthetic data investigate the applicability of the proposed algorithms to seismic erratic noise attenuation. Field data examples demonstrate the feasibility of a local outlier probabilities-denoising application as an effective tool in seismic denoising portfolio.

关键词： Erratic noise Seismic data processing unsupervised anomaly detection unsupervised machine learning

来源：评论

学校读者我要写书评

暂无评论

Robust Tracking via Learning Model Update With unsupervised anomaly detection Philosophy

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 2023年第5期33卷 2330-2341页

作者： Gao, Jie Zhong, Bineng Chen, Yan Guangxi Normal Univ Guangxi Key Lab Multisource Informat Min & Secur Guilin 541004 Peoples R China Huaqiao Univ Dept Comp Sci & Technol Xiamen 361021 Peoples R China Huaqiao Univ Dept Comp Sci & Technol Quanzhou 362021 Peoples R China

Template tracking is a typical paradigm to adaptively locate arbitrary objects in the tracking literature. Although existing works present diverse template updating approaches, one of the essential problems of template updating has not been solved effectively, i.e., when and how to update a template. In this work, we treat the updating time as an abnormal moment that indicates the previous template cannot depict the target accurately any more. Thus, we introduce an effective State-Edge Awareness (SEA) module that detect such abnormal moments via unsupervised anomaly detection. To be specific, by retaining multi search frames of a video, SEA firstly analysis the correlation features that generated by the template and search images. Then, it estimates the measurement for abnormal degree that is regarded as the sign for template updating. As a result, our method can not only capture the updating time automatically, but also update the templates effectively. Furthermore, the effectiveness of the proposed method has been verified on a representative CNN-based and Transformer-based tracker, respectively. The experimental results on five popular benchmarks show that our tracker can achieve the state-of-the-art performance.

关键词： Target tracking anomaly detection Reliability Noise measurement Visualization Philosophical considerations Transformers Visual tracking unsupervised anomaly detection template tracking template updating

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：