Process monitoring technology can help make the right decisions in manufacturing, but the complexity and scale of modern process industry processes render process monitoring difficult. Existing data-driven process mon...
详细信息
Process monitoring technology can help make the right decisions in manufacturing, but the complexity and scale of modern process industry processes render process monitoring difficult. Existing data-driven process monitoring methods utilize abundant monitoring data that are accumulated in industrial processes, but nonlinearity, high coupling, noise effects, and other problems continuously appear in process industry monitoring data. This study proposes a process monitoring method based on variational autoencoder and long short-term memory techniques. The method reconstructs the monitoring data by learning their distribution and time series characteristics under the controlled state, and then it monitors the state of the manufacturing process in real time by calculating the statistics. Evaluation is conducted using the Tennessee Eastman process case verification and experimental comparison method. Then, the proposed method is compared with the centralized process via principal component analysis and kernel principal component analysis. The results show that the proposed method can more significantly improve the effect of fault detection in distributed system process monitoring compared with the traditional method, and it has a better process monitoring effect.
Federated learning can achieve multi-party data-collaborative applications while safeguarding personal privacy. However, the process often leads to a decline in the quality of sample data due to a substantial amount o...
详细信息
Federated learning can achieve multi-party data-collaborative applications while safeguarding personal privacy. However, the process often leads to a decline in the quality of sample data due to a substantial amount of missing encrypted aligned data, and there is a lack of research on how to improve the model learning effect by increasing the number of samples of encrypted aligned data in federated learning. Therefore, this paper integrates the functional characteristics of deep learning models and proposes a variational autoencoder Gaussian Mixture Model Clustering Vertical Federated Learning Model (VAEGMMC-VFL), which leverages the feature extraction capability of the autoencoder and the clustering and pattern discovery capabilities of Gaussian mixture clustering on diverse datasets to further explore a large number of potentially usable samples. Firstly, the variational autoencoder is used to achieve dimensionality reduction and sample feature reconstruction of high-dimensional data samples. Subsequently, Gaussian mixture clustering is further employed to partition the dataset into multiple potential Gaussian-distributed clusters and filter the sample data using thresholding. Additionally, the paper introduces a labeled sample attribute value finding algorithm to fill in attribute values for encrypted unaligned samples that meet the requirements, allowing for the full recovery of encrypted unaligned data. In the experimental section, the paper selects four sets of datasets from different industries and compares the proposed method with three federated learning clustering methods in terms of clustering loss, reconstruction loss, and other metrics. Tests on precision, accuracy, recall, ROC curve, and F1-score indicate that the proposed method outperforms similar approaches.
Missing data is a critical challenge in industrial data analysis, particularly during anomaly incidents caused by system equipment malfunctions or, more critically, by cyberattacks in industrial systems. It impedes ef...
详细信息
Missing data is a critical challenge in industrial data analysis, particularly during anomaly incidents caused by system equipment malfunctions or, more critically, by cyberattacks in industrial systems. It impedes effective imputation and compromises data integrity. Existing statistical and machine learning techniques struggle with heavily missing data, often failing to restore original data characteristics. To address this, we propose Anomaly Signal Imputation Using Latent Coordination Relations, a framework employing a variational autoencoder (VAE) to learn from complete data and establish a robust imputation model based on latent space coordination points. Experimental results from a water treatment testbed show significant improvements in output signal fidelity despite substantial data loss, outperforming conventional techniques.
Deep generative models with latent variables have been used lately to learn joint representations and generative processes from multi-modal data, which depict an object from different viewpoints. These two learning me...
详细信息
Deep generative models with latent variables have been used lately to learn joint representations and generative processes from multi-modal data, which depict an object from different viewpoints. These two learning mechanisms can, however, conflict with each other and representations can fail to embed information on the data modalities. This research studies the realistic scenario in which all modalities and class labels are available for model training, e.g. images or handwriting, but where some modalities and labels required for downstream tasks are missing, e.g. text or annotations. We show, in this scenario, that the variational lower bound limits mutual information between joint representations and missing modalities. We, to counteract these problems, introduce a novel conditional multi-modal discriminative model that uses an informative prior distribution and optimizes a likelihood-free objective function that maximizes mutual information between joint representations and missing modalities. Extensive experimentation demonstrates the benefits of our proposed model, empirical results show that our model achieves state-of-the-art results in representative problems such as downstream classification, acoustic inversion, and image and annotation generation.
This work introduces a new condition monitoring approach for complex systems based on a standardized latent space representation. Latent variable models such as the variational autoencoders are widely used to analyze ...
详细信息
This work introduces a new condition monitoring approach for complex systems based on a standardized latent space representation. Latent variable models such as the variational autoencoders are widely used to analyze systems described by a high-dimensional physical space. The encoding of such space defines a low-dimensional and physically representative latent space. Of note, however, the latent space obtained for complex systems operating under multiple conditions is often difficult to exploit in defining an efficient Health Index, thanks to the non-deterministic and hyperparameter-dependent nature of the latent space. In addition, the distribution of the healthy cluster is not known a priori. The original contribution of this paper is to use the Nataf isoprobabilistic transform to map the latent space into a standardized space. This normalizes the spatial structure of the latent space and relaxes the model's sensitivity to hyperparameters during the learning process. Moreover, the characterization of the healthy condition in the standard Nataf space leads to the definition of two complementary health indices suitable for complex systems. An implementation in two case studies demonstrates the potential of the proposed approach. First, the approach was successfully applied within NASA's Commercial Modular Aero-Propulsion System Simulation dataset. The second case study consisted of analyzing multiple degradation in operating wind turbines. Encouraging results emerge from both case studies, with critical conditions being detected significantly earlier than in competing approaches. The proposed approach can be generalized to complex systems equipped with multiple sensors, and overcomes difficulties related to latent space analysis of multiple condition systems.
In this paper, a multineural network fusion freestyle metasurface on-demand design method is proposed. The on-demand design method involves rapidly generating corresponding metasurface patterns based on the user-defin...
详细信息
In this paper, a multineural network fusion freestyle metasurface on-demand design method is proposed. The on-demand design method involves rapidly generating corresponding metasurface patterns based on the user-defined spectrum. The generated patterns are then input into a simulator to predict their corresponding S-parameter spectrogram, which is subsequently analyzed against the real S-parameter spectrogram to verify whether the generated metasurface patterns meet the desired requirements. The methodology is based on three neural network models: a Wasserstein Generative Adversarial Network model with a U-net architecture (U-WGAN) for inverse structural design, a variational autoencoder (VAE) model for compression, and an LSTM + Attention model for forward S-parameter spectrum prediction validation. The U-WGAN is utilized for on-demand reverse structural design, aiming to rapidly discover high-fidelity metasurface patterns that meet specific electromagnetic spectrum responses. The VAE, as a probabilistic generation model, serves as a bridge, mapping input data to latent space and transforming it into latent variable data, providing crucial input for a forward S-parameter spectrum prediction model. The LSTM + Attention network, acting as a forward S-parameter spectrum prediction model, can accurately and efficiently predict the S-parameter spectrum corresponding to the latent variable data and compare it with the real spectrum. In addition, the digits "0" and "1" are used in the design to represent vacuum and metallic materials, respectively, and a 10 x 10 cell array of freestyle metasurface patterns is constructed. The significance of the research method proposed in this paper lies in the following: (1) The freestyle metasurface design significantly expands the possibility of metamaterial design, enabling the creation of diverse metasurface structures that are difficult to achieve with traditional methods. (2) The on-demand design approach can generate high-fidelit
There are different deep neural network (DNN) architectures and methods for performing augmentation on time series data, but not all the methods can be adapted for specific datasets. This article explores the developm...
详细信息
There are different deep neural network (DNN) architectures and methods for performing augmentation on time series data, but not all the methods can be adapted for specific datasets. This article explores the development of deep learning models for time series, applies data augmentation methods to conveyor belt (CB) tension signal data and investigates the influence of these methods on the accuracy of CB state classification. CB systems are one of the essential elements of production processes, enabling smooth transportation of various industrial items, therefore its analysis is highly important. For the purpose of this work, multi-domain tension data signals from five different CB load weight conditions (0.5 kg, 1 kg, 2 kg, 3 kg, 5 kg) and one damaged belt condition were collected and analysed. Four DNN models based on fully convolutional network (FCN), convolutional neural network combined with long short-term memory (CNN-LSTM) model, residual network (ResNet), and InceptionTime architectures were developed and applied to classification of CB states. Different time series augmentations, such as random Laplace noise, drifted Gaussian noise, uniform noise, and magnitude warping, were applied to collected data during the study. Furthermore, new CB tension signals were generated using a TimeVAE model. The study has shown that DNN models based on FCN, ResNet, and InceptionTime architectures are able to classify CB states accurately. The research has also shown that various data augmentation methods can improve the accuracy of the above-mentioned models, for example, the combined addition of random Laplace and drifted Gaussian noise improved FCN model's baseline (without augmentation) classification accuracy with 2.0 s-length signals by 4.5 degrees'c to 92.6 degrees'c +/- 1.54 degrees'c. FCN model demonstrated the best accuracy and classification performance despite its lowest amount of trainable parameters, thus demonstrating the importance of selecting and optimizing
This research introduces an innovative hybrid modeling framework tailored for interval prediction, aimed at forecasting water quality parameters in industrial sewage treatment plants (STPs). It tackles key challenges ...
详细信息
This research introduces an innovative hybrid modeling framework tailored for interval prediction, aimed at forecasting water quality parameters in industrial sewage treatment plants (STPs). It tackles key challenges in the field, including limited data availability, detecting anomalies, and selecting relevant features with precision. By addressing these critical gaps, the study advances predictive analytics for wastewater treatment systems. The main goal was to create a scalable and resilient model that consistently provides accurate forecasts for essential water quality indicators. To accomplish this, variational autoencoders (VAEs) were employed to generate synthetic datasets that mimic real-world patterns, improving data availability and enhancing the model's generalization capabilities. autoencoder paired with a Self-Organizing Map (SOM) was leveraged for anomaly detection and efficient feature selection. The study evaluated advanced architectures, including a Temporal Convolutional Network (TCN), TCN integrated with bidirectional Long Short-Term Memory (BiLSTM), and refined TCN_BiLSTM models featuring preand post-soft attention layers. The final model incorporated Multi- Head Attention mechanisms in both preand post-processing stages (TCN_BiLSTM_MultiHead_Attention), delivering a substantial performance improvement. The TCN_BiLSTM_MultiHead_Attention model proved to be the top performer, delivering state-of-the-art results with R2 scores of 0.9732, 0.9567, and 0.9638 for the training, validation, and test datasets, respectively. On the test set, it achieved an impressive MSE of 0.0008 and an MAE of 0.0198. These results underscore the model's exceptional accuracy in predicting key parameters, including BOD, COD, and AmmoniaNitrogen. The results highlight the significant potential of hybrid deep learning frameworks in capturing temporal patterns and complex dynamics within STP data. By integrating temporal pattern recognition, long-term dependency modeling, and
Security threats in Internet of Things (IoT) networks increased, but the lack of labeled data and limited resources hinder intrusion detection system design for IoT networks. We propose a robust hierarchical anomaly d...
详细信息
Security threats in Internet of Things (IoT) networks increased, but the lack of labeled data and limited resources hinder intrusion detection system design for IoT networks. We propose a robust hierarchical anomaly detection method based on a variational autoencoder for IoT networks. Our proposed approach includes a shallow detection stage for obvious outliers with an in-depth detection stage that explicitly measures the impact of individual features on latent representations using Shapley values, enhancing the ability to detect adversarial attacks without adversarial training. Simulations confirm the effectiveness against adversarial attacks, with almost 100% detection rates for NSL-KDD and CIC-IDS2017 datasets.
Natural Language Inference (NLI) seeks to deduce the relations of two texts: a premise and a hypothesis. These two texts may share similar or different basic contexts, while three distinct reasoning factors emerge in ...
详细信息
Natural Language Inference (NLI) seeks to deduce the relations of two texts: a premise and a hypothesis. These two texts may share similar or different basic contexts, while three distinct reasoning factors emerge in the inference from premise to hypothesis: entailment, neutrality, and contradiction. However, the entanglement of the reasoning factor with the basic context in the learned representation space often complicates the task of NLI models, hindering accurate classification and determination based on the reasoning factors. In this study, drawing inspiration from the successful application of disentangled variational autoencoders in other areas, we separate and extract the reasoning factor from the basic context of NLI data through latent variational inference. Meanwhile, we employ mutual information estimation when optimizing variational autoencoders (VAE)-disentangled reasoning factors further. Leveraging disentanglement optimization in NLI, our proposed a Directed NLI (DNLI) model demonstrates excellent performance compared to state-of-the-art baseline models in experiments on three widely used datasets: Stanford Natural Language Inference (SNLI), Multi-genre Natural Language Inference (MNLI), and Adversarial Natural Language Inference (ANLI). It particularly achieves the best average validation scores, showing significant improvements over the second-best models. Notably, our approach effectively addresses the interpretability challenges commonly encountered in NLI methods.
暂无评论