We introduce a novel mathematical formulation for the training of feed-forward neural networks with (potentially non-smooth) proximal maps as activation functions. This formulation is based on Bregman distances and a ...
详细信息
We introduce a novel mathematical formulation for the training of feed-forward neural networks with (potentially non-smooth) proximal maps as activation functions. This formulation is based on Bregman distances and a key advantage is that its partial derivatives with respect to the network's parameters do not require the computation of derivatives of the network's activation functions. Instead of estimating the parameters with a combination of first-order optimisation method and back-propagation (as is the state-of-the-art), we propose the use of non-smooth first-order optimisation methods that exploit the specific structure of the novel formulation. We present several numerical results that demonstrate that these training approaches can be equally well or even better suited for the training of neural network-based classifiers and (denoising) autoencoders with sparse coding compared to more conventional training frameworks.
In speech emotion recognition, training and test data used for system development usually tend to fit each other perfectly, but further 'similar' data may be available. Transfer learning helps to exploit such ...
详细信息
ISBN:
(纸本)9780769550480
In speech emotion recognition, training and test data used for system development usually tend to fit each other perfectly, but further 'similar' data may be available. Transfer learning helps to exploit such similar data for training despite the inherent dissimilarities in order to boost a recogniser's performance. In this context, this paper presents a sparse autoencoder method for feature transfer learning for speech emotion recognition. In our proposed method, a common emotion-specific mapping rule is learnt from a small set of labelled data in a target domain. Then, newly reconstructed data are obtained by applying this rule on the emotion-specific data in a different domain. The experimental results evaluated on six standard databases show that our approach significantly improves the performance relative to learning each source domain independently.
This paper proposes a dual autoencoder-based Intrusion Detection System (duAE-IDS) for the ever-changing network attacks. duAE-IDS is a protocol-based IDS, which divides traffic by its application-layer protocol. duAE...
详细信息
ISBN:
(纸本)9781665458139
This paper proposes a dual autoencoder-based Intrusion Detection System (duAE-IDS) for the ever-changing network attacks. duAE-IDS is a protocol-based IDS, which divides traffic by its application-layer protocol. duAE-IDS determines the traffic's abnormality by considering both the criteria and the application-layer protocol. The criteria are obtained by training our neural network model (duAE model) with traffic containing only one type of application-layer protocol. duAE-IDS represents each traffic flow with 67 features with eight new features for TCP traffic to improve detection accuracy. duAE-IDS uses two sparse autoencoders and one 1D CNN to extract features from traffic for every application-layer protocol. We conduct several experiments to prove the abilities and flexibilities of duAE-IDS. We prove that duAE-IDS trained with the known datasets can reach an F1-score of 0.87 for detecting attack traffic in an unknown network. We can run duAE-IDS in any network without pre-collecting the traffic of the network.
Forests suffer from heavy losses due to the occurrence of fires. A prediction model based on environmental condition, such as meteorological and vegetation indexes, is considered a promising tool to control forest fir...
详细信息
Forests suffer from heavy losses due to the occurrence of fires. A prediction model based on environmental condition, such as meteorological and vegetation indexes, is considered a promising tool to control forest fires. The construction of prediction models can be challenging due to (i) the requirement of selection of features most relevant to the prediction task, and (ii) heavily imbalanced data distribution where the number of large-scale forest fires is much less than that of small-scale ones. In this paper, we propose a forest fire prediction method that employs a sparse autoencoder-based deep neural network and a novel data balancing procedure. The method was tested on a forest fire dataset collected from the Montesinho Natural Park of Portugal. Compared to the best prediction results of other state-of-the-art methods, the proposed method could predict large-scale forest fires more accurately, and reduces the mean absolute error by 3-19.3 and root mean squared error by 0.95-19.3. The proposed method can better benefit the management of wildland fires in advance and the prevention of serious fire accidents. It is expected that the prediction performance could be further improved if additional information and more data are available.
Predictive maintenance methods assist early detection of failures and errors in machinery before they reach critical stages. This study proposes a data-driven predictive maintenance framework for the air production un...
详细信息
ISBN:
(纸本)9781665420990
Predictive maintenance methods assist early detection of failures and errors in machinery before they reach critical stages. This study proposes a data-driven predictive maintenance framework for the air production unit (APU) system of a train of Metro do Porto by deep learning based on a sparse autoencoder (SAE) network that efficiently detects abnormal data and considerably reduces the false alarm rate. Several analog and digital sensors installed on the APU system allow the detection of behavioral changes and deviations from the normal pattern by analyzing the collected data. We implemented two versions of the SAE network in which we inputted analog sensors data and digital sensors data, and the experimental results show that the failures due to air leakage problems are predicted by analog sensors data while other types of failures are identified by digital sensors data. A low pass filter is applied to the output of the SAE network, and a sequence of abnormal data is used as an alarm for the APU system failure. Performance indicators of the SAE network with digital sensors data, in terms of F1 Score, Recall, and Precision, are respectively, about 33.6%, 42%, and 28% better than those of the SAE network with analog sensors data. For comparison purposes, we also implemented a variational autoencoder (VAE). The results show that SAE performance is better than that of VAE by 14%, 77%, and 37% respectively, for Recall, Precision and F1 Score.
Exoskeleton robots have become an emerging technology in medical, industrial and military applications. Human gait phase recognition is the crucial technology for recognizing movement intention of the exoskeleton wear...
详细信息
Exoskeleton robots have become an emerging technology in medical, industrial and military applications. Human gait phase recognition is the crucial technology for recognizing movement intention of the exoskeleton wearer and controlling the exoskeleton robot. As a new biometric recognition method, gait phase recognition also plays an important role in clinical disease diagnosis, rehabilitation training and other fields. This paper proposes an integrated network model SBLSTM that combines sparse autoencoder (SAE), bidirectional long short-term memory (BiLSTM) and deep neural network (DNN) aiming at gait phase recognition during human movement. The model can accurately identify four phases in the gait cycle, including heel strike (HS), foot flat (FF), heel off (HO) and swing phase (SW). Normalization and feature extraction of collected sensor signals are performed to enhance the accuracy of recognition during the gait identification process. The processed data are input into the SBLSTM model. The introduction of SAE into the model can extract key information from gait characteristics. BiLSTM is used to learn temporal patterns and periodic changes in gait data. DNN is adopted to identify gait phases and output classification results. Different algorithms such as DNN, LSTM and SBLSTM are applied to the gait phase detection of subjects. The experimental results show that the SBLSTM algorithm is effective in gait recognition. The accuracy and F-score are outperformed by other algorithms, which verifies the effectiveness of the SBLSTM in practice.
Currently, the analysis method based on monitoring data has become an effective means of machinery fault diagnosis, and the fault diagnosis with obvious data features has achieved fruitful results. However, the incipi...
详细信息
Currently, the analysis method based on monitoring data has become an effective means of machinery fault diagnosis, and the fault diagnosis with obvious data features has achieved fruitful results. However, the incipient fault signals of equipment not only show the characteristics of weak intensity, quasi periodic and non-stationary, but also are submerged in strong background noise, which often makes it difficult to extract effective information directly from the original signals. Therefore, in order to effectively solve the problem of incipient fault diagnosis, and considering the capability of sparse autoencoder (SAE) to extract features automatically, this paper proposes a key-factor denoising strategy and an improved SAE network, and then an improved SAE network with key factor denoising strategy (KF-ISAE) based intelligent diagnosis method for quasi periodic non-stationary incipient faults is proposed. The main contributions of the proposed method are as follows. On the one hand, signal denoising that cannot be ignored in fault diagnosis is achieved by the developed incipient faults sensitivity based key-factor denoising strategy, and on the other hand, for SAE, the blindness of feature learning is handled by the formed weights constraints. In addition, the health condition identification and the fault severity level determination of machinery are completed by the improved SAE network designed in this paper. Finally, verification and comparative experiments show the effectiveness and practicability of the proposed method.
These days, face recognition systems are widely being employed in various daily applications such as smart phone unlocking, tracking school attendance, and secure online bank transactions, smarter border control, to n...
详细信息
These days, face recognition systems are widely being employed in various daily applications such as smart phone unlocking, tracking school attendance, and secure online bank transactions, smarter border control, to name a few. In spite of the remarkable progress, face recognition systems still suffer from occlusions, light variations, camera types and their resolutions. Face recognition is still a dynamic research field. In this paper, we propose an efficient face recognition system based on Gabor filter bank and a deep learning method known as sparse autoencoder (SAE). The main aim of the proposed system is to improve the features extracted by Gabor filter bank using SAE method. Then, these enhanced features are subjected to features reduction using principal component analysis and linear discriminant analysis (PCA + LDA) technique. Finally, the matching stage is accomplished via cosine Mahalanobis distance. Experiments on seven publicly available databases (i.e., JAFFE, AT & T, Yale, Georgia Tech, CASIA, Extended Yale, Essex) show that the proposed system can achieve promising results with the combination of Gabor and SAE, as well as outperform previously proposed methods.
Anomaly target detection is one of the major aims of Hyperspectral Image (HSI) processing. Since anomalous pixels Compose a small fraction of the hyperspectral data cube, the use of supervised neural networks presents...
详细信息
ISBN:
(数字)9781728172965
ISBN:
(纸本)9781728172965
Anomaly target detection is one of the major aims of Hyperspectral Image (HSI) processing. Since anomalous pixels Compose a small fraction of the hyperspectral data cube, the use of supervised neural networks presents many complications. The reason is that supervised networks need a large training set to fine-tune the model. In this paper we propose two semi-supervised autoencoder based anomaly detection methods using the reconstruction error of each pixel. The reconstruction error is the mean absolute error between each pixel and its reconstruction by the proposed autoencoder networks. The proposed networks are deep fully-connected sparse autoencoders (SAE) and deep one-dimensional convolutional autoencoders (CAE). In addition, a patch-based anomaly detection method is proposed which takes spatial correlation between neighbouring pixels into account. We use the San Diego airport hyperspectral data to carry out the experiments. The results are compared with some state-of-the-art HSI anomaly detection methods. Quantitative results employing ROC and AUC metrics demonstrate the superior performance of the proposed method compared to several anomaly detectors.
Time series classification (TSC) is a crucial and challenging problem in sequential analysis. However, most of the existing best-performing methods are time-consuming, even if coping with small-scale datasets. Broad l...
详细信息
Time series classification (TSC) is a crucial and challenging problem in sequential analysis. However, most of the existing best-performing methods are time-consuming, even if coping with small-scale datasets. Broad learning systems (BLS) have shown low time complexity and high accuracy in handling various tasks and have been applied to many fields. However, none of the BLS-based methods is suitable to tackle TSC due to (1) unsuitable structure in capturing temporal information of time series;(2) low interpretability in representing the evolving relations among states along time. Thus, this paper develops a broad fuzzy cognitive map system (BFCMS) to address time series classification efficiently, which consists of the sparse autoencoder (SAE) based feature extraction block, the highorder fuzzy cognitive map (HFCM) based spatiotemporal information aggregation block, and one multilayer perceptron (MLP) based prediction layer. The feature extraction block is designed to capture the underlying core evolving patterns, and the spatiotemporal information aggregation block is developed to model the underlying causal relationships and contextual dependencies. These two blocks are designed to overcome the limitations of BLS. MLP is applied to map the feature representation to the label of time series based on the aggregated feature representations from these two blocks. In addition, BFCMS develops three incremental learning strategies for fast updating in broad expansion without a retraining procedure if the model deems to be expanded. We compared BFCMS with other state-of-the-art baselines on 26 datasets. The experimental results demonstrate the superiority of BFCMS. Concretely, BFCMS achieves a lower training cost with on-par classification accuracy. (c) 2022 Elsevier B.V. All rights reserved.
暂无评论