检索结果-内蒙古大学图书馆

Perturbation of deep autoencoder weights for model compression and classification of tabular data

NEURAL NETWORKS 2022年 156卷 160-169页

作者： Abrar, Sakib Samad, Manar D. Tennessee State Univ Dept Comp Sci Nashville TN 37209 USA Tennessee State Univ Dept Comp Sci 3500 John A Merritt BlvdPOB 9604 Nashville TN 37209 USA

Fully connected deep neural networks (DNN) often include redundant weights leading to overfitting and high memory requirements. Additionally, in tabular data classification, DNNs are challenged by the often superior performance of traditional machine learning models. This paper proposes periodic perturbations (prune and regrow) of DNN weights, especially at the self-supervised pretraining stage of deep autoencoders. The proposed weight perturbation strategy outperforms dropout learning or weight regularization (L1 or L2) for four out of six tabular data sets in downstream classification tasks. Unlike dropout learning, the proposed weight perturbation routine additionally achieves 15% to 40% sparsity across six tabular data sets, resulting in compressed pretrained models. The proposed pretrained model compression improves the accuracy of downstream classification, unlike traditional weight pruning methods that trade off performance for model compression. Our experiments reveal that a pretrained deep autoencoder with weight perturbation can outperform traditional machine learning in tabular data classification, whereas baseline fully-connected DNNs yield the worst classification accuracy. However, traditional machine learning models are superior to any deep model when a tabular data set contains uncorrelated variables. Therefore, the performance of deep models with tabular data is contingent on the types and statistics of constituent variables.(c) 2022 Elsevier Ltd. All rights reserved.

关键词： Weight pruning autoencoder Tabular data Model compression Sparse learning Deep neural network

来源：评论

学校读者我要写书评

暂无评论

ConvAE-LSTM: Convolutional autoencoder Long Short-Term Memory Network for Smartphone-Based Human Activity Recognition

引用

IEEE ACCESS 2022年 10卷 4137-4156页

作者： Thakur, Dipanwita Biswas, Suparna Ho, Edmond S. L. Chattopadhyay, Samiran Banasthali Vidyapith Dept Comp Sci Jaipur 304022 Rajasthan India Maulana Abul Kalam Azad Univ Technol Dept Comp Sci & Engn Kolkata 700064 W Bengal India Northumbria Univ Dept Comp & Informat Sci Newcastle Upon Tyne NE1 8ST Tyne & Wear England TCG Ctr Res & Educ Sci & Technol Inst Adv Intelligence Kolkata 700091 India Jadavpur Univ Dept Informat Technol Kolkata 700032 India

The self-regulated recognition of human activities from time-series smartphone sensor data is a growing research area in smart and intelligent health care. Deep learning (DL) approaches have exhibited improvements over traditional machine learning (ML) models in various domains, including human activity recognition (HAR). Several issues are involved with traditional ML approaches;these include handcrafted feature extraction, which is a tedious and complex task involving expert domain knowledge, and the use of a separate dimensionality reduction module to overcome overfitting problems and hence provide model generalization. In this article, we propose a DL-based approach for activity recognition with smartphone sensor data, i.e., accelerometer and gyroscope data. Convolutional neural networks (CNNs), autoencoders (AEs), and long short-term memory (LSTM) possess complementary modeling capabilities, as CNNs are good at automatic feature extraction, AEs are used for dimensionality reduction and LSTMs are adept at temporal modeling. In this study, we take advantage of the complementarity of CNNs, AEs, and LSTMs by combining them into a unified architecture. We explore the proposed architecture, namely, "ConvAE-LSTM", on four different standard public datasets (WISDM, UCI, PAMAP2, and OPPORTUNITY). The experimental results indicate that our novel approach is practical and provides relative smartphone-based HAR solution performance improvements in terms of computational time, accuracy, F1-score, precision, and recall over existing state-of-the-art methods.

关键词： Feature extraction Convolution Activity recognition Computer architecture Convolutional neural networks Older adults Monitoring Deep Learning human activity recognition smartphone sensors autoencoder

来源：评论

学校读者我要写书评

暂无评论

Generative Model of Suitable Meme Sentences for Images Using autoencoder 20th

Generative Model of Suitable Meme Sentences for Images Using...

引用

20th Pacific Rim International Conference on Artificial Intelligence

作者： Yamatomi, Ryo Mahboubi, Shahrzad Ninomiya, Hiroshi Shonan Inst Technol SIT 1-1-25 Tsujido Nishikaigan Fujisawa Kanagawa 2518511 Japan

ISBN: (纸本)9789819970186;9789819970193

This paper proposes a new image caption generative model for Memes called GUMI-AE. Meme denotes a humorous short sentence suitable for the given image in this paper. An Image caption generative model usually consists of an image encoder and a sentence decoder. Furthermore, most conventional models use a pre-trained neural network model for the image encoder, e.g., ResNet152 trained using ImageNet. However, pre-trained ResNet152 may not be effective as an encoder for extracting features from arbitrary images. Because the training samples for the meme generative model can be obtained from the website "Bokete" (in Japanese) which is a website that provides a system for people to post images and humorous short sentences associated with these images. Images posted on Bokete include a wide variety of images such as illustrations and text-only images which may be outside of the training images of ImageNet. This paper proposes an image caption generative model incorporating autoencoder (AE) as the image encoder. AE can be trained with the training samples obtained from Bokete without the image annotation. This enables the proposed method to generate short sentences with humor for memes. Finally, the proposed model is compared with the conventional one, and the evaluation of the proposed GUMI-AE will be discussed.

关键词： Image caption generative model Meme Neural Networks autoencoder

来源：评论

学校读者我要写书评

暂无评论

Low-rank reconstruction-based autoencoder for robust fault detection

引用

CONTROL ENGINEERING PRACTICE 2022年第0期123卷

作者： Hu, Zhengwei Zhao, Haitao Peng, Jingchao East China Univ Sci & Technol Sch Informat Sci & Engn Automat Dept Shanghai 200237 Peoples R China

autoencoder (AE) has been widely used in multivariate statistical process monitoring (MSPM) and various AE-based methods have been applied in fault detection. Process data is often contaminated by outliers and measurement noise which may lead to the overfitting problem for AE-based methods. In this paper, a novel feature extraction method called low-rank reconstruction-based autoencoder (LRAE) is proposed for robust fault detection. LRAE decomposes the input into a combination of a low-rank data matrix and a noise matrix. By penalizing the rank of the data matrix, LRAE separates the low-rank clean data from the contaminated process data. Instead of directly reconstructing the loss between the input data and the output data, we design a low-rank reconstruction strategy, i.e. reconstruct the loss between the low-rank clean data and the output of the AE. The proposed LRAE can be trained end-to-end by jointly optimizing an AE and a low-rank approximation. LRAE is a nonlinear method which can tackle the complicated process data better than the linear methods such as principal component analysis (PCA). Moreover, the optimization of the low-rank approximation provides the robustness of LRAE to reconstruct the clean data in the output layer when the input process data is contaminated. After training, the features of the hidden layer can be computed for further fault detection. Extensive experiments demonstrate that LRAE outperforms traditional fault detection methods, including PCA, robust principal component analysis (RPCA), kernel principal component analysis (KPCA), AE, and denoising autoencoder (DAE). Especially, LRAE provides more robust results when the process data suffer from outliers and measurement noise.

关键词： Process monitoring Robust fault detection Low-rank optimization autoencoder

来源：评论

学校读者我要写书评

暂无评论

引用

NEUROCOMPUTING 2022年第0期490卷 482-494页

作者： Li, Nanjun Chang, Faliang Liu, Chunsheng Shandong Univ Sch Control Sci & Engn 17923 Jingshi Rd Jinan 250061 Peoples R China

Automatic detection of human-related anomalous events in surveillance videos is challenging, owing to unclear definition of anomalies and insufficiency of training data. Generally, the irregular human motion patterns can be regarded as human-related abnormal events. Therefore, we propose a novel method to operate directly on sequences of human skeleton graphs for discovering the normal patterns of human motion. The sequence of skeleton graphs is decomposed into two sub-components: global movement and local posture sequences. The global component is utilized to compute local component. The local component sequences are then input to our network for capturing normal spatial-temporal motion patterns of human skeleton. Our network is established on a Spatial-temporal Graph Convolutional autoencoder (ST-GCAE) and embedded with Long Short-Term Memory (LSTM) network in hidden layers for exploring the temporal cues, which is thus called Spatial-temporal Graph Convolutional autoencoder with Embedded Long Short-Term Memory Network (STGCAE-LSTM). Different from traditional autoencoder, STGCAE-LSTM owns a single-encoder-dual-decoder architecture, which is capable of reconstructing the input and predicting the unseen future simultaneously. Then, samples that deviate from normal patterns are detected as anomalies with fusion of reconstruction and prediction errors. Experimental results on four challenging datasets demonstrate advantages of our method over other state-of-the-art algorithms. (c) 2021 Elsevier B.V. All rights reserved.

来源：评论

学校读者我要写书评

暂无评论

Latent representation learning based autoencoder for unsupervised feature selection in hyperspectral imagery

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2022年第9期81卷 12061-12075页

作者： Wang, Xinxin Wang, Zhenyu Zhang, Yongshan Jiang, Xinwei Cal, Zhihua China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China

In hyperspectral image (HSI) analysis, high-dimensional data may contain noisy, irrelevant and redundant information. To mitigate the negative effect from these information, feature selection is one of the useful solutions. Unsupervised feature selection is a data preprocessing technique for dimensionality reduction, which selects a subset of informative features without using any label information. Different from the linear models, the autoencoder is formulated to nonlinearly select informative features. The adjacency matrix of HSI can be constructed to extract the underlying relationship between each data point, where the latent representation of original data can be obtained via matrix factorization. Besides, a new feature representation can be also learnt from the autoencoder. For a same data matrix, different feature representations should consistently share the potential information. Motivated by these, in this paper, we propose a latent representation learning based autoencoder feature selection (LRLAFS) model, where the latent representation learning is used to steer feature selection for the autoencoder. To solve the proposed model, we advance an alternative optimization algorithm. Experimental results on three HSI datasets confirm the effectiveness of the proposed model.

关键词： autoencoder Unsupervised feature selection Latent representation learning Hyperspectral image classification

来源：评论

学校读者我要写书评

暂无评论

Unsupervised feature selection via adaptive autoencoder with redundancy control

引用

NEURAL NETWORKS 2022年第0期150卷 87-101页

作者： Gong, Xiaoling Yu, Ling Wang, Jian Zhang, Kai Bai, Xiao Pal, Nikhil R. China Univ Petr East China Coll Control Sci & Engn Qingdao 266580 Peoples R China China Elect Power Res Inst Beijing 100085 Peoples R China China Univ Petr East China Coll Sci Qingdao 266580 Peoples R China China Univ Petr East China Sch Petr Engn Qingdao 266580 Peoples R China Beihang Univ Beijing Adv Innovat Ctr Big Data & Brain Comp Sch Comp Sci & Engn Jiangxi Res Inst Beijing 100191 Peoples R China Indian Stat Inst Ctr Artificial Intelligence & Machine Learning Kolkata 700108 W Bengal India Indian Stat Inst ECSU Kolkata 700108 W Bengal India

Unsupervised feature selection is one of the efficient approaches to reduce the dimension of unlabeled high-dimensional data. We present a novel adaptive autoencoder with redundancy control (AARC) as an unsupervised feature selector. By adding two Group Lasso penalties to the objective function, AARC integrates unsupervised feature selection and determination of a compact network structure into a single framework. Besides, a penalty based on a measure of dependency between features (such as Pearson correlation, mutual information) is added to the objective function for controlling the level of redundancy in the selected features. To realize the desired effects of different regularizers in different phases of the training, we introduce adaptive parameters which change with iterations. In addition, a smoothing function is utilized to approximate the three penalties since they are not differentiable at the origin. An ablation study is carried out to validate the capabilities of redundancy control and structure optimization of AARC. Subsequently, comparisons with nine state-of-the-art methods illustrate the efficiency of AARC for unsupervised feature selection. (c) 2022 Elsevier Ltd. All rights reserved.

关键词： Unsupervised feature selection autoencoder Group lasso Redundancy control

来源：评论

学校读者我要写书评

暂无评论

Chronos: DDoS Attack Detection Using Time-Based autoencoder

引用

IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 2022年第1期19卷 627-641页

作者： Salahuddin, Mohammad A. Pourahmadi, Vahid Alameddine, Hyame Assem Bari, Md Faizul Boutaba, Raouf Univ Waterloo David R Cheriton Sch Comp Sci Waterloo ON N2L 3G1 Canada Amirkabir Univ Technol Dept Elect Engn Tehran *** Iran Ericsson Canada Inc Ericsson Secur Res Montreal PQ Canada Spectrum Software & Consulting Dhaka 1215 Bangladesh

Cognitive network management is becoming quintessential to realize autonomic networking. However, the wide spread adoption of the Internet of Things (IoT) devices, increases the risk of cyber attacks. Adversaries can exploit vulnerabilities in IoT devices, which can be harnessed to launch massive Distributed Denial of Service (DDoS) attacks. Therefore, intelligent security mechanisms are needed to harden network security against these threats. In this paper, we propose Chronos, a novel time-based anomaly detection system. The anomaly detector, primarily an autoencoder, leverages time-based features over multiple time windows to efficiently detect anomalous DDoS traffic. We develop a threshold selection heuristic that maximizes the F1-score across various DDoS attacks. Further, we compare the performance of Chronos against state-of-the-art approaches. We show that Chronos marginally outperforms another time-based system using a less complex anomaly detection pipeline, while out classing flow-based approaches with superior precision. In addition, we showcase the robustness of Chronos in the face of zero-day attacks, noise in training data, and a small number of training packets, asserting its suitability for online deployment.

关键词： Denial-of-service attack Feature extraction Anomaly detection Computer crime Training Training data Supervised learning Security management distributed denial of service anomaly detection autoencoder

来源：评论

学校读者我要写书评

暂无评论

Graph dynamic autoencoder for fault detection

引用

CHEMICAL ENGINEERING SCIENCE 2022年 254卷

作者： Liu, Lu Zhao, Haitao Hu, Zhengwei East China Univ Sci & Technol Sch Informat Sci & Engn Automat Dept Shanghai Peoples R China

Dynamic information is a non-negligible part of time-correlated process data, and its full utilization can improve the performance of fault detection. Traditional dynamic methods concatenate the current process data with a certain number of previous process data into an extended vector before performing feature extraction. However, this simple way of using dynamic information inevitably increases the input dimensionality and it is inappropriate to treat previous process data as equally important. To address these problems, this paper proposes a novel nonlinear dynamic method, called graph dynamic autoencoder (GDAE), for fault detection. GDAE utilizes a graph structure to model the dynamic information between different data points. GDAE firstly embeds the current data point and previous data points as the features of the central node and its neighbors, respectively, then convolves the feature of the central node with the features of its neighbors to derive the updated feature for the central node, and finally, an encoder-decoder structure is adopted to extract the key low-dimensional feature. Due to the utilization of the graph structure, the extended high-dimensional vectors utilized by traditional dynamic fault detection methods are avoided in GDAE. Furthermore, with the dynamically constructed graph, GDAE is able to adaptively assign different weights to its neighbors by updating the adjacency matrix of the graph. Experimental results obtained from a numerical simulation and the Tennessee Eastman process illustrate the superiority of GDAE in terms of missed detection rate (MDR) and false alarm rate (FAR). The source code of GDAE can be found in https://***/luliu-fighting/Graph-Dynamic-autoencoder. (c) 2022 Elsevier Ltd. All rights reserved.

关键词： Process monitoring Dynamic fault detection Graph convolution autoencoder

来源：评论

学校读者我要写书评

暂无评论

autoencoder Image Denoising to Increase Optical Character Recognition Performance in Text Conversion

Autoencoder Image Denoising to Increase Optical Character Re...

引用

International Conference on Advanced Creative Networks and Intelligent Systems (ICACNIS) - Blockchain Technology, Intelligent Systems, and the Applications for Human Life

作者： Alamsyah, Nur Fauzan, Mohamad Nurkamal Putrada, Aji Gautama Pane, Syafrial Fachri Telkom Univ Adv & Creat Networks Res Ctr Bandung Indonesia

ISBN: (纸本)9798350334449

Document digitization has an important role in helping the company's activities be more efficient, such as detecting text in invoice document images using optical character recognition (OCR). However, writing in images has many problems, especially tediously saved documents that can cause noise or interference in the picture, resulting in difficultly recognized writing. Our research aims to build an autoencoder for denoising text images and evaluate the OCR's performance in converting the denoised image into text. The first step in the research is to test the OCR characteristics on the original text image and the text image given Gaussian noise. The next step is to build the optimal autoencoder model for denoising by studying the effect of dataset size and optimizer type. The last step is to test the OCR performance on the denoised text image produced by the optimum autoencoder model. The test results show that datasetsize affects denoising performance and OCR performance. From several autoencoder models compared, the autoencoder with dataset size = 40 has the optimum performance, where the MSE values of the model for train and validation are 1277 and 1385, respectively. With images denoised from the optimum model, the OCR performance in converting images into text is 100%.

关键词： optical character recognition Gaussian noise denoising autoencoder text image

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：