stochastic gradient descent (SGD) and its variants have established themselves as the go-to algorithms for large-scale machine learning problems with independent samples due to their generalization performance and int...
详细信息
ISBN:
(纸本)9781713829546
stochastic gradient descent (SGD) and its variants have established themselves as the go-to algorithms for large-scale machine learning problems with independent samples due to their generalization performance and intrinsic computational advantage. However, the fact that the stochastic gradient is a biased estimator of the full gradient with correlated samples has led to the lack of theoretical understanding of how SGD behaves under correlated settings and hindered its use in such cases. In this paper, we focus on the Gaussian process ( GP) and take a step forward towards breaking the barrier by proving minibatch SGD converges to a critical point of the full loss function and recovers model hyperparameters with rate O(1/K) up to a statistical error term depending on the minibatch size. Numerical studies on both simulated and real datasets demonstrate that minibatch SGD has better generalization over state-of-the-art GP methods while reducing the computational burden and opening up a new, previously unexplored, data size regime for GPs.
stochastic computing is based on probability concepts which are different from conventional mathematical operations. Advantages of stochastic computing in the fields of neural networks and digital imageprocessing hav...
详细信息
ISBN:
(数字)9781728168234
ISBN:
(纸本)9781728168234
stochastic computing is based on probability concepts which are different from conventional mathematical operations. Advantages of stochastic computing in the fields of neural networks and digital imageprocessing have been reported in literature recently. Arithmetic operations especially multiplications can be performed either by logical AND gates in unipolar format or by EXNOR gates in bipolar format in stochastic computation. stochastic computing is inherently fault-tolerant and requires fewer logic gates to implement arithmetic operations. Long computing time and low accuracy are the main drawbacks of this system. In this presentation, to reduce hardware requirement and delay, modified stochastic multiplication using AND gate array and multiplexer are used for the design of Finite Impulse Response Filter cores. Performance parameters such as area, power and delay for FIR filter using modified stochastic computing methods are compared with conventional floating point computation.
Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving minimization problems. In this paper, we...
ISBN:
(纸本)9781713871088
Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving minimization problems. In this paper, we provide a comprehensive generalization analysis of MC-SGMs for both minimization and minimax problems through the lens of algorithmic stability in the framework of statistical learning theory. For empirical risk minimization (ERM) problems, we establish the optimal excess population risk bounds for both smooth and non-smooth cases by introducing on-average argument stability. For minimax problems, we develop a quantitative connection between on-average argument stability and generalization error which extends the existing results for uniform stability [38]. We further develop the first nearly optimal convergence rates for convex-concave problems both in expectation and with high probability, which, combined with our stability results, show that the optimal generalization bounds can be attained for both smooth and non-smooth cases. To the best of our knowledge, this is the first generalization analysis of SGMs when the gradients are sampled from a Markov process.
Deep learning image classifiers are known to be vulnerable to small adversarial perturbations of input images. In this paper, we derive the locally optimal generalized likelihood ratio test based detector for detectin...
详细信息
Deep learning image classifiers are known to be vulnerable to small adversarial perturbations of input images. In this paper, we derive the locally optimal generalized likelihood ratio test based detector for detecting stochastic targeted universal adversarial perturbations to a classifier's input. We employ a two-stage process to learn the detector's parameters, which involves unsupervised maximum likelihood estimation followed by supervised training and demonstrates better performance of the detector compared to other detection methods on several popular image classification datasets.
image visual effects can be enhanced primarily through edge and texture enhancement or contrast enhancement. image enhancement based on fractional differential can effectively enhance image details such as edge and te...
详细信息
In recent years, gun detection and threat surveillance became a popular issue as gun violence continued to threaten public safety. Convolution Neural Networks (CNN) has achieved impressive gun detection precision with...
详细信息
In real-world applications, objects can be sampled by different equipment or collected from different angles, which appears a significant and challenging problem for classifying samples from these heterogeneous views....
详细信息
ISBN:
(纸本)9781728190198
In real-world applications, objects can be sampled by different equipment or collected from different angles, which appears a significant and challenging problem for classifying samples from these heterogeneous views. Although effectiveness is achieved, most of the previous methods could not learn a discriminant and complementary representation simultaneously. Differently, in this paper, we propose a novel model termed Discriminant and Complementary Autoencoder Network (DCAEN), which integrates discriminant information between views into a compact representation by the variant of the traditional LDA-like framework. The proposed method has the following merits: (1) our model incorporates supervised information into the common representation subspace, i.e., maximize the between-class distance and minimize the within-class distance from the inter-view and intra-view. (2) our model performs view-specific representation and common representation in a unified framework. The proposed model is solved by the stochastic gradient descent (SGD) algorithm and its performance is far better than many state-of-the-art methods, demonstrating its superiority.
Compared to conventional object detection which focuses on high-level image content, face manipulation detection pays more attention to low-level artifacts and temporal discrepancies. However, there are few methods co...
详细信息
ISBN:
(纸本)9781509066315
Compared to conventional object detection which focuses on high-level image content, face manipulation detection pays more attention to low-level artifacts and temporal discrepancies. However, there are few methods considering both of these two characteristics. In this work, we propose a novel manipulation detection framework, named SSTNet, which detects tampered faces through Spatial, Steganalysis and Temporal features. Spatial features are extracted by a deep neural network for finding visible tampering traces like unnatural color, shape and texture. We propose a constraint on convolutional filters to extract steganalysis features for detecting hidden tampering artifacts like abnormal statistical characteristics of image pixels. Temporal features are extracted by a recurrent network for discovering inconsistency between consecutive frames. Experimental results on Face-Forensics++ dataset demonstrate that SSTNet outperforms other methods and achieves state-of-the-art performance on accuracy and robustness to compression. Furthermore, the generalization capability of SSTNet is verified on the GAN-based DeepFakes dataset.
This study explores the feasibility of employing eXplainable Artificial Intelligence (XAI) methodologies for the analysis of cough patterns in respiratory diseases. A cohort of 20 adult patients, all presenting persis...
详细信息
ISBN:
(数字)9798350371499
ISBN:
(纸本)9798350371505
This study explores the feasibility of employing eXplainable Artificial Intelligence (XAI) methodologies for the analysis of cough patterns in respiratory diseases. A cohort of 20 adult patients, all presenting persistent cough as a symptom of respiratory disease, was monitored for 24 hours using a smartphone. The audio signals underwent frequency domain transformation to yield 1-second spectrograms, subsequently processed by a CNN to detect cough events. Quantitative analysis of spectrogram regions relevant for cough detection highlighted by occlusion maps, revealed significant differences between patient groups. Notably, distinctions were observed between the Chronic Obstructive Pulmonary Disease (COPD) patient group and groups with other respiratory pathologies, both chronic and non-chronic. In conclusion, interpretability analysis methods applied to neural networks offer insights into cough-related distinctions among patients with varying respiratory conditions.
The development of technology caused a significant increase in the use of images in forensic cases. It is common that manipulated images are presented as evidence in courts which requires an authenticity check. In thi...
详细信息
ISBN:
(纸本)9781728172064
The development of technology caused a significant increase in the use of images in forensic cases. It is common that manipulated images are presented as evidence in courts which requires an authenticity check. In this study, we analyze splicing forgeries where the manipulations are obtained by combining different images. The proposed method divides the original images and the manipulated images into small sub-blocks. After the distinctive statistical information of the images is extracted using ELA (Error Level Analysis), the necessary discrimitative information is learned using a convolutional neural network. The method was tested on the CASIA dataset and is shown to perform comparable or better than some existing methods.
暂无评论