stochastic gradient descent (SGD) and its variants have established themselves as the go-to algorithms for large-scale machine learning problems with independent samples due to their generalization performance and int...
ISBN:
(纸本)9781713829546
stochastic gradient descent (SGD) and its variants have established themselves as the go-to algorithms for large-scale machine learning problems with independent samples due to their generalization performance and intrinsic computational advantage. However, the fact that the stochastic gradient is a biased estimator of the full gradient with correlated samples has led to the lack of theoretical understanding of how SGD behaves under correlated settings and hindered its use in such cases. In this paper, we focus on the Gaussian process (GP) and take a step forward towards breaking the barrier by proving minibatch SGD converges to a critical point of the full loss function and recovers model hyperparameters with rate O(1/k) up to a statistical error term depending on the minibatch size. Numerical studies on both simulated and real datasets demonstrate that minibatch SGD has better generalization over state-of-the-art GP methods while reducing the computational burden and opening up a new, previously unexplored, data size regime for GPs.
Stationary stochastic processes (SPs) are a key component of many probabilistic models, such as those for off-the-grid spatio-temporal data. They enable the statistical symmetry of underlying physical phenomena to be ...
ISBN:
(纸本)9781713829546
Stationary stochastic processes (SPs) are a key component of many probabilistic models, such as those for off-the-grid spatio-temporal data. They enable the statistical symmetry of underlying physical phenomena to be leveraged, thereby aiding generalization. Prediction in such models can be viewed as a translation equivariant map from observed data sets to predictive SPs, emphasizing the intimate relationship between stationarity and equivariance. Building on this, we propose the Convolutional Neural Process (ConvNP), which endows Neural Processes (NPs) with translation equivariance and extends convolutional conditional NPs to allow for dependencies in the predictive distribution. The latter enables ConvNPs to be deployed in settings which require coherent samples, such as Thompson sampling or conditional image completion. Moreover, we propose a new maximum-likelihood objective to replace the standard ELBO objective in NPs, which conceptually simplifies the framework and empirically improves performance. We demonstrate the strong performance and generalization capabilities of ConvNPs on 1D regression, image completion, and various tasks with real-world spatio-temporal data.
Several water index-based methods have been proposed in the literature, which, combine satellite multispectral bands in an algebraic expression. The objective of these water index-based methods is to increase the inte...
详细信息
ISBN:
(数字)9781510629684
ISBN:
(纸本)9781510629684
Several water index-based methods have been proposed in the literature, which, combine satellite multispectral bands in an algebraic expression. The objective of these water index-based methods is to increase the intensity contrast between water-pixels (surface water-body) and non-water pixels (built-up, soil, vegetation, etc.). The present investigation evaluates the Modified Normalized Difference Water Index (MNDWI) and the Automated Water Extraction Index (AWEI) using the Satellite data from Landsat 5 TM, Landsat 8 and Sentinel 2A at different time scenes. Based on visual inspection of the Lake Metztitlan water body mapping results, a high performance of AWEI approached via the OLI and the MSI sensors is observed. In the selected study area of [9210x9380]m, a statistical water pixel percentage of 30.703616% is observed in a flooding season and 9.884537% for a dry season of the year.
Convolutional neural networks (CNNs) have been widely used in image super-resolution (SR). Most existing CNN-based methods focus on achieving better performance by designing deeper/wider networks, while suffering from...
详细信息
ISBN:
(数字)9781728163956
ISBN:
(纸本)9781728163963
Convolutional neural networks (CNNs) have been widely used in image super-resolution (SR). Most existing CNN-based methods focus on achieving better performance by designing deeper/wider networks, while suffering from heavy computational cost problem, thus hindering the deployment of such models in mobile devices with limited resources. To relieve such problem, we propose a novel and efficient SR model, named Feature Affinity-based Knowledge Distillation (FAKD), by transferring the structural knowledge of a heavy teacher model to a lightweight student model. To transfer the structural knowledge effectively, FAKD aims to distill the second-order statistical information from feature maps and trains a lightweight student network with low computational and memory cost. Experimental results demonstrate the efficacy of our method and the effectiveness over other knowledge distillation based methods in terms of both quantitative and visual metrics.
Liver cancer is one of the leading causes of cancer deaths in Asia and Africa. The largest internal organ of our body is the liver. It performs many of the life sustaining functions and elementarily has effect on ever...
详细信息
ISBN:
(数字)9781728150703
ISBN:
(纸本)9781728150710
Liver cancer is one of the leading causes of cancer deaths in Asia and Africa. The largest internal organ of our body is the liver. It performs many of the life sustaining functions and elementarily has effect on every physiological process occurring in our body. Liver is the second most organ which is metastatistically affected by cancer. Now a days, liver cancer becomes the main cause of death worldwide. At that time, the early prediction of liver cancer becomes necessary for increasing patient survival. In this paper, a comparison is made to know which method from genetic, data mining and digital imageprocessing is best suited for early detection of liver cancer. After doing comparison, in this paper we decide to explore genetical approach. All the statisticalmethods work on microarray expression dataset and come under the category of genetical approach. Microarray data analysis evolved as an advanced research topic during recent decades. The key reason for analyzing microarray dataset is to identify and predict the gene that displays various values of expression under various testing conditions. The purpose of this research paper is to examine various forms of parametric statisticalmethods proposed to analyze data on microarray expression to classify the genes expressed differently and to investigate the mechanism of disease growth. Additionally, we have estimated the best situation for each system where they are doing better. In our research, we use the simulated datasets to thoroughly analyze and compare the various types of parametric methods.
With the development of intelligent manufacturing, machine vision has become one of the indispensable technologies in many fields. The so-called machine learning is to acquire images through image sensors, analyze and...
详细信息
With the evolution of wireless networks, new techniques including massive multiple-input multiple-output (MIMO) and millimeter wave are adopted to satisfy the demands for diversified services. However, it has been ver...
详细信息
ISBN:
(纸本)9781728112206
With the evolution of wireless networks, new techniques including massive multiple-input multiple-output (MIMO) and millimeter wave are adopted to satisfy the demands for diversified services. However, it has been verified by field tests that the traditional wide sense stationary assumption for wireless channel does not hold anymore. As a result, traditional channel state information (CSI) acquisition methods, especially the statistical CSI acquisition, cannot be applied straightforwardly in such a circumstance. In this paper, we propose a pre-processing method for channel sensing in the non-stationary environment. Specifically, the data sampled from channel training is treated as a channel image, where the statistical channel state is represented by gray-scale. Then the computer vision technique, specifically, the edge detection method, is used on the channel image to detect the homogeneous sub-regions. Within each sub-region, the channel is statistically stationary, and then the CSI can be obtained by existing methods. It is verified by simulation results that, the proposed method can help to improve the CSI acquisition accuracy in the non-stationary environment.
Ultrasound images have an inherently low lateral resolution due to the size of transducers that are used in standard clinical scanners. This makes for low resolution images, as well as imprecise lateral displacement e...
详细信息
ISBN:
(数字)9781728119908
ISBN:
(纸本)9781728119915
Ultrasound images have an inherently low lateral resolution due to the size of transducers that are used in standard clinical scanners. This makes for low resolution images, as well as imprecise lateral displacement estimation. In speckle tracking, the well known discipline of estimating displacement by tracking pixel movement, lateral interpolation is often used to get subsample accurate displacement estimation. Standard methods for interpolation are known as inverse distance weighting methods, of which the well known cubic interpolation method is a part. Kriging interpolation, however, is a stochastic approach that uses statistical data to calculate interpolated data points as opposed to the purely mathematical methods of more traditional interpolators. This analysis tests the efficacy of one variety of Kriging interpolation, called Simple Kriging, on ultrasound data. Simple Kriging is tested on its accuracy to interpolate a sparse ultrasound image frame, as well as its usefulness in interpolating the correlation map to estimate subsample displacement. The applied bias of the estimation using Simple Kriging is also tested by interpolating the autocorrelation map where displacement is zero. Simple Kriging is an alternative interpolation scheme that could be used with image data and its accuracy is comparable to the accuracy of using the cubic interpolation.
Digital holography is one of the 3D imaging systems that suffer Speckle noise. With respect to the importance of quality in 3D images, we develop an efficient general-purpose blind/no-reference holography image qualit...
ISBN:
(数字)9781728168326
ISBN:
(纸本)9781728168333
Digital holography is one of the 3D imaging systems that suffer Speckle noise. With respect to the importance of quality in 3D images, we develop an efficient general-purpose blind/no-reference holography image quality assessment metric for evaluating the quality of digital holography images. The main novelty of our approach to blind image quality assessment is based on the hypothesis that each digital holography has statistical properties that are changing in the presence of speckle noise. This change can be measured by some full reference metrics that are applied to input image and a new image, which were made by adding a known level of speckle noise to input image. These full reference measurements have the ability of identifying the distortion afflicting the input image and perform a no-reference quality assessment. In fact, adding noise to input image leads to quality loss, and the value of this loss give information about the input image quality. Finally, the result of the proposed method in estimating the quality of digital holography images were compared with some well-known full reference methods in order to demonstrate its ability.
Minimax optimal convergence rates for numerous classes of stochastic convex optimization problems are well characterized, where the majority of results utilize iterate averaged stochastic gradient descent (SGD) with p...
详细信息
Minimax optimal convergence rates for numerous classes of stochastic convex optimization problems are well characterized, where the majority of results utilize iterate averaged stochastic gradient descent (SGD) with polynomially decaying step sizes. In contrast, the behavior of SGD's final iterate has received much less attention despite the widespread use in practice. Motivated by this observation, this work provides a detailed study of the following question: what rate is achievable using the final iterate of SGD for the streaming least squares regression problem with and without strong convexity? First, this work shows that even if the time horizon T (i.e. the number of iterations that SGD is run for) is known in advance, the behavior of SGD's final iterate with any polynomially decaying learning rate scheme is highly sub-optimal compared to the statistical minimax rate (by a condition number factor in the strongly convex case and a factor of T in the non-strongly convex case). In contrast, this paper shows that Step Decay schedules, which cut the learning rate by a constant factor every constant number of epochs (i.e., the learning rate decays geometrically) offer significant improvements over any polynomially decaying step size schedule. In particular, the behavior of the final iterate with step decay schedules is off from the statistical minimax rate by only log factors (in the condition number for the strongly convex case, and in T in the non-strongly convex case). Finally, in stark contrast to the known horizon case, this paper shows that the anytime (i.e. the limiting) behavior of SGD's final iterate is poor (in that it queries iterates with highly sub-optimal function value infinitely often, i.e. in a limsup sense) irrespective of the stepsize scheme employed. These results demonstrate the subtlety in establishing optimal learning rate schedules (for the final iterate) for stochastic gradient procedures in fixed time horizon settings.
暂无评论