检索结果-内蒙古大学图书馆

Johnson-lindenstrauss lemma, linear and nonlinear random projections, random fourier features, and random kitchen sinks: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science & David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on the Johnson-Lindenstrauss (JL) lemma and linear and nonlinear random projections. We start with linear random projection and then justify its correctness by JL lemma and its proof. Then, sparse random projections with `1 norm and interpolation norm are introduced. Two main applications of random projection, which are low-rank matrix approximation and approximate nearest neighbor search by random projection onto hypercube, are explained. Random Fourier Features (RFF) and Random Kitchen Sinks (RKS) are explained as methods for nonlinear random projection. Some other methods for nonlinear random projection, including extreme learning machine, randomly weighted neural networks, and ensemble of random projections, are also introduced. Copyright © 2021, The Authors. All rights reserved.

关键词： Surveys

Uniform manifold approximation and projection (UMAP) and its variants: Tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

Uniform Manifold Approximation and Projection (UMAP) is one of the state-of-the-art methods for dimensionality reduction and data visualization. This is a tutorial and survey paper on UMAP and its variants. We start with UMAP algorithm where we explain probabilities of neighborhood in the input and embedding spaces, optimization of cost function, training algorithm, derivation of gradients, and supervised and semisupervised embedding by UMAP. Then, we introduce the theory behind UMAP by algebraic topology and category theory. Then, we introduce UMAP as a neighbor embedding method and compare it with t-SNE and LargeVis algorithms. We discuss negative sampling and repulsive forces in UMAP's cost function. DensMAP is then explained for density-preserving embedding. We then introduce parametric UMAP for embedding by deep learning and progressive UMAP for streaming and out-of-sample data embedding. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper on factor analysis, probabilistic Principal Component Analysis (PCA), variational inference, and Variational Autoencoder (VAE). These methods, which are tightly related, are dimensionality reduction and generative models. They assume that every data point is generated from or caused by a lowdimensional latent factor. By learning the parameters of distribution of latent space, the corresponding low-dimensional factors are found for the sake of dimensionality reduction. For their stochastic and generative behaviour, these models can also be used for generation of new data points in the data space. In this paper, we first start with variational inference where we derive the Evidence Lower Bound (ELBO) and Expectation Maximization (EM) for learning the parameters. Then, we introduce factor analysis, derive its joint and marginal distributions, and work out its EM steps. Probabilistic PCA is then explained, as a special case of factor analysis, and its closed-form solutions are derived. Finally, VAE is explained where the encoder, decoder and sampling from the latent space are introduced. Training VAE using both EM and backpropagation are explained. Copyright © 2021, The Authors. All rights reserved.

关键词： Maximum principle

Sufficient dimension reduction for high-dimensional regression and low-dimensional embedding: tutorial and survey

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science & David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on various methods for Sufficient Dimension Reduction (SDR). We cover these methods with both statistical high-dimensional regression perspective and machine learning approach for dimensionality reduction. We start with introducing inverse regression methods including Sliced Inverse Regression (SIR), Sliced Average Variance Estimation (SAVE), contour regression, directional regression, Principal Fitted Components (PFC), Likelihood Acquired Direction (LAD), and graphical regression. Then, we introduce forward regression methods including Principal Hessian Directions (pHd), Minimum Average Variance Estimation (MAVE), Conditional Variance Estimation (CVE), and deep SDR methods. Finally, we explain Kernel Dimension Reduction (KDR) both for supervised and unsupervised learning. We also show that supervised KDR and supervised PCA are equivalent. Copyright © 2021, The Authors. All rights reserved.

关键词： Regression analysis

Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper for nonlinear dimensionality and feature extraction methods which are based on the Laplacian of graph of data. We first introduce adjacency matrix, definition of Laplacian matrix, and the interpretation of Laplacian. Then, we cover the cuts of graph and spectral clustering which applies clustering in a subspace of data. Different optimization variants of Laplacian eigenmap and its out-of-sample extension are explained. Thereafter, we introduce the locality preserving projection and its kernel variant as linear special cases of Laplacian eigenmap. Versions of graph embedding are then explained which are generalized versions of Laplacian eigenmap and locality preserving projection. Finally, diffusion map is introduced which is a method based on Laplacian of data and random walks on the data graph. Copyright © 2021, The Authors. All rights reserved.

关键词： Surveys

Unified Framework for Spectral Dimensionality Reduction, Maximum Variance Unfolding, and Kernel learning By Semidefinite Programming: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2021年

This is a tutorial and survey paper on unification of spectral dimensionality reduction methods, kernel learning by Semidefinite Programming (SDP), Maximum Variance Unfolding (MVU) or Semidefinite Embedding (SDE), and its variants. We first explain how the spectral dimensionality reduction methods can be unified as kernel Principal Component Analysis (PCA) with different kernels. This unification can be interpreted as eigenfunction learning or representation of kernel in terms of distance matrix. Then, since the spectral methods are unified as kernel PCA, we say let us learn the best kernel for unfolding the manifold of data to its maximum variance. We first briefly introduce kernel learning by SDP for the transduction task. Then, we explain MVU in detail. Various versions of supervised MVU using nearest neighbors graph, by class-wise unfolding, by Fisher criterion, and by colored MVU are explained. We also explain out-of-sample extension of MVU using eigenfunctions and kernel mapping. Finally, we introduce other variants of MVU including action respecting embedding, relaxed MVU, and landmark MVU for big data. Copyright © 2021, The Authors. All rights reserved.

关键词： Surveys

End-to-end simultaneous learning of single-particle orientation and 3D map reconstruction from cryo-electron microscopy data

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Nashed, Youssef S.G. Poitevin, Frederic Gupta, Harshit Woollard, Geoffrey Kagan, Michael Yoon, Chuck Ratner, Daniel Machine Learning Initiative SLAC National Accelerator Laboratory United States Department of LCLS Data Analytics SLAC National Accelerator Laboratory United States Department of Mathematics University of British Columbia Canada Fundamental Physics Directorate SLAC National Accelerator Laboratory United States

Cryogenic electron microscopy (cryo-EM) provides images from different copies of the same biomolecule in arbitrary orientations. Here, we present an end-to-end unsupervised approach that learns individual particle orientations from cryo-EM data while reconstructing the average 3D map of the biomolecule, starting from a random initialization. The approach relies on an auto-encoder architecture where the latent space is explicitly interpreted as orientations used by the decoder to form an image according to the linear projection model. We evaluate our method on simulated data and show that it is able to reconstruct 3D particle maps from noisy- and CTF-corrupted 2D projection images of unknown particle orientations. © 2021, CC BY.

关键词： Biomolecules

A Generic and Robust System for Automated Detection of Different Classes of Arrhythmia

学校读者我要写书评

暂无评论

Procedia Computer Science 2020年 167卷 1801-1810页

作者： Vandana Singh U. Srinivasulu Reddy G. Manju Bhargavia M.Tech (Data Analytics) National Institute of Technology Tiruchirappalli 620015 India Assistant Professor Machine Learning and Data Analytics Lab Department of Computer Applications National Institute of Technology Tiruchirappalli 620015 India Post-Doc Department of Physics National Institute of Technology Tiruchirappalli 620015 India

Cardiovascular Arrhythmias (irregular beat) are related to the sudden death, can be characterized into two kinds, life-threatening (dangerous) and non-life-threatening. In clinical research and diagnosis analysis electrocardiography (ECG) plays an important role. The Advancement of Medical Instrumentation (AAMI) suggests, all the heartbeat of MIT-BIH dataset into four classes, namely, normal or bundle branch block (N), supraventricular ectopic (S), ventricular ectopic (V) and fusion of ventricular and normal (F). The objective of this paper is the classification of arrhythmias on the ECG as per AAMI standards. First beat detection and then on a given window size beat segmentation is performed. Then, features computed from ECG signals are RR (separation between two successive heartbeat pulses), local binary pattern(LBP), morphological changes, wavelet, high order statistics, and several amplitude values; Pre-RR, Post-RR, Local RR and Global RR. The main reason is to computing many morphological features to comparison analysis with the model when trained and tested on single features vs combining all features together. Model gives highest performance which is 90.8% accuracy using LBP as single feature compared with other features. Combined all important features together and optimal features (total 104 ) are given as an input to Single SVM classifier as well as ensembles of SVM which maps the component feature vectors to the particular class name. Principal Component Analysis (PCA) technique is also applied to the model which reduces the total number of features from 104 to 30 features. The Single SVM system can accurately classify four beat types and has an overall classification rate of 94.05% without PCA and using PCA 92.84% while the Ensemble SVM classifies and with an accuracy of 92.96%

关键词： Electrocardiogram (ECG) signal MIT-BIH dataset Wavelet Transform Support Vector machines OVO

Deep CNN: A machine learning approach for driver drowsiness detection based on eye state

学校读者我要写书评

暂无评论

Revue d'Intelligence Artificielle 2019年第6期33卷 461-466页

作者： Reddy Chirra, Venkata Rami Uyyala, Srinivasulu Reddy Kishore Kolli, Venkata Krishna Department of Computer Applications National Institute of Technology Tiruchirappalli620015 India Machine Learning and Data Analytics Lab Department of Computer Applications National Institute of Technology Tiruchirappalli620015 India Department of Computer Science and Engineering VFSTR Guntur522213 India

Driver drowsiness is one of the reasons for large number of road accidents these days. With the advancement in Computer Vision technologies, smart/intelligent cameras are developed to identify drowsiness in drivers, thereby alerting drivers which in turn reduce accidents when they are in fatigue. In this work, a new framework is proposed using deep learning to detect driver drowsiness based on Eye state while driving the vehicle. To detect the face and extract the eye region from the face images, Viola-Jones face detection algorithm is used in this work. Stacked deep convolution neural network is developed to extract features from dynamically identified key frames from camera sequences and used for learning phase. A SoftMax layer in CNN classifier is used to classify the driver as sleep or non-sleep. This system alerts driver with an alarm when the driver is in sleepy mood. The proposed work is evaluated on a collected dataset and shows better accuracy with 96.42% when compared with traditional CNN. The limitation of traditional CNN such as pose accuracy in regression is overcome with the proposed Staked Deep CNN. © 2019 Lavoisier. All rights reserved.

关键词： Face recognition