检索结果-内蒙古大学图书馆

Incremental intervention effects in studies with dropout and many timepoints

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Kim, Kwangho Kennedy, Edward H. Naimi, Ashley I. Department of Statistics & Data Science Machine Learning Department Carnegie Mellon University 5000 Forbes Ave PittsburghPA15213 United States Department of Statistics & Data Science Carnegie Mellon University 5000 Forbes Ave PittsburghPA15213 United States Department of Epidemiology Rollins School of Public Health Emory University AtlantaGA United States

Modern longitudinal studies collect feature data at many timepoints, often of the same order of sample size. Such studies are typically affected by dropout and positivity violations. We tackle these problems by generalizing effects of recent incremental interventions (which shift propensity scores rather than set treatment values deterministically) to accommodate multiple outcomes and subject dropout. We give an identifying expression for incremental intervention effects when dropout is conditionally ignorable (without requiring treatment positivity), and derive the nonparametric efficiency bound for estimating such effects. Then we present efficient nonparametric estimators, showing that they converge at fast parametric rates and yield uniform inferential guarantees, even when nuisance functions are estimated flexibly at slower rates. We also study the variance ratio of incremental intervention effects relative to more conventional deterministic effects in a novel infinite time horizon setting, where the number of timepoints can grow with sample size, and show that incremental intervention effects yield near-exponential gains in statistical precision in this setup. Finally we conclude with simulations and apply our methods in a study of the effect of low-dose aspirin on pregnancy *** Codes 62G05 Copyright © 2019, The Authors. All rights reserved.

关键词：

L1 Trend Filtering: A Modern Statistical Tool for Time-Domain Astronomy and Astronomical Spectroscopy

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Politsch, Collin A. Cisewski-Kehe, Jessi Croft, Rupert A.C. Wasserman, Larry Department of Statistics & Data Science Carnegie Mellon University PittsburghPA15213 Machine Learning Department Carnegie Mellon University PittsburghPA15213 Department of Statistics and Data Science Yale University New HavenCT06520 Department of Physics Carnegie Mellon University PittsburghPA15213 McWilliams Center for Cosmology Carnegie Mellon University PittsburghPA15213

The problem of estimating a one-dimensional signal possessing mixed degrees of smoothness is ubiquitous in time-domain astronomy and astronomical spectroscopy. For example, in the time domain, an astronomical object may exhibit a smoothly varying intensity that is occasionally interrupted by abrupt dips or spikes. Likewise, in the spectroscopic setting, a noiseless spectrum typically contains intervals of relative smoothness mixed with localized higher frequency components such as emission peaks and absorption lines. In this work, we present L1 trend filtering (Steidl et al.;Kim et al.), a modern nonparametric statistical tool that yields significant improvements in this broad problem space of estimating spatially heterogeneous signals. When the underlying signal is spatially heterogeneous, the L1 trend filter has been shown to be strictly superior to any estimator that is a linear combination of the observed data, including smoothing splines, kernels, and Gaussian processes. Moreover, the L1 trend filter does not require the restrictive setup of wavelets — the definitive classical approach for modeling spatially heterogeneous signals. In the spirit of illustrating the wide applicability of L1 trend filtering, we briefly demonstrate its utility on several relevant astrophysical data sets: two Kepler light curves (an exoplanet transit and an eclipsing binary system), a Palomar Transient Factory supernova light curve, and an SDSS galaxy spectrum. Furthermore, we present a more rigorous analysis of the Lyman-alpha forest of SDSS quasar spectra — a standard cosmological tool for probing the large-scale structure of the high redshift intergalactic medium. Copyright © 2019, The Authors. All rights reserved.

关键词： Cosmology

Optimization of smooth functions with noisy observations: Local minimax rates

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Wang, Yining Balakrishnan, Sivaraman Singh, Aarti Machine Learning Department Carnegie Mellon University Department of Statistics and Data Science Carnegie Mellon University

We consider the problem of global optimization of an unknown non-convex smooth function with zeroth-order feedback. In this setup, an algorithm is allowed to adaptively query the underlying function at different locations and receives noisy evaluations of function values at the queried points (i.e. the algorithm has access to zeroth-order information). Optimization performance is evaluated by the expected difference of function values at the estimated optimum and the true optimum. In contrast to the classical optimization setup, first-order information like gradients are not directly accessible to the optimization algorithm. We show that the classical minimax framework of analysis, which roughly characterizes the worst-case query complexity of an optimization algorithm in this setting, leads to excessively pessimistic results. We propose a local minimax framework to study the fundamental difficulty of optimizing smooth functions with adaptive function evaluations, which provides a refined picture of the intrinsic difficulty of zeroth-order optimization. We show that for functions with fast level set growth around the global minimum, carefully designed optimization algorithms can identify a near global minimizer with many fewer queries. For the special case of strongly convex and smooth functions, our implied convergence rates match the ones developed for zeroth-order convex optimization problems [1, 22]. At the other end of the spectrum, for worst-case smooth functions no algorithm can converge faster than the minimax rate of estimating the entire unknown function in the 8-norm. We provide an intuitive and efficient algorithm that attains the derived upper error bounds. Finally, using the local minimax framework we are able to clearly dichotomize adaptive and non-adaptive algorithms by showing that non-adaptive algorithms, although optimal in a global minimax sense, do not attain the optimal local minimax rate. Copyright © 2018, The Authors. All rights reserved.

关键词： Function evaluation

Augmenting Adjusted Plus-Minus in Soccer with FIFA Ratings

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Matano, Francesca Richardson, Lee F. Pospisil, Taylor Eubanks, Collin Qin, Jining Department of Statistics and Data Science Carnegie Mellon University Machine Learning Department Carnegie Mellon University

In basketball and hockey, state-of-the-art player value statistics are often variants of Adjusted Plus-Minus (APM). But APM hasn’t had the same impact in soccer, since soccer games are low scoring with a low number of substitutions. In soccer, perhaps the most comprehensive player value statistics come from video games, and in particular FIFA. FIFA ratings combine the subjective evaluations of over 9000 scouts, coaches, and season-ticket holders into ratings for over 18,000 players. This paper combines FIFA ratings and APM into a single metric, which we call Augmented APM. The key idea is recasting APM into a Bayesian framework, and incorporating FIFA ratings into the prior distribution. We show that Augmented APM predicts better than both standard APM and a model using only FIFA ratings. We also show that Augmented APM decorrelates players that are highly collinear. Copyright © 2018, The Authors. All rights reserved.

关键词： Sports

Cautious deep learning

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Hechtlinger, Yotam Póczos, Barnabás Wasserman, Larry Department of Statistics and Data Science Carnegie Mellon University Machine Learning Department Carnegie Mellon University

Most classifiers operate by selecting the maximum of an estimate of the conditional distribution p(yjx) where x stands for the features of the instance to be classified and y denotes its label. This often results in a hubristic bias: Overconfidence in the assignment of a definite label. Usually, the observations are concentrated on a small volume but the classifier provides definite predictions for the entire space. We propose constructing conformal prediction sets (Vovk et al., 2005) which contain a set of labels rather than a single label. These conformal prediction sets contain the true label with probability 1-α . Our construction is based on p(xjy) rather than p(yjx) which results in a classifier that is very cautious: It outputs the null set-meaning "I don't know"-when the object does not resemble the training examples. An important property of our approach is that adversarial attacks are likely to be predicted as the null set or would also include the true label. We demonstrate the performance on the ImageNet ILSVRC dataset and the CelebA and IMDB-Wiki facial datasets using high dimensional features obtained from state of the art convolutional neural networks. Copyright © 2018, The Authors. All rights reserved.

关键词： Neural networks

Analysis of a mode clustering diagram

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Verdinelli, Isabella Wasserman, Larry Department of Statistics and Data Science Carnegie Mellon University Machine Learning Department Carnegie Mellon University

Mode-based clustering methods define clusters to be the basins of attraction of the modes of a density estimate. The most common version is mean shift clustering which uses a gradient ascent algorithm to find the basins. Rodriguez and Laio (2014) introduced a new method that is faster and simpler than mean shift clustering. Furthermore, they define a clustering diagram that provides a simple, two-dimensional summary of the mode clustering information. We study the statistical properties of this diagram and we propose some improvements and extensions. In particular, we show a connection between the diagram and robust linear regression. Copyright © 2018, The Authors. All rights reserved.

关键词：

Cosmological N-body simulations: A challenge for scalable generative models

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Perraudin, Nathanaël Srivastava, Ankit Lucchi, Aurelien Kacprzak, Tomasz Hofmann, Thomas Réfrégier, Alexandre Swiss Data Science Center ETH Zurich Universitätstrasse 25 Zurich8006 Switzerland Institute for Particle Physics and Astrophysics ETH Zurich Wolfgang-Pauli-Str. 27 Zurich8093 Switzerland Institute for Machine Learning ETH Zurich Universitätstrasse 6 Zurich8006 Switzerland

Deep generative models, such as Generative Adversarial Networks (GANs) or Variational Autoencoders (VAs) have been demonstrated to produce images of high visual quality. However, the existing hardware on which these models are trained severely limits the size of the images that can be generated. The rapid growth of high dimensional data in many fields of science therefore poses a significant challenge for generative models. In cosmology, the large-scale, three-dimensional matter distribution, modeled with N-body simulations, plays a crucial role in understanding of evolution of structures in the universe. As these simulations are computationally very expensive, GANs have recently generated interest as a possible method to emulate these datasets, but they have been, so far, mostly limited to two dimensional data. In this work, we introduce a new benchmark for the generation of three dimensional N-body simulations, in order to stimulate new ideas in the machine learning community and move closer to the practical use of generative models in cosmology. As a first benchmark result, we propose a scalable GAN approach for training a generator of N-body three-dimensional cubes. Our technique relies on two key building blocks, (i) splitting the generation of the high-dimensional data into smaller parts, and (ii) using a multi-scale approach that efficiently captures global image features that might otherwise be lost in the splitting process. We evaluate the performance of our model for the generation of N-body samples using various statistical measures commonly used in cosmology. Our results show that the proposed model produces samples of high visual quality, although the statistical analysis reveals that capturing rare features in the data poses significant problems for the generative models. We make the data, quality evaluation routines, and the proposed GAN architecture publicly available at https://***/nperraud/3DcosmoGAN. Copyright © 2019, The Authors. All right

关键词： Cosmology

Nonparametric density estimation with adversarial losses

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Singh, Shashank Uppal, Ananya Li, Boyue Li, Chun-Liang Zaheer, Manzil Póczos, Barnabás Machine Learning Department Department of Statistics and Data Science Department of Mathematical Sciences Language Technologies Institute Carnegie Mellon University

We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical Lp losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the losses encoded by discriminator networks in generative adversarial networks (GANs). In a general framework, we study how the choice of loss and the assumed smoothness of the underlying density together determine the minimax rate. We also discuss implications for training GANs based on deep ReLU networks, and more general connections to learning implicit generative models in a minimax statistical sense. Copyright © 2018, The Authors. All rights reserved.

关键词： Generative adversarial networks

Realization of spatial sparseness by deep ReLU nets with massive data

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Chui, Charles K. Lin, Shao-Bo Zhang, Bo Zhou, Ding-Xuan Department of Mathematics Hong Kong Baptist University Department of Statistics Stanford University CA94305 United States Center of Intelligent Decision-making and Machine Learning School of Management Xi'an Jiaotong University Xi'an China School of Data Science Department of Mathematics City University of Hong Kong Hong Kong

The great success of deep learning poses urgent challenges for understanding its working mechanism and rationality. The depth, structure, and massive size of the data are recognized to be three key ingredients for deep learning. Most of the recent theoretical studies for deep learning focus on the necessity and advantages of depth and structures of neural networks. In this paper, we aim at rigorous verification of the importance of massive data in embodying the out-performance of deep learning. To approximate and learn spatially sparse and smooth functions, we establish a novel sampling theorem in learning theory to show the necessity of massive data. We then prove that implementing the classical empirical risk minimization on some deep nets facilitates in realization of the optimal learning rates derived in the sampling theorem. This perhaps explains why deep learning performs so well in the era of big data. Copyright © 2019, The Authors. All rights reserved.

关键词： Metadata