Scalable Gaussian-process regression and variable selection using Vecchia approximations

作     者：Cao, Jian Guinness, Joseph Genton, Marc G. Katzfuss, Matthias 

作者机构：Department of Statistics Institute of Data Science Texas A&M University United States Department of Statistics Cornell University United States Statistics Program King Abdullah University of Science and Technology Saudi Arabia 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2022年

核心收录：

主　　题：Gaussian noise (electronic) 

摘      要：Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to select covariates that are related to the response. For this setting, we propose a novel, scalable algorithm, coined VGPR, which optimizes a penalized GP log-likelihood based on the Vecchia GP approximation, an ordered conditional approximation from spatial statistics that implies a sparse Cholesky factor of the precision matrix. We traverse the regularization path from strong to weak penalization, sequentially adding candidate covariates based on the gradient of the log-likelihood and deselecting irrelevant covariates via a new quadratic constrained coordinate descent algorithm. We propose Vecchia-based mini-batch subsampling, which provides unbiased gradient estimators. The resulting procedure is scalable to millions of responses and thousands of covariates. Theoretical analysis and numerical studies demonstrate the improved scalability and accuracy relative to existing methods. Copyright © 2022, The Authors. All rights reserved.