检索结果-内蒙古大学图书馆

Network-based Neighborhood Regression

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhen, Yaoming Du, Jin-Hong Department of Statistical Sciences University of Toronto TorontoONM5G 1X6 Canada Department of Statistics and Data Science Carnegie Mellon University PittsburghPA15213 United States Machine Learning Department Carnegie Mellon University PittsburghPA15213 United States

Given the ubiquity of modularity in biological systems, module-level regulation analysis is vital for understanding biological systems across various levels and their dynamics. Current statistical analysis on biological modules predominantly focuses on either detecting the functional modules in biological networks or sub-group regression on the biological features without using the network data. This paper proposes a novel network-based neighborhood regression framework whose regression functions depend on both the global community-level information and local connectivity structures among entities. An efficient community-wise least square optimization approach is developed to uncover the strength of regulation among the network modules while enabling asymptotic inference. With random graph theory, we derive non-asymptotic estimation error bounds for the proposed estimator, achieving exact minimax optimality. Unlike the root-n consistency typical in canonical linear regression, our model exhibits linear consistency in the number of nodes n, highlighting the advantage of incorporating neighborhood information. The effectiveness of the proposed framework is further supported by extensive numerical experiments. Application to whole-exome sequencing and RNA-sequencing Autism datasets demonstrates the usage of the proposed method in identifying the association between the gene modules of genetic variations and the gene modules of genomic differential expressions. Copyright © 2024, The Authors. All rights reserved.

关键词： Linear regression

Precise Asymptotics of Bagging Regularized M-estimators

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Koriyama, Takuya Patil, Pratik Du, Jin-Hong Tan, Kai Bellec, Pierre C. Booth School of Business The University of Chicago ChicagoIL60637 United States Department of Statistics University of California BerkeleyCA94720 United States Department of Statistics and Data Science Carnegie Mellon University PittsburghPA15213 United States Machine Learning Department Carnegie Mellon University PittsburghPA15213 United States Department of Statistics Rutgers University New BrunswickNJ08854 United States

We characterize the squared prediction risk of ensemble estimators obtained through subagging (subsample bootstrap aggregating) regularized M-estimators and construct a consistent estimator for the risk. Specifically, we consider a heterogeneous collection of M ≥ 1 regularized M-estimators, each trained with (possibly different) subsample sizes, convex differentiable losses, and convex regularizers. We operate under the proportional asymptotics regime, where the sample size n, feature size p, and subsample sizes km for m ∈ [M] all diverge with fixed limiting ratios n/p and km/n. Key to our analysis is a new result on the joint asymptotic behavior of correlations between the estimator and residual errors on overlapping subsamples, governed through a (provably) contractible nonlinear system of equations. Of independent interest, we also establish convergence of trace functionals related to degrees of freedom in the non-ensemble setting (with M = 1) along the way, extending previously known cases for square loss and ridge, lasso regularizers. When specialized to homogeneous ensembles trained with a common loss, regularizer, and subsample size, the risk characterization sheds some light on the implicit regularization effect due to the ensemble and subsample sizes (M, k). For any ensemble size M, optimally tuning subsample size yields sample-wise monotonic risk. For the full-ensemble estimator (when M → ∞), the optimal subsample size k* tends to be in the overparameterized regime (k* ≤ min{n, p}), when explicit regularization is vanishing. Finally, joint optimization of subsample size, ensemble size, and regularization can significantly outperform regularizer optimization alone on the full data (without any subagging). Copyright © 2024, The Authors. All rights reserved.

关键词： Nonlinear equations

The Benefits of Mixup for Feature learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zou, Difan Cao, Yuan Li, Yuanzhi Gu, Quanquan Department of Computer Science Institute of Data Science The University of Hong Kong Hong Kong Department of Statistics and Actuarial Science Department of Mathematics The University of Hong Kong Hong Kong Machine Learning Department Carnegie Mellon University PittsburghPA United States Department of Computer Science University of California Los AngelesCA United States

Mixup, a simple data augmentation method that randomly mixes two data points via linear interpolation, has been extensively applied in various deep learning applications to gain better generalization. However, the theoretical underpinnings of its efficacy are not yet fully understood. In this paper, we aim to seek a fundamental understanding of the benefits of Mixup. We first show that Mixup using different linear interpolation parameters for features and labels can still achieve similar performance to the standard Mixup. This indicates that the intuitive linearity explanation in Zhang et al. (2018) may not fully explain the success of Mixup. Then we perform a theoretical study of Mixup from the feature learning perspective. We consider a feature-noise data model and show that Mixup training can effectively learn the rare features (appearing in a small fraction of data) from its mixture with the common features (appearing in a large fraction of data). In contrast, standard training can only learn the common features but fails to learn the rare features, thus suffering from bad generalization performance. Moreover, our theoretical analysis also shows that the benefits of Mixup for feature learning are mostly gained in the early training phase, based on which we propose to apply early stopping in Mixup. Experimental results verify our theoretical findings and demonstrate the effectiveness of the early-stopped Mixup training. © 2023, CC BY-SA.

关键词： Deep learning

The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cao, Yuan Zou, Difan Li, Yuanzhi Gu, Quanquan Department of Statistics and Actuarial Science Department of Mathematics The University of Hong Kong Hong Kong Department of Computer Science Institute of Data Science The University of Hong Kong Hong Kong Machine Learning Department Carnegie Mellon University PittsburghPA United States Department of Computer Science University of California Los AngelesCA United States

We study the implicit bias of batch normalization trained by gradient descent. We show that when learning a linear model with batch normalization for binary classification, gradient descent converges to a uniform margin classifier on the training data with an exp(−Ω(log2 t)) convergence rate. This distinguishes linear models with batch normalization from those without batch normalization in terms of both the type of implicit bias and the convergence rate. We further extend our result to a class of two-layer, single-filter linear convolutional neural networks, and show that batch normalization has an implicit bias towards a patch-wise uniform margin. Based on two examples, we demonstrate that patch-wise uniform margin classifiers can outperform the maximum margin classifiers in certain learning problems. Our results contribute to a better theoretical understanding of batch normalization. © 2023, CC BY-SA.

关键词： Convolution

Nonlinear Regression with Residuals: Causal Estimation with Time-varying Treatments and Covariates

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Bates, Stephen Kennedy, Edward Tibshirani, Robert Ventura, Valérie Wasserman, Larry Departments of EECS and Statistics University of California Berkeley United States Department of Statistics & Data Science Carnegie Mellon University United States Departments of Biomedical Data Science and Statistics Stanford University United States Department of Statistics Data Science and Neuroscience Institute Carnegie Mellon University United States Departments of Statistics & Data Science and of Machine Learning Carnegie Mellon University United States

Standard regression adjustment gives inconsistent estimates of causal effects when there are time-varying treatment effects and time-varying covariates. Loosely speaking, the issue is that some covariates are post-treatment variables because they may be affected by prior treatment status, and regressing out post-treatment variables causes bias. More precisely, the bias is due to certain non-confounding latent variables that create colliders in the causal graph. These latent variables, which we call phantoms, do not harm the identifiability of the causal effect, but they render naive regression estimates inconsistent. Motivated by this, we ask: how can we modify regression methods so that they hold up even in the presence of phantoms? We develop an estimator for this setting based on regression modeling (linear, log-linear, probit and Cox regression), proving that it is consistent for a reasonable causal estimand. In particular, the estimator is a regression model fit with a simple adjustment for collinearity, making it easy to understand and implement with standard regression software. The proposed estimators are instances of the parametric g-formula, extending the regression-with-residuals approach to several canonical nonlinear models. © 2022, CC BY.

关键词： Phantoms

Bounding the number of reticulation events for displaying multiple trees in a phylogenetic network

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Yufeng Zhang, Louxin School of Computing University of Connecticut StorrsCT06269 United States Department of Mathematics Center for Data Science and Machine Learning National University of Singapore Singapore119076 Singapore

Reconstructing a parsimonious phylogenetic network that displays multiple phylogenetic trees is an important problem in phylogenetics, where the complexity of the inferred networks is measured by reticulation numbers. The reticulation number for a set of trees is defined as the minimum number of reticulations in a phylogenetic network that displays those trees. A mathematical problem is bounding the reticulation number for multiple trees over a fixed number of taxa. While this problem has been extensively studied for two trees, much less is known about the upper bounds on the reticulation numbers for three or more arbitrary trees. In this paper, we present a few non-trivial upper bounds on reticulation numbers for three or more *** Codes 5C30 © 2024, CC BY.

关键词： Abiotic

A Minimax Optimal Control Approach for Robust Neural ODEs

学校读者我要写书评

暂无评论

A Minimax Optimal Control Approach for Robust Neural ODEs

European Control Conference (ECC)

作者： Cristina Cipriani Alessandro Scagliotti Tobias Wöhrer Department of Mathematics Technical University Munich (TUM) Munich Germany Munich Data Science Institute (MDSI) Munich Germany Munich Center for Machine Learning (MCML) Munich Germany

ISBN: (数字)9783907144107

ISBN: (纸本)9798331540920

In this paper, we address the adversarial training of neural ODEs from a robust control perspective. This is an alternative to the classical training via empirical risk minimization, and it is widely used to enforce reliable outcomes for input perturbations. Neural ODEs allow the interpretation of deep neural networks as discretizations of control systems, unlocking powerful tools from control theory for the development and the understanding of machine learning. In this specific case, we formulate the adversarial training with perturbed data as a minimax optimal control problem, for which we derive first order optimality conditions in the form of Pontryagin's Maximum Principle. We provide a novel interpretation of robust training leading to an alternative weighted technique, which we test on a low-dimensional classification task.

关键词： Training Weight measurement Robust control Risk minimization Perturbation methods Optimal control Artificial neural networks

Closed-form solutions for Bernoulli and compound Poisson branching processes in random environments

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kutsenko, Anton A. University of Hamburg MIN Faculty Department of Mathematics Hamburg20146 Germany Mathematical Institute for Machine Learning and Data Science Katholische Universität Eichstätt Ingolstadt Germany

For branching processes, the generating functions for limit distributions of so-called ratios of probabilities of rare events satisfy the Schröder-type integral-functional equations. Excepting limited special cases, the corresponding equations can not be solved analytically. I found a large class of Poisson-type offspring distributions, for which the Schröder-type functional equations can be solved analytically. Moreover, for the asymptotics of limit distributions, the power and constant factor can be written explicitly. As a bonus, Bernoulli branching processes in random environments are treated. The beauty of this example is that the explicit formula for the generating function is unknown. Still, the closed-form expression for the power and constant factor in the asymptotic can be written with the help of some outstanding tricks. Also, the asymptotic expansion contains oscillatory terms absent in the Poisson case. It is shown that at the power 3 in the Bernoulli binomial kernels short-phase discrete oscillations turn into long-phase continuous ones. Copyright © 2024, The Authors. All rights reserved.

关键词： Poisson distribution

Existence of Direct Density Ratio Estimators

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Banzato, Erika Drton, Mathias Saraf-Poor, Kian Shi, Hongjian Department of Statistical Sciences University of Padova Italy TUM School of Computation Information and Technology Munich Data Science Institute Technical University of Munich Munich Center for Machine Learning Germany Department of Statistics Columbia University United States TUM School of Computation Information and Technology Technical University of Munich Germany

Many two-sample problems call for a comparison of two distributions from an exponential family. Density ratio estimation methods provide ways to solve such problems through direct estimation of the differences in natural parameters. The term direct indicates that one avoids estimating both marginal distributions. In this context, we consider the Kullback–Leibler Importance Estimation Procedure (KLIEP), which has been the subject of recent work on differential networks. Our main result shows that the existence of the KLIEP estimator is characterized by whether the average sufficient statistic for one sample belongs to the convex hull of the set of all sufficient statistics for data points in the second sample. For high-dimensional problems it is customary to regularize the KLIEP loss by adding the product of a tuning parameter and a norm of the vector of parameter differences. We show that the existence of the regularized KLIEP estimator requires the tuning parameter to be no less than the dual norm-based distance between the average sufficient statistic and the convex hull. The implications of these existence issues are explored in applications to differential network analysis. Copyright © 2025, The Authors. All rights reserved.

关键词： Parameter estimation