检索结果-内蒙古大学图书馆

Sparse high-dimensional linear mixed modeling with a partitioned empirical Bayes ECM algorithm

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zgodic, Anja Bai, Ray Zhang, Jiajia Olejua, Peter McLain, Alexander C. Department of Epidemiology and Biostatistics University of South Carolina United States Department of Statistics University of South Carolina United States

High-dimensional longitudinal data is increasingly used in a wide range of scientific studies. To properly account for dependence between longitudinal observations, statistical methods for high-dimensional linear mixed models (LMMs) have been developed. However, few packages implementing these high-dimensional LMMs are available in the statistical software R. Additionally, some packages suffer from scalability issues. This work presents an efficient and accurate Bayesian framework for high-dimensional LMMs. We use empirical Bayes estimators of hyperparameters for increased flexibility and an expectation-Conditional-Minimization (ECM) algorithm for computationally efficient maximum a posteriori probability (MAP) estimation of parameters. The novelty of the approach lies in its partitioning and parameter expansion as well as its fast and scalable computation. We illustrate Linear Mixed Modeling with PaRtitiOned empirical Bayes ECM (LMM-PROBE) in simulation studies evaluating fixed and random effects estimation along with computation time. A real-world example is provided using data from a study of lupus in children, where we identify genes and clinical factors associated with a new lupus biomarker and predict the biomarker over time. Supplementary materials are available online. Copyright © 2023, The Authors. All rights reserved.

关键词： expectation maximization algorithm

Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zgodic, Anja Bai, Ray Zhang, Jiajia Wang, Yuan Rorden, Chris McLain, Alexander C. Department of Epidemiology and Biostatistics University of South Carolina South Carolina United States Department of Statistics University of South Carolina South Carolina United States Department of Psychology University of South Carolina South Carolina United States

Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. For example, Aphasia Quotient (AQ) is a critical measure of language impairment and informs treatment decisions, but it is challenging to measure in stroke patients. It is of interest to use high-resolution T2 neuroimages of brain damage to predict AQ. However, sparse regression models show marked evidence of heteroscedastic error even after transformations are applied. This violation of the homoscedasticity assumption can lead to bias in estimated coefficients, prediction intervals (PI) with improper length, and increased type I errors. Bayesian heteroscedastic linear regression models relax the homoscedastic error assumption but can enforce restrictive prior assumptions on parameters, and many are computationally infeasible in the high-dimensional setting. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes expectation Conditional maximization (H-PROBE) algorithm. H-PROBE is a computationally efficient maximum a posteriori estimation approach that requires minimal prior assumptions and can incorporate covariates hypothesized to impact heterogeneity. We apply this method by using high-dimensional neuroimages to predict and provide PIs for AQ that accurately quantify predictive uncertainty. Our analysis demonstrates that H-PROBE can provide narrower PI widths than standard methods without sacrificing coverage. Narrower PIs are clinically important for determining the risk of moderate to severe aphasia. Additionally, through extensive simulation studies, we exhibit that H-PROBE results in superior prediction, variable selection, and predictive inference compared to alternative methods. Copyright © 2023, The Authors. All rights reserved.

关键词： expectation maximization algorithm

The Principle of Uncertain Maximum Entropy

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Bogert, Kenneth Kothe, Matthew Department of Computer Science University of North Carolina Asheville 1 University Heights AshevilleNC28801 United States

The principle of maximum entropy is a well-established technique for choosing a distribution that matches available information while minimizing bias. It finds broad use across scientific disciplines and in machine learning. However, the principle as defined by Jaynes (1957) is susceptible to noise and error in observations. This forces real-world practitioners to use relaxed versions of the principle in an ad hoc way, negatively impacting interpretation. To address this situation, we present a new principle we call uncertain maximum entropy that generalizes the classic principle and provides interpretable solutions irrespective of the observational methods in use. We introduce a convex approximation and expectation-maximization based algorithm for finding solutions to our new principle. Finally, we contrast this new technique with two simpler generally applicable solutions theoretically and experimentally show our technique provides superior accuracy. © 2023, CC BY.

关键词： expectation maximization algorithm

Extended target tracking with mobility based on GPR-AUKF

学校读者我要写书评

暂无评论

Heliyon 2024年第23期10卷 e40506页

作者： Zhang, Renli Zhang, Yan Chen, Jintao Sun, Ziwen Li, Jing Tan, Zhuangbin Jiao, Zhongxing School of Aeronautics and Astronautics Sun Yat-sen University Shenzhen 518107 China School of Electronics and Communication Engineering Sun Yat-sen University Shenzhen 518107 China

Simultaneously estimating the kinematic state and extent of extended targets is a nonlinear and high-dimensional problem. While the extended Kalman filter (EKF) is widely employed to achieve this goal, it may not be sufficient for mobility targets. To address this issue, this paper first proposes to embed unscented Kalman filter (UKF) into Gaussian process regression (GPR) since the superiority of UKF to high nonlinear. Furthermore, given the widely-existed environment with time-varying noise, it is crucial to study the change of measurement noise covariance caused by time-varying noise for high-precision tracking of extended targets. However, traditional UKF filter considers measurement noise covariance as constant value. To this end, an adaptive unscented Kalman filter (AUKF) algorithm combining with GPR model (GPR-AUKF) is proposed to address the issue. Specifically, the GPR-AUKF algorithm is built based on expectation maximization (EM) algorithm to track the target state and covariance, and which updates the measurement noise covariance in real-time. Experimental results show that GPR-AUKF is more accurate and robust than other methods for tracking extended targets. © 2024

关键词： Adaptive unscented Kalman filter expectation maximization algorithm Extended target Gaussian process regression Time-varying noise

Causal Temporal Regime Structure Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Rahmani, Abdellah Frossard, Pascal LTS4 EPFL Lausanne Switzerland

Understanding causal relationships in multivariate time series is essential for predicting and controlling dynamic systems in fields like economics, neuroscience, and climate science. However, existing causal discovery methods often assume stationarity, limiting their effectiveness when time series consist of sequential regimes, consecutive temporal segments with unknown boundaries and changing causal structures. In this work, we firstly introduce a framework to describe and model such time series. Then, we present CASTOR, a novel method that concurrently learns the Directed Acyclic Graph (DAG) for each regime while determining the number of regimes and their sequential arrangement. CASTOR optimizes the data log-likelihood using an expectation-maximization algorithm, alternating between assigning regime indices (expectation step) and inferring causal relationships in each regime (maximization step). We establish the identifiability of the regimes and DAGs within our framework. Extensive experiments show that CASTOR consistently outperforms existing causal discovery models in detecting different regimes and learning their DAGs across various settings, including linear and nonlinear causal relationships, on both synthetic and real world datasets. © 2023, CC BY.

关键词： expectation maximization algorithm

Achieving Margin maximization Exponentially Fast via Progressive Norm Rescaling

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Mingze Min, Zeping Wu, Lei School of Mathematical Sciences Peking University Beijing China Center for Machine Learning Research Peking University Beijing China

In this work, we investigate the margin-maximization bias exhibited by gradient-based algorithms in classifying linearly separable data. We present an in-depth analysis of the specific properties of the velocity field associated with (normalized) gradients, focusing on their role in margin maximization. Inspired by this analysis, we propose a novel algorithm called Progressive Rescaling Gradient Descent (PRGD) and show that PRGD can maximize the margin at an exponential rate. This stands in stark contrast to all existing algorithms, which maximize the margin at a slow polynomial rate. Specifically, we identify mild conditions on data distribution under which existing algorithms such as gradient descent (GD) and normalized gradient descent (NGD) provably fail in maximizing the margin efficiently. To validate our theoretical findings, we present both synthetic and real-world experiments. Notably, PRGD also shows promise in enhancing the generalization performance when applied to linearly non-separable datasets and deep neural networks. Copyright © 2023, The Authors. All rights reserved.

关键词： expectation maximization algorithm

Gibbs optimal design of experiments

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Overstall, Antony M. Holloway-Brown, Jacinta McGree, James M. School of Mathematical Sciences University of Southampton SouthamptonSO17 1BJ United Kingdom School of Computer and Mathematical Sciences University of Adelaide Adelaide5005 Australia School of Mathematical Sciences Queensland University of Technology Brisbane4001 Australia

Bayesian optimal design is a well-established approach to planning experiments. A distribution for the responses, i.e. a statistical model, is assumed which is dependent on unknown parameters. A utility function is then specified giving gain in information in estimating the true values of the parameters, using the Bayesian posterior distribution. A Bayesian optimal design is given by maximising expectation of the utility with respect to the distribution implied by statistical model and prior distribution for the true parameter values. The approach accounts for the experimental aim, via specification of the utility, and of assumed sources of uncertainty. However, it is predicated on the statistical model being correct. Recently, a new type of statistical inference, known as Gibbs inference, has been proposed. This is Bayesian-like, i.e. uncertainty for unknown quantities is represented by a posterior distribution, but does not necessarily require specification of a statistical model. The resulting inference is less sensitive to misspecification of the statistical model. This paper introduces Gibbs optimal design: a framework for optimal design of experiments under Gibbs inference. A computational approach to find designs in practice is outlined and the framework is demonstrated on exemplars including linear models, and experiments with count and time-to-event responses. © 2023, CC BY.

关键词： expectation maximization algorithm

Prophet Inequalities: Separating Random Order from Order Selection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Giambartolomei, Giordano Mallmann-Trenn, Frederik Saona, Raimundo King’s College London United Kingdom Institute of Science and Technology Austria

Prophet inequalities are a central object of study in optimal stopping theory. A gambler is sent values in an online fashion, sampled from an instance of independent distributions, in an adversarial, random or selected order, depending on the model. When observing each value, the gambler either accepts it as a reward or irrevocably rejects it and proceeds to observe the next value. The goal of the gambler, who cannot see the future, is maximising the expected value of the reward while competing against the expectation of a prophet (the offline maximum). In other words, one seeks to maximise the gambler-to-prophet ratio of the expectations. The model, in which the gambler selects the arrival order first, and then observes the values, is known as Order Selection. In this model a ratio of 0.7251 is attainable for any instance. Recently, this has been improved up to 0.7258 by Bubna and Chiplunkar (2023). If the gambler chooses the arrival order (uniformly) at random, we obtain the Random Order model. The worst case ratio over all possible instances has been extensively studied for at least 40 years. In a computer-assisted proof, Bubna and Chiplunkar (2023) also showed that this ratio is at most 0.7254 for the Random Order model, thus establishing for the first time that carefully choosing the order, instead of simply taking it at random, benefits the gambler. We give an alternative, non-simulation-assisted proof of this fact, by showing mathematically that in the Random Order model, no algorithm can achieve a ratio larger than 0.7235. This sets a new state-of-the-art hardness for this model, and establishes more formally that there is a real benefit in choosing the order. Copyright © 2023, The Authors. All rights reserved.

关键词： expectation maximization algorithm

Outlier-Insensitive Kalman Filtering: Theory and Applications

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Truzman, Shunit Revach, Guy Shlezinger, Nir Klein, Itzik The Hatter Dept. of Marine Technologies University of Haifa Haifa Israel D-ITET ETH Zürich Switzerland The School of ECE Ben-Gurion University of the Negev Be’er Sheva Israel

State estimation of dynamical systems from noisy observations is a fundamental task in many applications. It is commonly addressed using the linear Kalman filter (KF), whose performance can significantly degrade in the presence of outliers in the observations, due to the sensitivity of its convex quadratic objective function. To mitigate such behavior, outlier detection algorithms can be applied. In this work, we propose a parameter-free algorithm which mitigates the harmful effect of outliers while requiring only a short iterative process of the standard KF’s update step. To that end, we model each potential outlier as a normal process with unknown variance and apply online estimation through either expectation maximization or alternating maximization algorithms. Simulations and field experiment evaluations demonstrate our method’s competitive performance, showcasing its robustness to outliers in filtering scenarios compared to alternative algorithms. Copyright © 2023, The Authors. All rights reserved.

关键词： expectation maximization algorithm