检索结果-内蒙古大学图书馆

SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Du, Chaoqun Han, Yizeng Huang, Gao Department of Automation BNRist Tsinghua University Beijing China

Recent advancements in semi-supervised learning have focused on a more realistic yet challenging task: addressing imbalances in labeled data while the class distribution of unlabeled data remains both unknown and potentially mismatched. Current approaches in this sphere often presuppose rigid assumptions regarding the class distribution of unlabeled data, thereby limiting the adaptability of models to only certain distribution ranges. In this study, we propose a novel approach, introducing a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization (EM) algorithm by explicitly decoupling the modeling of conditional and marginal class distributions. This separation facilitates a closed-form solution for class distribution estimation during the maximization phase, leading to the formulation of a Bayes classifier. The Bayes classifier, in turn, enhances the quality of pseudo-labels in the expectation phase. Remarkably, the SimPro framework not only comes with theoretical guarantees but also is straightforward to implement. Moreover, we introduce two novel class distributions broadening the scope of the evaluation. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios. Our code is available at https://***/ LeapLabTHU/SimPro. Copyright © 2024, The Authors. All rights reserved.

关键词： expectation maximization algorithm

Policy Choice in Time Series by Empirical Welfare maximization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Kitagawa, Toru Wang, Weining Xu, Mengshan Department of Economics Brown University United States Department of Economics University College London United Kingdom Faculty of economics and business University of Groningen Netherlands Department of Economics University of Mannheim Germany

This paper develops a novel method for policy choice in a dynamic setting where the available data is a multi-variate time series. Building on the statistical treatment choice framework, we propose Time-series Empirical Welfare maximization (T-EWM) methods to estimate an optimal policy rule by maximizing an empirical welfare criterion constructed using nonparametric potential outcome time series. We characterize conditions under which T-EWM consistently learns a policy choice that is optimal in terms of conditional welfare given the time-series history. We derive a nonasymptotic upper bound for conditional welfare regret. To illustrate the implementation and uses of T-EWM, we perform simulation studies and apply the method to estimate optimal restriction rules against Covid-19. © 2022, CC BY.

关键词： expectation maximization algorithm

Scalable likelihood-based estimation and variable selection for the Cox model with incomplete covariates

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kwok, Ngok Sang Wong, Kin Yau Department of Applied Mathematics The Hong Kong Polytechnic University Hong Kong

Regression analysis with missing data is a long-standing and challenging problem, particularly when there are many missing variables with arbitrary missing patterns. Likelihood-based methods, although theoretically appealing, are often computationally inefficient or even infeasible when dealing with a large number of missing variables. In this paper, we consider the Cox regression model with incomplete covariates that are missing at random. We develop an expectation-maximization (EM) algorithm for nonparametric maximum likelihood estimation, employing a transformation technique in the E-step so that it involves only a one-dimensional integration. This innovation makes our methods scalable with respect to the dimension of the missing variables. In addition, for variable selection, we extend the proposed EM algorithm to accommodate a LASSO penalty in the likelihood. We demonstrate the feasibility and advantages of the proposed methods over existing methods by large-scale simulation studies and apply the proposed methods to a cancer genomic study. © 2024, CC BY-NC-ND.

关键词： expectation maximization algorithm

Probabilistic Targeted Factor Analysis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Herculano, Miguel C. Montoya-Blandón, Santiago Adam Smith Business School University of Glasgow United Kingdom

We develop a probabilistic variant of Partial Least Squares (PLS) we call Probabilistic Targeted Factor Analysis (PTFA), which can be used to extract common factors in predictors that are useful to predict a set of predetermined target variables. Along with the technique, we provide an efficient expectation-maximization (EM) algorithm to learn the parameters and forecast the targets of interest. We develop a number of extensions to missing-at-random data, stochastic volatility, and mixed-frequency data for real-time forecasting. In a simulation exercise, we show that PTFA outperforms PLS at recovering the common underlying factors affecting both features and target variables delivering better in-sample fit, and providing valid forecasts under contamination such as measurement error or outliers. Finally, we provide two applications in Economics and Finance where PTFA performs competitively compared with PLS and Principal Component Analysis (PCA) at out-of-sample forecasting. © 2024, CC BY-NC-SA.

关键词： expectation maximization algorithm

Empirical Welfare maximization with Constraints

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sun, Liyang UCL United Kingdom CEMFI Spain

Empirical Welfare maximization (EWM) is a framework that can be used to select welfare program eligibility policies based on data. This paper extends EWM by allowing for uncertainty in estimating the budget needed to implement the selected policy, in addition to its welfare. Due to the additional estimation error, I show there exist no rules that achieve the highest welfare possible while satisfying a budget constraint uniformly over a wide range of DGPs. This differs from the setting without a budget constraint where uniformity is achievable. I propose an alternative trade-off rule and illustrate it with Medicaid expansion, a setting with imperfect take-up and varying program costs. © 2021, CC BY.

关键词： expectation maximization algorithm

Variational Probabilistic Multi-Hypothesis Tracking

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Shin, Hyo-Sang Xu, Shuoyuan Tsourdos, Antonios School of Aerospace Transport and Manufacturing Cranfield University CranfieldMK43 0AL United Kingdom Cho Chun Shik Graduate School of Mobility Korea Advanced Institute of Science and Technology Daejeon34141 Korea Republic of

This paper proposes a novel multi-target tracking (MTT) algorithm for scenarios with arbitrary numbers of measurements per target. We propose the variational probabilistic multi-hypothesis tracking (VPMHT) algorithm based on the variational Bayesian expectation-maximisation (VBEM) algorithm to overcome limitations in the classic probabilistic multi-hypothesis tracking (PMHT) algorithm. The introduction of variational inference allows VPMHT to handle track-loss much better than the conventional PMHT, while preserving a similar or even better tracking accuracy. Extensive numerical simulations are conducted to validate the proposed algorithm. © 2024, The Authors. All rights reserved.

关键词： expectation maximization algorithm

UNIVERSAL WYNER-ZIV CODING FOR GAUSSIAN SOURCES

学校读者我要写书评

暂无评论

UNIVERSAL WYNER-ZIV CODING FOR GAUSSIAN SOURCES

IEEE International Conference on Acoustics, Speech and Signal Processing

作者： Elsa DUPRAZ Aline ROUMY Michel KIEFFER L2S - CNRS - SUPELEC - Univ Paris-Sud 91192 Gif-sur-Yvette France INRIA Campus de Beaulieu 35042 Rennes France

ISBN: (纸本)9781479903573

This paper considers the problem of lossy source coding with side information at the decoder only, for Gaussian sources, when the joint statistics of the sources are partly unknown. We propose a practical universal coding scheme based on scalar quantization and non-binary LDPC codes, which avoids the binarization of the quantized coefficients. We first explain how to choose the rate and to construct the LDPC coding matrix. Then, a decoding algorithm that jointly estimates the source sequence and the joint statistics of the sources is proposed. The proposed coding scheme suffers no loss compared to the practical coding scheme with same rate but known variance.

关键词： Source coding with side information Rate-distortion Low Density Parity Check codes expectation maximization algorithm low density parity check codes expectation-maximisation algorithm Rate distortion labeling scheme Gauss scalar quantization lossy source coding binarization

A Gradient Analysis Framework for Rewarding Good and Penalizing Bad Examples in Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tuan, Yi-Lin Wang, William Yang University of California Santa Barbara United States

Beyond maximum likelihood estimation (MLE), the standard objective of a language model (LM) that optimizes good examples probabilities, many studies have explored ways that also penalize bad examples for enhancing the quality of output distribution, including unlikelihood training, exponential maximizing average treatment effect (ExMATE), and direct preference optimization (DPO). To systematically compare these methods and further provide a unified recipe for LM optimization, in this paper, we present a unique angle of gradient analysis of loss functions that simultaneously reward good examples and penalize bad ones in LMs. Through both mathematical results and experiments on CausalDialogue and Anthropic HH-RLHF datasets, we identify distinct functional characteristics among these methods. We find that ExMATE serves as a superior surrogate for MLE, and that combining DPO with ExMATE instead of MLE further enhances both the statistical (5-7%) and generative (+18% win rate) performance. © 2024, CC BY.

关键词： expectation maximization algorithm

Estimation and imputation of missing data in longitudinal models with Zero-Inflated Poisson response variable

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Martinez-Lobo, D.S. Martinez, Oscar Orlando Melo Gutierrez, Nelson Alirio Cruz Universidad del Bosque Colombia Departamento de Estadistica Facultad de Ciencias Universidad Nacional de Colombia Colombia Universitat de les Illes Balears Departament de Matemàtiques i Informàtica Spain

This research deals with the estimation and imputation of missing data in longitu- dinal models with a Poisson response variable inflated with zeros. A methodology is proposed that is based on the use of maximum likelihood, assuming that data is missing at random and that there is a correlation between the response vari- ables. In each of the times, the expectation maximization (EM) algorithm is used: in step E, a weighted regression is carried out, conditioned on the previous times that are taken as covariates. In step M, the estimation and imputation of the missing data are performed. The good performance of the methodology in dif- ferent loss scenarios is demonstrated in a simulation study comparing the model only with complete data, and estimating missing data using the mode of the data of each individual. Furthermore, in a study related to the growth of corn, it is tested on real data to develop the algorithm in a practical scenario. © 2024, CC BY-NC-ND.

关键词： expectation maximization algorithm