检索结果-内蒙古大学图书馆

Asymptotic and compound e-values: multiple testing and empirical Bayes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ignatiadis, Nikolaos Wang, Ruodu Ramdas, Aaditya Department of Statistics and Data Science Institute University of Chicago United States Department of Statistics and Actuarial Science University of Waterloo Canada Departments of Statistics & Machine Learning Carnegie Mellon University United States

We explicitly define the notions of (exact, approximate or asymptotic) compound p-values and e-values, which have been implicitly presented and extensively used in the recent multiple testing literature. While it is known that the e-BH procedure with compound e-values controls the FDR, we show the converse: every FDR controlling procedure can be recovered by instantiating the e-BH procedure with certain compound e-values. Since compound e-values are closed under averaging, this allows for combination and derandomization of FDR procedures. We then connect compound e-values to empirical Bayes. In particular, we use the fundamental theorem of compound decision theory to derive the log-optimal simple separable compound e-value for testing a set of point nulls against point alternatives: it is a ratio of mixture likelihoods. We extend universal inference to the compound setting. As one example, we construct approximate compound e-values for multiple t-tests, where the (nuisance) variances may be different across hypotheses. Finally, we provide connections to related notions in the literature stated in terms of p-values. Copyright © 2024, The Authors. All rights reserved.

关键词： Decision theory

Optimal Ridge Regularization for Out-of-Distribution Prediction 41

学校读者我要写书评

暂无评论

Optimal Ridge Regularization for Out-of-Distribution Predict...

41st International Conference on machine learning, ICML 2024

作者： Patil, Pratik Du, Jin-Hong Tibshirani, Ryan J. Department of Statistics University of California BerkeleyCA94720 United States Department of Statistics and Data Science Carnegie Mellon University PittsburghPA15213 United States Machine Learning Department Carnegie Mellon University PittsburghPA15213 United States

We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-distribution prediction, where the test distribution deviates arbitrarily from the train distribution. We establish general conditions that determine the sign of the optimal regularization level under covariate and regression shifts. These conditions capture the alignment between the covariance and signal structures in the train and test data and reveal stark differences compared to the in-distribution setting. For example, a negative regularization level can be optimal under covariate shift or regression shift, even when the training features are isotropic or the design is underparameterized. Furthermore, we prove that the optimally tuned risk is monotonic in the data aspect ratio, even in the out-of-distribution setting and when optimizing over negative regularization levels. In general, our results do not make any modeling assumptions for the train or the test distributions, except for moment bounds, and allow for arbitrary shifts and the widest possible range of (negative) regularization levels. Copyright 2024 by the author(s)

关键词： Aspect ratio

On Fake News Detection with LLM Enhanced Semantics Mining

学校读者我要写书评

暂无评论

On Fake News Detection with LLM Enhanced Semantics Mining

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Ma, Xiaoxiao Zhang, Yuchen Ding, Kaize Yang, Jian Wu, Jia Fan, Hao School of Computing Macquarie University Sydney Australia Amazon Machine Learning Sydney Australia School of Information Management Wuhan University Hubei China Department of Statistics and Data Science Northwestern University IL United States

ISBN: (纸本)9798891761643

Large language models (LLMs) have emerged as valuable tools for enhancing textual features in various text-related tasks. Despite their superiority in capturing the lexical semantics between tokens for text analysis, our preliminary study on two popular LLMs, i.e., GPT-3.5 and Llama2, shows that simply applying news embeddings from LLMs is ineffective for fake news detection. Such embeddings only encapsulate the language styles between tokens. Meanwhile, the high-level semantics among named entities and topics, which reveal the deviating patterns of fake news, have been ignored. Therefore, we propose a topic model together with a set of specially designed prompts to extract topics and real entities from LLMs and model the relations among news, entities, and topics as a heterogeneous graph to facilitate investigating news semantics. We then propose a Generalized Page-Rank model and a consistent learning criterion for mining the local and global semantics centered on each news piece through the adaptive propagation of features across the graph. Our model shows superior performance on five benchmark datasets over seven baseline methods and the efficacy of the key ingredients has been thoroughly validated. © 2024 Association for Computational Linguistics.

关键词： Embeddings

Stability Bounds for Smooth Optimal Transport Maps and their Statistical Implications

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Balakrishnan, Sivaraman Manole, Tudor Department of Statistics and Data Science Machine Learning Department Carnegie Mellon University United States Statistics and Data Science Center Massachusetts Institute of Technology United States

We study estimators of the optimal transport (OT) map between two probability distributions. We focus on plugin estimators derived from the OT map between estimates of the underlying distributions. We develop novel stability bounds for OT maps which generalize those in past work, and allow us to reduce the problem of optimally estimating the transport map to that of optimally estimating densities in the Wasserstein distance. In contrast, past work provided a partial connection between these problems and relied on regularity theory for the Monge-Ampère equation to bridge the gap, a step which required unnatural assumptions to obtain sharp guarantees. We also provide some new insights into the connections between stability bounds which arise in the analysis of plugin estimators and growth bounds for the semi-dual functional which arise in the analysis of Brenier potential-based estimators of the transport map. We illustrate the applicability of our new stability bounds by revisiting the smooth setting studied by Manole et al. [2024], analyzing two of their estimators under more general conditions. Critically, our bounds do not require smoothness or boundedness assumptions on the underlying measures. As an illustrative application, we develop and analyze a novel tuning parameter-free estimator for the OT map between two strongly log-concave distributions. Copyright © 2025, The Authors. All rights reserved.

关键词： Probability distributions

Sequential Kernelized Stein Discrepancy

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Martinez-Taboada, Diego Ramdas, Aaditya Department of Statistics & Data Science Carnegie Mellon University United States Department of Statistics & Data Science Machine Learning Department Carnegie Mellon University United States

We present a sequential version of the kernelized Stein discrepancy goodness-of-fit test, which allows for conducting goodness-of-fit tests for unnormalized densities that are continuously monitored and adaptively stopped. That is, the sample size need not be fixed prior to data collection;the practitioner can choose whether to stop the test or continue to gather evidence at any time while controlling the false discovery rate. In stark contrast to related literature, we do not impose uniform boundedness on the Stein kernel. Instead, we exploit the potential boundedness of the Stein kernel at arbitrary point evaluations to define test martingales, that give way to the subsequent novel sequential tests. We prove the validity of the test, as well as an asymptotic lower bound for the logarithmic growth of the wealth process under the alternative. We further illustrate the empirical performance of the test with a variety of distributions, including restricted Boltzmann machines. Copyright © 2024, The Authors. All rights reserved.

关键词：

Identifying General Mechanism Shifts in Linear Causal Representations 38

学校读者我要写书评

暂无评论

Identifying General Mechanism Shifts in Linear Causal Repres...

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Chen, Tianyu Bello, Kevin Locatello, Francesco Aragam, Bryon Ravikumar, Pradeep Department of Statistics and Data Sciences University of Texas Austin United States Booth School of Business University of Chicago United States Machine Learning Department Carnegie Mellon University United States Institute of Science and Technology Austria Austria

We consider the linear causal representation learning setting where we observe a linear mixing of d unknown latent factors, which follow a linear structural causal model. Recent work has shown that it is possible to recover the latent factors as well as the underlying structural causal model over them, up to permutation and scaling, provided that we have at least d environments, each of which corresponds to perfect interventions on a single latent node (factor). After this powerful result, a key open problem faced by the community has been to relax these conditions: allow for coarser than perfect single-node interventions, and allow for fewer than d of them, since the number of latent factors d could be very large. In this work, we consider precisely such a setting, where we allow a smaller than d number of environments, and also allow for very coarse interventions that can very coarsely change the entire causal graph over the latent factors. On the flip side, we relax what we wish to extract to simply the list of nodes that have shifted between one or more environments. We provide a surprising identifiability result that it is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes. Our identifiability proof moreover is a constructive one: we explicitly provide necessary and sufficient conditions for a node to be a shifted node, and show that we can check these conditions given observed data. Our algorithm lends itself very naturally to the sample setting where instead of just interventional distributions, we are provided datasets of samples from each of these distributions. We corroborate our results on both synthetic experiments as well as an interesting psychometric dataset. The code can be found at https://***/TianyuCodings/iLCS. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

TRACKING THE RISK OF A DEPLOYED MODEL AND DETECTING HARMFUL DISTRIBUTION SHIFTS 10

学校读者我要写书评

暂无评论

TRACKING THE RISK OF A DEPLOYED MODEL AND DETECTING HARMFUL ...

10th International Conference on learning Representations, ICLR 2022

作者： Podkopaev, Aleksandr Ramdas, Aaditya Department of Statistics & Data Science Carnegie Mellon University United States Machine Learning Department Carnegie Mellon University United States

When deployed in the real world, machine learning models inevitably encounter changes in the data distribution, and certain-but not all-distribution shifts could result in significant performance degradation. In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially, making interventions by a human expert (or model retraining) unnecessary. While several works have developed tests for distribution shifts, these typically either use non-sequential methods, or detect arbitrary shifts (benign or harmful), or both. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate. In this work, we design simple sequential tools for testing if the difference between source (training) and target (test) distributions leads to a significant increase in a risk function of interest, like accuracy or calibration. Recent advances in constructing time-uniform confidence sequences allow efficient aggregation of statistical evidence accumulated during the tracking process. The designed framework is applicable in settings where (some) true labels are revealed after the prediction is performed, or when batches of labels become available in a delayed fashion. We demonstrate the efficacy of the proposed framework through an extensive empirical study on a collection of simulated and real datasets. © 2022 ICLR 2022 - 10th International Conference on learning Representationss. All rights reserved.

关键词：

Empirical Bernstein in smooth Banach spaces

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Martinez-Taboada, Diego Ramdas, Aaditya Department of Statistics & Data Science United States Machine Learning Department Carnegie Mellon University United States

Existing concentration bounds for bounded vector-valued random variables include extensions of the scalar Hoeffding and Bernstein inequalities. While the latter is typically tighter, it requires knowing a bound on the variance of the random variables. We derive a new vector-valued empirical Bernstein inequality, which makes use of an empirical estimator of the variance instead of the true variance. The bound holds in 2-smooth separable Banach spaces, which include finite dimensional Euclidean spaces and separable Hilbert spaces. The resulting confidence sets are instantiated for both the batch setting (where the sample size is fixed) and the sequential setting (where the sample size is a stopping time). The confidence set width asymptotically exactly matches that achieved by Bernstein in the leading term. The method and supermartingale proof technique combine several tools of Pinelis (1994) and Waudby-Smith and Ramdas (2024). Copyright © 2024, The Authors. All rights reserved.

关键词： Banach spaces

Hypothesis testing with e-values

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ramdas, Aaditya Wang, Ruodu Department of Statistics and Data Science Machine Learning Department Carnegie Mellon University United States Department of Statistics and Actuarial Science University of Waterloo Canada

This book is written to offer a humble, but unified, treatment of e-values inhypothesis testing. The book is organized into three parts: FundamentalConcepts, Core Ideas, and Advanced Topics. The first part includes threechapters that introduce the basic concepts. The second part includes fivechapters of core ideas such as universal inference, log-optimality,e-processes, operations on e-values, and e-values in multiple testing. Thethird part contains five chapters of advanced topics. We hope that, by puttingthe materials together in this book, the concept of e-values becomes moreaccessible for educational, research, and practical use. Copyright © 2024, The Authors. All rights reserved.

关键词：