检索结果-内蒙古大学图书馆

Robust and Efficient Imbalanced Positive-Unlabeled learning with Self-supervision

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Dorigatti, Emilio Schweisthal, Jonas Bischl, Bernd Rezaei, Mina Statistical Learning and Data Science Ludwig-Maximilians-University Munich Germany

learning from positive and unlabeled (PU) data is a setting where the learner only has access to positive and unlabeled samples while having no information on negative examples. Such PU setting is of great importance in various tasks such as medical diagnosis, social network analysis, financial markets analysis, and knowledge base completion, which also tend to be intrinsically imbalanced, i.e., where most examples are actually negatives. Most existing approaches for PU learning, however, only consider artificially balanced datasets and it is unclear how well they perform in the realistic scenario of imbalanced and long-tail data distribution. This paper proposes to tackle this challenge via robust and efficient self-supervised pretraining. However, training conventional self-supervised learning methods when applied with highly imbalanced PU distribution needs better reformulation. In this paper, we present ImPULSeS, a unified representation learning framework for Imbalanced Positive Unlabeled learning leveraging Self-Supervised debiase pre-training. ImPULSeS uses a generic combination of large-scale unsupervised learning with debiased contrastive loss and additional reweighted PU loss. We performed different experiments across multiple datasets to show that ImPULSeS is able to halve the error rate of the previous stat-of-the-art, even compared with previous methods that are given the true prior. Moreover, our method showed increased robustness to prior misspecification and superior performance even when pretraining was performed on an unrelated dataset. We anticipate such robustness and efficiency will make it much easier for practitioners to obtain excellent results on other PU datasets of interest. © 2022, CC BY.

关键词： Supervised learning

PREDICTION-BASED VARIABLE SELECTION FOR COMPONENT-WISE GRADIENT BOOSTING

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Potts, Sophie Bergherr, Elisabeth Griesbach, Colin Reinke, Constantin Spatial Data Science and Statistical Learning University of Goettingen Goettingen Germany Empirical Methods in Social Science and Demography University of Rostock Rostock Germany

Model-based component-wise gradient boosting is a popular tool for data-driven variable selection. In order to improve its prediction and selection qualities even further, several modifications of the original algorithm have been developed, that mainly focus on different stopping criteria, leaving the actual variable selection mechanism untouched. We investigate different prediction-based mechanisms for the variable selection step in model-based component-wise gradient boosting. These approaches include Akaikes Information Criterion (AIC) as well as a selection rule relying on the component-wise test error computed via cross-validation. We implemented the AIC and cross-validation routines for Generalized Linear Models and evaluated them regarding their variable selection properties and predictive performance. An extensive simulation study revealed improved selection properties whereas the prediction error could be lowered in a real world application with age-standardized COVID-19 incidence rates. © 2023, CC BY-NC-SA.

关键词： Forecasting

Conformal online model aggregation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gasparin, Matteo Ramdas, Aaditya Machine Learning Department Carnegie Mellon University United States Department of Statistics and Data Science Carnegie Mellon University United States Department of Statistical Sciences University of Padova Italy

Conformal prediction equips machine learning models with a reasonable notion of uncertainty quantification without making strong distributional assumptions. It wraps around any black-box prediction model and converts point predictions into set predictions that have a predefined marginal coverage guarantee. However, conformal prediction only works if we fix the underlying machine learning model in advance. A relatively unaddressed issue in conformal prediction is that of model selection and/or aggregation: for a given problem, which of the plethora of prediction methods (random forests, neural nets, regularized linear models, etc.) should we conformalize? This paper proposes a new approach towards conformal model aggregation in online settings that is based on combining the prediction sets from several algorithms by voting, where weights on the models are adapted over time based on past performance. Copyright © 2024, The Authors. All rights reserved.

关键词： Forecasting

Merging uncertainty sets via majority vote

学校读者我要写书评

暂无评论

arXiv 2024年

Given K uncertainty sets that are arbitrarily dependent — for example, confidence intervals for an unknown parameter obtained with K different estimators, or prediction sets obtained via conformal prediction based on K different algorithms on shared data — we address the question of how to efficiently combine them in a black-box manner to produce a single uncertainty set. We present a simple and broadly applicable majority vote procedure that produces a merged set with nearly the same error guarantee as the input sets. We then extend this core idea in a few ways: we show that weighted averaging can be a powerful way to incorporate prior information, and a simple randomization trick produces strictly smaller merged sets without altering the coverage guarantee. Further improvements can be obtained if the sets are exchangeable. We also show that many modern methods, like split conformal prediction, median of means, HulC and cross-fitted "double machine learning", can be effectively derandomized using these ideas. Copyright © 2024, The Authors. All rights reserved.

关键词： Prediction models

Topological Analysis on Multi-scenario Graphs: Applications Toward Discerning Variability in SARS-CoV-2 and Topic Similarity in Research

学校读者我要写书评

暂无评论

Transactions of the Indian National Academy of Engineering : an international journal of engineering and technology 2022年第1期7卷 365-374页

作者： Sourav Biswas Malay Bhattacharyya Sanghamitra Bandyopadhyay Indian Statistical Institute Kolkata India. University of Calcutta Kolkata India. Machine Intelligence Unit Centre for Artificial Intelligence and Machine Learning Technology Innovation Hub on Data Science Big Data Analytics and Data Curation Indian Statistical Institute Kolkata India.

A network is often an obvious choice for modeling real-life interconnected systems, where the nodes represent interacting objects and the edges represent their associations. There has been immense progress in complex network analysis with methods and tools that can provide important insights into the respective scenario. In the advancement of information technology and globalization, the amount of data is increasing day by day, and it is indeed incomprehensible without the help of network science. This work highlights how we can model multiple interaction scenarios under a single umbrella to uncover novel insights. We show that a varying scenario gets reflected by the change of topological patterns in interaction networks. We construct multi-scenario graphs, a novel framework proposed by us, from real-life environments followed by topological analysis. We focus on two different application areas: analyzing geographical variations in SARS-CoV-2 and studying topic similarity in citation patterns.

关键词： Citation Network Graphlet and Graphlet Degree Distribution Multi-scenario Graphs SARS-CoV-2 Topic Similarity Network Topological Analysis

Combining exchangeable p-values

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gasparin, Matteo Wang, Ruodu Ramdas, Aaditya Department of Statistical Sciences University of Padova Italy Department of Statistics and Actuarial Science University of Waterloo Canada Department of Statistics and Data Science Carnegie Mellon University United States Machine Learning Department Carnegie Mellon University United States

The problem of combining p-values is an old and fundamental one, and the classic assumption of independence is often violated or unverifiable in many applications. There are many well-known rules that can combine a set of arbitrarily dependent p-values (for the same hypothesis) into a single p-value. We show that essentially all these existing rules can be strictly improved when the p-values are exchangeable, or when external randomization is allowed (or both). For example, we derive randomized and/or exchangeable improvements of well known rules like "twice the median" and "twice the average", as well as geometric and harmonic means. Exchangeable p-values are often produced one at a time (for example, under repeated tests involving data splitting), and our rules can combine them sequentially as they are produced, stopping when the combined p-values stabilize. Our work also improves rules for combining arbitrarily dependent p-values, since the latter becomes exchangeable if they are presented to the analyst in a random order. The main technical advance is to show that all existing combination rules can be obtained by calibrating the p-values to e-values (using an α-dependent calibrator), averaging those e-values, converting to a level-α test using Markov’s inequality, and finally obtaining p-values by combining this family of tests;the improvements are delivered via recent randomized and exchangeable variants of Markov’s inequality. © 2024, CC BY-NC-ND.

关键词： Markov processes

HPO × ELA: Investigating Hyperparameter Optimization Landscapes by Means of Exploratory Landscape Analysis

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Schneider, Lennart Schäpermeier, Lennart Prager, Raphael Patrick Bischl, Bernd Trautmann, Heike Kerschke, Pascal Statistical Learning and Data Science LMU Munich Germany Big Data Analytics in Transportation TU Dresden Germany Data Science: Statistics and Optimization University of Münster Germany Data Management and Biometrics Group University of Twente Netherlands

Hyperparameter optimization (HPO) is a key component of machine learning models for achieving peak predictive performance. While numerous methods and algorithms for HPO have been proposed over the last years, little progress has been made in illuminating and examining the actual structure of these black-box optimization problems. Exploratory landscape analysis (ELA) subsumes a set of techniques that can be used to gain knowledge about properties of unknown optimization problems. In this paper, we evaluate the performance of five different black-box optimizers on 30 HPO problems, which consist of two-, three- and five-dimensional continuous search spaces of the XGBoost learner trained on 10 different data sets. This is contrasted with the performance of the same optimizers evaluated on 360 problem instances from the black-box optimization benchmark (BBOB). We then compute ELA features on the HPO and BBOB problems and examine similarities and differences. A cluster analysis of the HPO and BBOB problems in ELA feature space allows us to identify how the HPO problems compare to the BBOB problems on a structural meta-level. We identify a subset of BBOB problems that are close to the HPO problems in ELA feature space and show that optimizer performance is comparably similar on these two sets of benchmark problems. We highlight open challenges of ELA for HPO and discuss potential directions of future research and applications. © 2022, CC BY.

关键词： Benchmarking

Joint Debiased Representation learning and Imbalanced data Clustering

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Rezaei, Mina Dorigatti, Emilio Ruegamer, David Bischl, Bernd Statistical Learning and Data Science Ludwig-Maximilians-University Munich Germany

One of the most promising approaches for unsupervised learning is combining deep representation learning and deep clustering. Some recent works propose to simultaneously learn representation using deep neural networks and perform clustering by defining a clustering loss on top of embedded features. However, these approaches are sensitive to imbalanced data and out-of-distribution samples. As a consequence, these methods optimize clustering by pushing data close to randomly initialized cluster centers. This is problematic when the number of instances varies largely in different classes or a cluster with few samples has less chance to be assigned a good centroid. To overcome these limitations, we introduce a new unsupervised framework for joint debiased representation learning and image clustering. We simultaneously train two deep learning models, a deep representation network that captures the data distribution, and a deep clustering network that learns embedded features and performs clustering. Specifically, the clustering network and learning representation network both take advantage of our proposed statistics pooling block that represents mean, variance, and cardinality to handle the out-of-distribution samples and class imbalance. Our experiments show that using these representations, one can considerably improve results on imbalanced image clustering across a variety of image datasets. Moreover, the learned representations generalize well when transferred to the out-of-distribution dataset. © 2021, CC BY.

关键词： Deep neural networks

Existence of Direct Density Ratio Estimators

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Banzato, Erika Drton, Mathias Saraf-Poor, Kian Shi, Hongjian Department of Statistical Sciences University of Padova Italy TUM School of Computation Information and Technology Munich Data Science Institute Technical University of Munich Munich Center for Machine Learning Germany Department of Statistics Columbia University United States TUM School of Computation Information and Technology Technical University of Munich Germany

Many two-sample problems call for a comparison of two distributions from an exponential family. Density ratio estimation methods provide ways to solve such problems through direct estimation of the differences in natural parameters. The term direct indicates that one avoids estimating both marginal distributions. In this context, we consider the Kullback–Leibler Importance Estimation Procedure (KLIEP), which has been the subject of recent work on differential networks. Our main result shows that the existence of the KLIEP estimator is characterized by whether the average sufficient statistic for one sample belongs to the convex hull of the set of all sufficient statistics for data points in the second sample. For high-dimensional problems it is customary to regularize the KLIEP loss by adding the product of a tuning parameter and a norm of the vector of parameter differences. We show that the existence of the regularized KLIEP estimator requires the tuning parameter to be no less than the dual norm-based distance between the average sufficient statistic and the convex hull. The implications of these existence issues are explored in applications to differential network analysis. Copyright © 2025, The Authors. All rights reserved.

关键词： Parameter estimation