检索结果-内蒙古大学图书馆

37th Annual Conference on Learning Theory, COLT 2024

作者： Hanneke, Steve Moran, Shay Waknine, Tom Department of Computer Science Purdue University United States Departments of Mathematics Computer Science and Data and Decision Sciences Technion Google Research Israel Department of Mathematics Technion Israel

List learning is a variant of supervised classification where the learner outputs multiple plausible labels for each instance rather than just one. We investigate classical principles related to generalization within the context of list learning. Our primary goal is to determine whether classical principles in the PAC setting retain their applicability in the domain of list PAC learning. We focus on uniform convergence (which is the basis of Empirical Risk Minimization) and on sample compression (which is a powerful manifestation of Occam's Razor). In classical PAC learning, both uniform convergence and sample compression satisfy a form of 'completeness': whenever a class is learnable, it can also be learned by a learning rule that adheres to these principles. We ask whether the same completeness holds true in the list learning setting. We show that uniform convergence remains equivalent to learnability in the list PAC learning setting. In contrast, our findings reveal surprising results regarding sample compression: we prove that when the label space is Y = {0, 1, 2}, then there are 2-list-learnable classes that cannot be compressed. This refutes the list version of the sample compression conjecture by Littlestone and Warmuth (1986). We prove an even stronger impossibility result, showing that there are 2-list-learnable classes that cannot be compressed even when the reconstructed function can work with lists of arbitrarily large size. We prove a similar result for (1-list) PAC learnable classes when the label space is unbounded. This generalizes a recent result by Pabbaraju (2023). In our impossibility results on sample compression, we employ direct-sum arguments which might be of independent interest. In fact, these arguments raise natural open questions that we leave for future research. Our findings regarding uniform convergence rely on a coding theoretic perspective. © 2024 S. Hanneke, S. Moran & T. Waknine.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

Open problem: Direct Sums in Learning Theory 37

Open problem: Direct Sums in Learning Theory

引用

37th Annual Conference on Learning Theory, COLT 2024

来源：评论

学校读者我要写书评

暂无评论

Dual VC Dimension Obstructs Sample Compression by Embeddings 37

Dual VC Dimension Obstructs Sample Compression by Embeddings

引用

37th Annual Conference on Learning Theory, COLT 2024

作者： Chase, Zachary Chornomaz, Bogdan Hanneke, Steve Moran, Shay Yehudayoff, Amir Department of Mathematics Technion Israel Department of Computer Science Purdue University United States Departments of Mathematics Computer Science and Data and Decision Sciences Technion Google Research Israel Department of Computer Science Copenhagen University Denmark

This work studies embedding of arbitrary VC classes in well-behaved VC classes, focusing particularly on extremal classes. Our main result expresses an impossibility: such embeddings necessarily require a significant increase in dimension. In particular, we prove that for every d there is a class with VC dimension d that cannot be embedded in any extremal class of VC dimension smaller than exponential in d. In addition to its independent interest, this result has an important implication in learning theory, as it reveals a fundamental limitation of one of the most extensively studied approaches to tackling the long-standing sample compression conjecture (Warmuth, 2003). Concretely, the approach proposed by Floyd and Warmuth (1995) entails embedding any given VC class into an extremal class of a comparable dimension, and then applying an optimal sample compression scheme for extremal classes. However, our results imply that this strategy would in some cases result in a sample compression scheme at least exponentially larger than what is predicted by the sample compression conjecture. The above implications follow from a general result we prove: any extremal class with VC dimension d has dual VC dimension at most 2d + 1. This bound is exponentially smaller than the classical bound 2d+1-1 of Assouad (1983), which applies to general concept classes (and is known to be unimprovable for some classes). We in fact prove a stronger result, establishing that 2d + 1 upper bounds the dual Radon number of extremal classes. This theorem represents an abstraction of the classical Radon theorem for convex sets, extending its applicability to a wider combinatorial framework, without relying on the specifics of Euclidean convexity. The proof utilizes the topological method and is primarily based on variants of the Topological Radon Theorem (Bajmóczy and Bárány, 1979). © 2024 Z. Chase, B. Chornomaz, S. Hanneke, S. Moran & A. Yehudayoff.

关键词： Topology

来源：评论

学校读者我要写书评

暂无评论

Challenges in the Differential Classification of Individual Diagnoses from Co-Occurring Autism and ADHD Using Survey data

Challenges in the Differential Classification of Individual ...

引用

2024 IEEE-EMBS International Conference on Biomedical and Health Informatics, BHI 2024

作者： Jaiswal, Aditi Wall, Dennis P. Washington, Peter University of Hawaii at Manoa Information and Computer Sciences Department Honolulu United States Stanford University Departments of Pediatrics & Biomedical Data Science Stanford United States

ISBN: (纸本)9798350351552

Autism and Attention-Deficit Hyperactivity Disorder (ADHD) are two of the most commonly observed neurodevelopmental conditions in childhood. Providing a specific computational assessment to distinguish between the two can prove difficult and time intensive. Given the high prevalence of their co-occurrence, there is a need for scalable and accessible methods for distinguishing the co-occurrence of autism and ADHD from individual diagnoses. The first step is to identify a core set of features that can serve as the basis for behavioral feature extraction. We trained machine learning models on data from the National Survey of Children's Health to identify behaviors to target as features in automated clinical decision support systems. A model trained on the binary task of distinguishing either developmental delay (autism or ADHD) vs. neither achieved sensitivity >92% and specificity >94%, while a model trained on the 4-way classification task of autism vs. ADHD vs. both vs. none demonstrated >65% sensitivity and >66% specificity. While the performance of the binary model was respectable, the relatively low performance in the differential classification of autism and ADHD highlights the challenges that persist in achieving specificity within clinical decision support tools for developmental delays. Nevertheless, this study demonstrates the potential of applying behavioral questionnaires not traditionally used for clinical purposes towards supporting digital screening assessments for pediatric developmental delays. © 2024 IEEE.

关键词： Diseases

来源：评论

学校读者我要写书评

暂无评论

Universally Instance-Optimal Mechanisms for Private Statistical Estimation 37

Universally Instance-Optimal Mechanisms for Private Statisti...

引用

37th Annual Conference on Learning Theory, COLT 2024

作者： Asi, Hilal Duchi, John C. Haque, Saminul Li, Zewei Ruan, Feng Apple United States Departments of Statistics and Electrical Engineering Stanford University United States Department of Computer Science Stanford University United States Department of Statistics and Data Science Northwestern University United States

We consider the problem of instance-optimal statistical estimation under the constraint of differential privacy where mechanisms must adapt to the difficulty of the input dataset. We prove a new instance specific lower bound using a new divergence and show it characterizes the local minimax optimal rates for private statistical estimation. We propose two new mechanisms that are universally instance-optimal for general estimation problems up to logarithmic factors. Our first mechanism, the total variation mechanism, builds on the exponential mechanism with stable approximations of the total variation distance, and is universally instance-optimal in the high privacy regime Ε ≤ 1/√n. Our second mechanism, the T-mechanism, is based on the T-estimator framework (Birgé, 2006) using the clipped log likelihood ratio as a stable test: it attains instance-optimal rates for any Ε ≤ 1 up to logarithmic factors. Finally, we study the implications of our results to robust statistical estimation, and show that our algorithms are universally optimal for this problem, characterizing the optimal minimax rates for robust statistical estimation. © 2024 H. Asi, J.C. Duchi, S. Haque, Z. Li & F. Ruan.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Unified Characterization of Private Learnability via Graph Theory 37

A Unified Characterization of Private Learnability via Graph...

引用

37th Annual Conference on Learning Theory, COLT 2024

作者： Alon, Noga Moran, Shay Schefler, Hilla Yehudayoff, Amir Department of Mathematics Princeton University United States Departments of Mathematics Computer Science and Data and Decision Sciences Technion Google Research Israel Department of Mathematics Technion Israel

We provide a unified framework for characterizing pure and approximate differentially private (DP) learnability. The framework uses the language of graph theory: for a concept class H, we define the contradiction graph G of H. Its vertices are realizable datasets and two datasets S, S′ are connected by an edge if they contradict each other (i.e., there is a point x that is labeled differently in S and S′). Our main finding is that the combinatorial structure of G is deeply related to learning H under DP. Learning H under pure DP is captured by the fractional clique number of G. Learning H under approximate DP is captured by the clique number of G. Consequently, we identify graph-theoretic dimensions that characterize DP learnability: the clique dimension and fractional clique dimension. Along the way, we reveal properties of the contradiction graph which may be of independent interest. We also suggest several open questions and directions for future research. © 2024 N. Alon, S. Moran, H. Schefler & A. Yehudayoff.

关键词： Differential privacy

来源：评论

学校读者我要写书评

暂无评论

Invertibility of Discrete-Time Linear Systems with Sparse Inputs 63

Invertibility of Discrete-Time Linear Systems with Sparse In...

引用

63rd IEEE Conference on Decision and Control, CDC 2024

作者： Poe, Kyle Mallada, Enrique Vidal, Rene University of Pennsylvania Applied Mathematics and Computational Science Group PA19104 United States Johns Hopkins University Mallada Is with the Department of Electrical and Computer Engineering MD21218 United States University of Pennsylvania Radiology Computer and Information Science Statistics and Data Science Departments of Electrical and Systems Engineering PA19104 United States

ISBN: (纸本)9798350316339

One of the fundamental problems of interest for discrete-time linear systems is whether its input sequence may be recovered given its output sequence, a.k.a. the left inversion problem. Many conditions on the state space geometry, dynamics, and spectral structure of a system have been used to characterize the well-posedness of this problem, without assumptions on the inputs. However, certain structural assumptions, such as input sparsity, have been shown to translate to practical gains in the performance of inversion algorithms, surpassing classical guarantees. Establishing necessary and sufficient conditions for left invertibility of systems with sparse inputs is therefore a crucial step toward understanding the performance limits of system inversion under structured input assumptions. In this work, we provide the first necessary and sufficient characterizations of left invertibility for linear systems with sparse inputs, echoing classic characterizations for standard linear systems. The key insight in deriving these results is in establishing the existence of two novel geometric invariants unique to the sparseinput setting, the weakly unobservable and strongly reachable subspace arrangements. By means of a concrete example, we demonstrate the utility of these characterizations. We conclude by discussing extensions and applications of this framework to several related problems in sparse control. © 2024 IEEE.

关键词： State space methods

来源：评论

学校读者我要写书评

暂无评论

On Reductions and Representations of Learning Problems in Euclidean Spaces

arXiv

引用

arXiv 2024年

作者： Chornomaz, Bogdan Moran, Shay Waknine, Tom Departments of Mathematics Technion Departments of Mathematics Computer Science and Data and Decision Sciences Technion Google Research

Many practical prediction algorithms represent inputs in Euclidean space and replace the discrete 0/1 classification loss with a real-valued surrogate loss, effectively reducing classification tasks to stochastic optimization. In this paper, we investigate the expressivity of such reductions in terms of key resources, including dimension and the role of randomness. We establish bounds on the minimum Euclidean dimension D needed to reduce a concept class with VC dimension d to a Stochastic Convex Optimization (SCO) problem in RD, formally addressing the intuitive interpretation of the VC dimension as the number of parameters needed to learn the class. To achieve this, we develop a generalization of the Borsuk-Ulam Theorem that combines the classical topological approach with convexity considerations. Perhaps surprisingly, we show that, in some cases, the number of parameters D must be exponentially larger than the VC dimension d, even if the reduction is only slightly non-trivial. We also present natural classification tasks that can be represented in much smaller dimensions by leveraging randomness, as seen in techniques like random initialization. This result resolves an open question posed by Kamath, Montasser, and Srebro (COLT 2020). Our findings introduce new variants of dimension complexity (also known as sign-rank), a well-studied parameter in learning and complexity theory. Specifically, we define an approximate version of sign-rank and another variant that captures the minimum dimension required for a reduction to SCO. We also propose several open questions and directions for future *** Codes 68Q32 © 2024, CC BY-SA.

关键词： Stochastic systems

来源：评论

学校读者我要写书评

暂无评论

Learnability Gaps of Strategic Classification 37

Learnability Gaps of Strategic Classification

引用

37th Annual Conference on Learning Theory, COLT 2024

作者： Cohen, Lee Mansour, Yishay Moran, Shay Shao, Han Stanford United States Tel Aviv University Google Research Israel Departments of Mathematics Computer Science and Data and Decision Sciences Technion Google Research Israel Toyota Technological Institute of Chicago United States

In contrast with standard classification tasks, strategic classification involves agents strategically modifying their features in an effort to receive favorable predictions. For instance, given a classifier determining loan approval based on credit scores, applicants may open or close their credit cards and bank accounts to fool the classifier. The learning goal is to find a classifier robust against strategic manipulations. Various settings, based on what and when information is known, have been explored in strategic classification. In this work, we focus on addressing a fundamental question: the learnability gaps between strategic classification and standard learning. We essentially show that any learnable class is also strategically learnable: we first consider a fully informative setting, where the manipulation structure (which is modeled by a manipulation graph G*) is known and during training time the learner has access to both the pre-manipulation data and post-manipulation data. We provide nearly tight sample complexity and regret bounds, offering significant improvements over prior results. Then, we relax the fully informative setting by introducing two natural types of uncertainty. First, following Ahmadi et al. (2023), we consider the setting in which the learner only has access to the post-manipulation data. We improve the results of Ahmadi et al. (2023) and close the gap between mistake upper bound and lower bound raised by them. Our second relaxation of the fully informative setting introduces uncertainty to the manipulation structure. That is, we assume that the manipulation graph is unknown but belongs to a known class of graphs. We provide nearly tight bounds on the learning complexity in various unknown manipulation graph settings. Notably, our algorithm in this setting is of independent interest and can be applied to other problems such as multi-label learning. © 2024 L. Cohen, Y. Mansour, S. Moran & H. Shao.

关键词： Contrastive Learning

来源：评论

学校读者我要写书评

暂无评论

Conformal frequency estimation using discrete sketched data with coverage for distinct queries

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2023年第1期24卷 16735-16814页

作者： Matteo Sesia Stefano Favaro Edgar Dobriban Departments of Data Sciences and Operations and of Computer Science University of Southern California Los Angeles CA Department of Economics and Statistics University of Torino and Collegio Carlo Alberto Torino Italy Departments of Statistics and Data Science and of Computer and Information Science University of Pennsylvania Philadelphia PA

This paper develops conformal inference methods to construct a confidence interval for the frequency of a queried object in a very large discrete data set, based on a sketch with a lower memory footprint. This approach requires no knowledge of the data distribution and can be combined with any sketching algorithm, including but not limited to the renowned count-min sketch, the count-sketch, and variations thereof. After explaining how to achieve marginal coverage for exchangeable random queries, we extend our solution to provide stronger inferences that can account for the discreteness of the data and for heterogeneous query frequencies, increasing also robustness to possible distribution shifts. These results are facilitated by a novel conformal calibration technique that guarantees valid coverage for a large fraction of distinct random queries. Finally, we show our methods have improved empirical performance compared to existing frequentist and Bayesian alternatives in simulations as well as in examples of text and SARS-CoV-2 DNA data.

关键词： conformal inference discrete data distribution shifts sketching uncertainty

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：