Approximating Metric Magnitude of Point Sets

作     者：Andreeva, Rayna Ward, James Skraba, Primoz Gao, Jie Sarkar, Rik 

作者机构：School of Informatics University of Edinburgh United Kingdom Queen Mary University of London United Kingdom Rutgers University United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Optimization algorithms 

摘      要：Metric magnitude is a measure of the size of point clouds with many desirable geometric properties. It has been adapted to various mathematical contexts and recent work suggests that it can enhance machine learning and optimization algorithms. But its usability is limited due to the computational cost when the dataset is large or when the computation must be carried out repeatedly (e.g. in model training). In this paper, we study the magnitude computation problem, and show efficient ways of approximating it. We show that it can be cast as a convex optimization problem, but not as a submodular optimization. The paper describes two new algorithms - an iterative approximation algorithm that converges fast and is accurate, and a subset selection method that makes the computation even faster. It has been previously proposed that magnitude of model sequences generated during stochastic gradient descent is correlated to generalization gap. Extension of this result using our more scalable algorithms shows that longer sequences in fact bear higher correlations. We also describe new applications of magnitude in machine learning - as an effective regularizer for neural network training, and as a novel clustering criterion. Copyright © 2024, The Authors. All rights reserved.