Star-Shaped Space of Solutions of the Spherical Negative Perceptron

作     者：Brandon Livio Annesi Clarissa Lauditi Carlo Lucibello Enrico M. Malatesta Gabriele Perugini Fabrizio Pittorino Luca Saglietti 

作者机构：Department of Computing Sciences Bocconi University 20136 Milano Italy Department of Applied Science and Technology Politecnico di Torino 10129 Torino Italy Bocconi Institute for Data Science and Analytics 20136 Milano Italy Department of Electronics Information and Bioengineering Politecnico di Milano 20125 Milano Italy 

出 版 物：《Physical Review Letters》 (Phys Rev Lett)

年 卷 期：2023年第131卷第22期

页      面：227301-227301页

核心收录：

学科分类：07[理学] 0702[理学-物理学] 

主　　题：Optimization problems Artificial neural networks Replica methods 

摘      要：Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pairs of distant solutions can be constructed. Here, we consider the spherical negative perceptron, a prototypical nonconvex neural network model framed as a continuous constraint satisfaction problem. We introduce a general analytical method for computing energy barriers in the simplex with vertex configurations sampled from the equilibrium. We find that in the overparametrized regime the solution manifold displays simple connectivity properties. There exists a large geodesically convex component that is attractive for a wide range of optimization dynamics. Inside this region we identify a subset of atypical high-margin solutions that are geodesically connected with most other solutions, giving rise to a star-shaped geometry. We analytically characterize the organization of the connected space of solutions and show numerical evidence of a transition, at larger constraint densities, where the aforementioned simple geodesic connectivity breaks down.