检索结果-内蒙古大学图书馆

Efficient Numerical Algorithm for Large-Scale Damped Natural Gradient Descent

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Yixiao Xie, Hao Wang, Han Program in Applied and Computational Mathematics Princeton University United States Beijing National Laboratory for Condensed Matter Physics Institute of Physics Chinese Academy of Sciences China Laboratory of Computational Physics Institute of Applied Physics and Computational Mathematics

We propose a new algorithm for efficiently solving the damped Fisher matrix in large-scale scenarios where the number of parameters significantly exceeds the number of available samples. This problem is fundamental for natural gradient descent and stochastic reconfiguration. Our algorithm is based on Cholesky decomposition and is generally applicable. Benchmark results show that the algorithm is significantly faster than existing methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

Multiscale simulations in simple metals: A density-functional-based methodology

学校读者我要写书评

暂无评论

Physical Review B 2005年第9期71卷 094101-094101页

作者： Nicholas Choly Gang Lu Weinan E Efthimios Kaxiras Division of Engineering and Applied Sciences Harvard University Cambridge Massachusetts 02138 USA Department of Mathematics and Program in Applied and Computational Mathematics Princeton University Princeton New Jersey 08544 USA

We present a formalism for coupling a density-functional-theory-based quantum simulation to a classical simulation for the treatment of simple metallic systems. The formalism is applicable to multiscale simulations in which the part of the system requiring quantum-mechanical treatment is spatially confined to a small region. Such situations often arise in physical systems where chemical interactions in a small region can affect the macroscopic mechanical properties of a metal. We describe how this coupled treatment can be accomplished efficiently, and we present a coupled simulation for a bulk aluminum system.

关键词：

Community Detection and Stochastic Block Models

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Abbe, Emmanuel Program in Applied and Computational Mathematics Department of Electrical Engineering Princeton University Princeton United States

The stochastic block model (SBM) is a random graph model with different group of vertices connecting differently. It is widely employed as a canonical model to study clustering and community detection, and provides a fertile ground to study the information-theoretic and computational tradeoffs that arise in combinatorial statistics and more generally data science. This monograph surveys the recent developments that establish the fundamental limits for community detection in the SBM, both with respect to information-theoretic and computational tradeoffs, and for various recovery requirements such as exact, partial and weak recovery. The main results discussed are the phase transitions for exact recovery at the Chernoff-Hellinger threshold, the phase transition for weak recovery at the Kesten-Stigum threshold, the optimal SNR-mutual information tradeoff for partial recovery, and the gap between information-theoretic and computational thresholds. The monograph gives a principled derivation of the main algorithms developed in the quest of achieving the limits, in particular two-round algorithms via graph-splitting, semi-definite programming, (linearized) belief propagation, classical/nonbacktracking spectral methods and graph powering. Extensions to other block models, such as geometric block models, and a few open problems are also discussed. Copyright © 2017, The Authors. All rights reserved.

关键词： Recovery

A Comparative Analysis of Optimization and Generalization Properties of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wu, Lei Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

A fairly comprehensive analysis is presented for the gradient descent dynamics for training two-layer neural network models in the situation when the parameters in both layers are updated. General initialization schemes as well as general regimes for the network width and training data size are considered. In the over-parametrized regime, it is shown that gradient descent dynamics can achieve zero training loss exponentially fast regardless of the quality of the labels. In addition, it is proved that throughout the training process the functions represented by the neural network model are uniformly close to that of a kernel method. For general values of the network width and training data size, sharp estimates of the generalization error is established for target functions in the appropriate reproducing kernel Hilbert *** Codes 41A99, 49M99 Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods

A Priori Estimates of the Population Risk for Residual Networks

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wang, Qingcan Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

Optimal a priori estimates are derived for the population risk, also known as the generalization error, of a regularized residual network model. An important part of the regularized model is the usage of a new path norm, called the weighted path norm, as the regularization term. The weighted path norm treats the skip connections and the nonlinearities differently so that paths with more nonlinearities are regularized by larger weights. The error estimates are a priori in the sense that the estimates depend only on the target function, not on the parameters obtained in the training process. The estimates are optimal, in a high dimensional setting, in the sense that both the bound for the approximation and estimation errors are comparable to the Monte Carlo error rates. A crucial step in the proof is to establish an optimal bound for the Rademacher complexity of the residual networks. Comparisons are made with existing norm-based generalization error bounds. Copyright © 2019, The Authors. All rights reserved.

关键词： Error analysis

Machine learning from a continuous viewpoint

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wu, Lei Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, very much in the spirit of classical numerical analysis and statistical physics. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the shallow neural network model and the residual neural network model, can all be recovered as particular discretizations of different continuous formulations. We also present examples of new models, such as the ow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

LATTICE-VALUED BOTTLENECK DUALITY

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ghrist, Robert Gould, Julian Lopez, Miguel Department of Mathematics Department of Electrical & Systems Engineering and Program in Applied Mathematics & Computational Science University of Pennsylvania United States

This note reformulates certain classical combinatorial duality theorems in the context of order lattices. For source-target networks, we generalize bottleneck path-cut and flow-cut duality results to edges with capacities in a distributive lattice. For posets, we generalize a bottleneck version of Dilworth’s theorem, again weighted in a distributive lattice. These results are applicable to a wide array of non-numerical network flow problems, as shown. All results, proofs, and applications were created in collaboration with AI language models. An appendix documents their role and *** Codes 06D05, 90C35, 06A07 © 2024, CC BY.

关键词： Combinatorial optimization

Probabilistic Theory of Mean Field Games with Applications II 1

学校读者我要写书评

暂无评论

丛书名： Probability Theory and Stochastic Modelling

1000年

作者： René Carmona François Delarue

ISBN: (数字)9783319564364

ISBN: (纸本)9783319564357;9783030132590

Together, both Volume I and Volume II will greatly benefit mathematical graduate students and researchers interested in mean field games. The authors provide a detailed road map through the book allowing different access points for different readers and building up the level of technical detail. The accessible approach and overview will allow interested researchers in the applied sciences to obtain a clear overview of the state of the art in mean field games.

关键词： Probability Theory and Stochastic Processes Calculus of Variations and Optimal Control Optimization Partial Differential Equations Economic Theory/Quantitative Economics/Mathematical Methods

Analysis of the gradient descent algorithm for a deep neural network model with skip-connections

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wang, Qingcan Wu, Lei Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

The behavior of the gradient descent (GD) algorithm is analyzed for a deep neural network model with skip-connections. It is proved that in the over-parametrized regime, for a suitable initialization, with high probability GD can find a global minimum exponentially fast. Generalization error estimates along the GD path are also established. As a consequence, it is shown that when the target function is in the reproducing kernel Hilbert space (RKHS) with a kernel defined by the initialization, there exist generalizable earlystopping solutions along the GD path. In addition, it is also shown that the GD path is uniformly close to the functions given by the related random feature model. Consequently, in this \implicit regularization" setting, the deep neural network model deteriorates to a random feature model. Our results hold for neural networks of any width larger than the input dimension. Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods