检索结果-内蒙古大学图书馆

AN OPTIMAL SCHEDULED LEARNING RATE FOR A RANDOMIZED KACZMARZ ALGORITHM

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Marshall, Nicholas F. Mickelin, Oscar Department of Mathematics Oregon State University United States Program in Applied and Computational Mathematics Princeton University United States

We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving Ax ≈ b + Ε, where Ax = b is a consistent linear system and Ε has independent mean zero random entries. We derive a learning rate schedule which optimizes a bound on the expected error that is sharp in certain cases;in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert-W function of an exponential. Copyright © 2022, The Authors. All rights reserved.

关键词： Linear systems

End-to-end symmetry preserving inter-atomic potential energy model for finite and extended systems 18

学校读者我要写书评

暂无评论

End-to-end symmetry preserving inter-atomic potential energy...

Proceedings of the 32nd International Conference on Neural Information Processing Systems

作者： Linfeng Zhang Jiequn Han Han Wang Wissam A. Saidi Roberto Car E. Weinan Program in Applied and Computational Mathematics Princeton University Institute of Applied Physics and Computational Mathematics China and CAEP Software Center for High Performance Numerical Simulation China Department of Mechanical Engineering and Materials Science University of Pittsburgh Program in Applied and Computational Mathematics Princeton University and Department of Chemistry and Department of Physics Princeton University and Princeton Institute for the Science and Technology of Materials Princeton University Program in Applied and Computational Mathematics Princeton University and Department of Mathematics Princeton University and Beijing Institute of Big Data Research China

Machine learning models are changing the paradigm of molecular modeling, which is a fundamental tool for material science, chemistry, and computational biology. Of particular interest is the inter-atomic potential energy surface (PES). Here we develop Deep Potential - Smooth Edition (DeepPot-SE), an end-to-end machine learning-based PES model, which is able to efficiently represent the PES of a wide variety of systems with the accuracy of ab initio quantum mechanics models. By construction, DeepPot-SE is extensive and continuously differentiable, scales linearly with system size, and preserves all the natural symmetries of the system. Further, we show that DeepPot-SE describes finite and extended systems including organic molecules, metals, semiconductors, and insulators with high fidelity.

关键词：

Efficient Numerical Algorithm for Large-Scale Damped Natural Gradient Descent

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Yixiao Xie, Hao Wang, Han Program in Applied and Computational Mathematics Princeton University United States Beijing National Laboratory for Condensed Matter Physics Institute of Physics Chinese Academy of Sciences China Laboratory of Computational Physics Institute of Applied Physics and Computational Mathematics

We propose a new algorithm for efficiently solving the damped Fisher matrix in large-scale scenarios where the number of parameters significantly exceeds the number of available samples. This problem is fundamental for natural gradient descent and stochastic reconfiguration. Our algorithm is based on Cholesky decomposition and is generally applicable. Benchmark results show that the algorithm is significantly faster than existing methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Stochastic systems

Community Detection and Stochastic Block Models

学校读者我要写书评

暂无评论

arXiv 2017年

作者： Abbe, Emmanuel Program in Applied and Computational Mathematics Department of Electrical Engineering Princeton University Princeton United States

The stochastic block model (SBM) is a random graph model with different group of vertices connecting differently. It is widely employed as a canonical model to study clustering and community detection, and provides a fertile ground to study the information-theoretic and computational tradeoffs that arise in combinatorial statistics and more generally data science. This monograph surveys the recent developments that establish the fundamental limits for community detection in the SBM, both with respect to information-theoretic and computational tradeoffs, and for various recovery requirements such as exact, partial and weak recovery. The main results discussed are the phase transitions for exact recovery at the Chernoff-Hellinger threshold, the phase transition for weak recovery at the Kesten-Stigum threshold, the optimal SNR-mutual information tradeoff for partial recovery, and the gap between information-theoretic and computational thresholds. The monograph gives a principled derivation of the main algorithms developed in the quest of achieving the limits, in particular two-round algorithms via graph-splitting, semi-definite programming, (linearized) belief propagation, classical/nonbacktracking spectral methods and graph powering. Extensions to other block models, such as geometric block models, and a few open problems are also discussed. Copyright © 2017, The Authors. All rights reserved.

关键词： Recovery

Multiscale simulations in simple metals: A density-functional-based methodology

学校读者我要写书评

暂无评论

Physical Review B 2005年第9期71卷 094101-094101页

作者： Nicholas Choly Gang Lu Weinan E Efthimios Kaxiras Division of Engineering and Applied Sciences Harvard University Cambridge Massachusetts 02138 USA Department of Mathematics and Program in Applied and Computational Mathematics Princeton University Princeton New Jersey 08544 USA

We present a formalism for coupling a density-functional-theory-based quantum simulation to a classical simulation for the treatment of simple metallic systems. The formalism is applicable to multiscale simulations in which the part of the system requiring quantum-mechanical treatment is spatially confined to a small region. Such situations often arise in physical systems where chemical interactions in a small region can affect the macroscopic mechanical properties of a metal. We describe how this coupled treatment can be accomplished efficiently, and we present a coupled simulation for a bulk aluminum system.

关键词：

A Comparative Analysis of Optimization and Generalization Properties of Two-layer Neural Network and Random Feature Models Under Gradient Descent Dynamics

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wu, Lei Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

A fairly comprehensive analysis is presented for the gradient descent dynamics for training two-layer neural network models in the situation when the parameters in both layers are updated. General initialization schemes as well as general regimes for the network width and training data size are considered. In the over-parametrized regime, it is shown that gradient descent dynamics can achieve zero training loss exponentially fast regardless of the quality of the labels. In addition, it is proved that throughout the training process the functions represented by the neural network model are uniformly close to that of a kernel method. For general values of the network width and training data size, sharp estimates of the generalization error is established for target functions in the appropriate reproducing kernel Hilbert *** Codes 41A99, 49M99 Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods

A Priori Estimates of the Population Risk for Residual Networks

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wang, Qingcan Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

Optimal a priori estimates are derived for the population risk, also known as the generalization error, of a regularized residual network model. An important part of the regularized model is the usage of a new path norm, called the weighted path norm, as the regularization term. The weighted path norm treats the skip connections and the nonlinearities differently so that paths with more nonlinearities are regularized by larger weights. The error estimates are a priori in the sense that the estimates depend only on the target function, not on the parameters obtained in the training process. The estimates are optimal, in a high dimensional setting, in the sense that both the bound for the approximation and estimation errors are comparable to the Monte Carlo error rates. A crucial step in the proof is to establish an optimal bound for the Rademacher complexity of the residual networks. Comparisons are made with existing norm-based generalization error bounds. Copyright © 2019, The Authors. All rights reserved.

关键词： Error analysis

Machine learning from a continuous viewpoint

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Weinan, E. Ma, Chao Wu, Lei Department of Mathematics Princeton University Program in Applied and Computational Mathematics Princeton University Beijing Institute of Big Data Research

We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, very much in the spirit of classical numerical analysis and statistical physics. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the shallow neural network model and the residual neural network model, can all be recovered as particular discretizations of different continuous formulations. We also present examples of new models, such as the ow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework. Copyright © 2019, The Authors. All rights reserved.

关键词： Machine learning

LATTICE-VALUED BOTTLENECK DUALITY

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ghrist, Robert Gould, Julian Lopez, Miguel Department of Mathematics Department of Electrical & Systems Engineering and Program in Applied Mathematics & Computational Science University of Pennsylvania United States

This note reformulates certain classical combinatorial duality theorems in the context of order lattices. For source-target networks, we generalize bottleneck path-cut and flow-cut duality results to edges with capacities in a distributive lattice. For posets, we generalize a bottleneck version of Dilworth’s theorem, again weighted in a distributive lattice. These results are applicable to a wide array of non-numerical network flow problems, as shown. All results, proofs, and applications were created in collaboration with AI language models. An appendix documents their role and *** Codes 06D05, 90C35, 06A07 © 2024, CC BY.

关键词： Combinatorial optimization