检索结果-内蒙古大学图书馆

A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics

引用

Science China mathematics 2020年第7期63卷 1235-1258页

作者： Weinan E Chao Ma Lei Wu Department of Mathematics Princeton UniversityPrincetonNJ 08544USA The Program in Applied and Computational Mathematics Princeton UniversityNJ 08544USA Beijing Institute of Big Data Research Beijing 100871China

A fairly comprehensive analysis is presented for the gradient descent dynamics for training two-layer neural network models in the situation when the parameters in both layers are *** initialization schemes as well as general regimes for the network width and training data size are *** the overparametrized regime,it is shown that gradient descent dynamics can achieve zero training loss exponentially fast regardless of the quality of the *** addition,it is proved that throughout the training process the functions represented by the neural network model are uniformly close to those of a kernel *** general values of the network width and training data size,sharp estimates of the generalization error are established for target functions in the appropriate reproducing kernel Hilbert space.

关键词： two-layer neural network random feature model Gram matrix generalization error

来源：评论

学校读者我要写书评

暂无评论

Machine learning from a continuous viewpoint,I

引用

Science China mathematics 2020年第11期63卷 2233-2266页

作者： Weinan E Chao Ma Lei Wu Department of Mathematics Princeton UniversityPrincetonNJ 08544USA The Program in Applied and Computational Mathematics Princeton UniversityPrincetonNJ 08544USA Beijing Institute of Big Data Research Beijing 100871China

We present a continuous formulation of machine learning,as a problem in the calculus of variations and differential-integral equations,in the spirit of classical numerical *** demonstrate that conventional machine learning models and algorithms,such as the random feature model,the two-layer neural network model and the residual neural network model,can all be recovered(in a scaled form)as particular discretizations of different continuous *** also present examples of new models,such as the flow-based random feature model,and new algorithms,such as the smoothed particle method and spectral method,that arise naturally from this continuous *** discuss how the issues of generalization error and implicit regularization can be studied under this framework.

关键词： machine learning continuous formulation flow-based model gradient flow particle approximation

来源：评论

学校读者我要写书评

暂无评论

Faithful recovery of vector valued functions from incomplete data recolorization and art restoration

引用

1st International Conference on Scale Space and Variational Methods in Computer Vision, SSVM 2007

作者： Fornasier, Massimo Program in Applied and Computational Mathematics Princeton University Princeton NJ 08544 United States

ISBN: (纸本)9783540728221

On March 11, 1944, the famous Eremitani Church in Padua (Italy) was destroyed in an Allied bombing along with the inestimable frescoes by Andrea Mantegna et al. contained in the Ovetari Chapel. In the last 60 years, several attempts have been made to restore the fresco fragments by traditional methods, but without much success. We have developed an efficient pattern recognition algorithm to map the original position and orientation of the fragments, based on comparisons with an old gray level image of the fresco prior to the damage. This innovative technique allowed for the partial reconstruction of the frescoes. Unfortunately, the surface covered by the fragments is only 77 m 2, while the original area was of several hundreds. This means that we can reconstruct only a fraction (less than 8%) of this inestimable artwork. In particular the original color of the blanks is not known. This begs the question of whether it is possible to estimate mathematically the original colors of the frescoes by making use of the potential information given by the available fragments and the gray level of the pictures taken before the damage. Can one estimate how faithful such restoration is? In this paper we retrace the development of the recovery of the frescoes as an inspiring and challenging real-life problem for the development of new mathematical methods. We introduce two models for the recovery of vector valued functions from incomplete data, with applications to the fresco recolorization problem. The models are based on the minimization of a functional which is formed by the discrepancy with respect to the data and additional regularization constraints. The latter refer to joint sparsity measures with respect to frame expansions for the first functional and functional total variation for the second. We establish the relations between these two models. As a byproduct we develop the basis of a theory of fidelity in color recovery, which is a crucial issue in art restoration and

关键词： Vectors

来源：评论

学校读者我要写书评

暂无评论

Some observations on high-dimensional partial differential equations with Barron data 2

Some observations on high-dimensional partial differential e...

引用

2nd Mathematical and Scientific Machine Learning Conference, MSML 2021

作者： Weinan, E. Wojtowytsch, Stephan Program for Applied and Computational Mathematics Princeton University PrincetonNJ08544 United States

We use explicit representation formulas to show that solutions to certain partial differential equations lie in Barron spaces or multilayer spaces if the PDE data lie in such function spaces. Consequently, these solutions can be represented efficiently using artificial neural networks, even in high dimension. Conversely, we present examples in which the solution fails to lie in the function space associated to a neural network under consideration. © 2021 W. E & S. Wojtowytsch.

关键词： Partial differential equations

来源：评论

学校读者我要写书评

暂无评论

On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers 2

On the emergence of simplex symmetry in the final and penult...

引用

2nd Mathematical and Scientific Machine Learning Conference, MSML 2021

作者： Weinan, E. Wojtowytsch, Stephan Program for Applied and Computational Mathematics Princeton University PrincetonNJ08544 United States

A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer. Namely, if h(x) = Af(x) + b where A is a linear map and f is the output of the penultimate layer of the network (after activation), then all data points xi1, . . ., xi,Ni in a class Ci are mapped to a single point yi by f and the points yi are located at the vertices of a regular k − 1-dimensional standard simplex in a high-dimensional Euclidean space. We explain this observation analytically in toy models for highly expressive deep neural networks. In complementary examples, we demonstrate rigorously that even the final output of the classifier h is not uniform over data samples from a class Ci if h is a shallow network (or if the deeper layers do not bring the data samples into a convenient geometric configuration). © 2021 W. E & S. Wojtowytsch.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Effective bifurcation analysis: a time-stepper-based approach (vol 15, pg 491, 2002)

引用

NONLINEARITY 2002年第3期15卷 955-955页

作者： Runborg, O Theodoropoulos, C Kevrekidis, IG Program in Applied and Computational Mathematics Princeton University Princeton NJ 08544 USA Department of Chemical Engineering Princeton University Princeton NJ 08544 USA

We introduce a numerical approach to perform the effective (coarse-scale) bifurcation analysis of solutions of dissipative evolution equations with spatially varying coefficients. The advantage of this approach is that the `coarse model' (the averaged, effective equation) need not be explicitly constructed. The method only uses a time-integrator code for the detailed problem and judicious choices of initial data and integration times; the bifurcation computations are based on the so-called recursive projection method (Shroff and Keller 1993 SIAM J. Numer. Anal. 30 1099-120).

关键词：

来源：评论

学校读者我要写书评

暂无评论

On the particle approximation of lagged Feynman–Kac formulae

引用

Stochastic Processes and their Applications 2025年 188卷

作者： Awadelkarim, Elsiddig Caffarel, Michel Del Moral, Pierre Jasra, Ajay Applied Mathematics and Computational Science Program Computer Electrical and Mathematical Sciences and Engineering Division King Abdullah University of Science and Technology Thuwal 23955-6900 Saudi Arabia University of Toulouse CNRS-Lab. de Chimie et Physique Quantiques Toulouse 31062 France Centre de Recherche Inria Bordeaux Sud-Ouest Talence 33405 France School of Data Science The Chinese University of Hong Kong Shenzhen Shenzhen China

In this paper we examine the numerical approximation of the limiting invariant measure associated with Feynman–Kac formulae. These are expressed in a discrete time formulation and are associated with a Markov chain and a potential function. The typical application considered here is the computation of eigenvalues associated with non-negative operators as found, for example, in physics or particle simulation of rare-events. We focus on a novel lagged approximation of this invariant measure, based upon the introduction of a ratio of time-averaged Feynman–Kac marginals associated with a positive operator iterated l∈N times;a lagged Feynman–Kac formula. This estimator and its approximation using Diffusion Monte Carlo (DMC) are commonly used in the physics literature. In short, DMC is an iterative algorithm involving N∈N particles or walkers simulated in parallel, that undergo sampling and resampling operations. In this work, it is shown that for the DMC approximation of the lagged Feynman–Kac formula, one has an almost sure characterization of the L₁-error as the time parameter (iteration) goes to infinity and this is at most of O(exp{−κl}/N), for κ>0. In addition a non-asymptotic in time, and time uniform L₁−bound is proved which is O(l/N). We also prove a novel central limit theorem to give a characterization of the exact asymptotic in time variance. This analysis demonstrates that the strategy used in physics, namely, to run DMC with N and l small and, for long time enough, is mathematically justified. Our results also suggest how one should choose N and l in practice. We emphasize that these results are not restricted to physical applications;they have broad relevance to the general problem of particle simulation of the Feynman–Kac formula, which is utilized in a great variety of scientific and engineering fields. © 2025 Elsevier B.V.

关键词： Diffusion Monte Carlo Eigenvalue approximation Feynman–Kac formula

来源：评论

学校读者我要写书评

暂无评论

Variational Boundary Conditions for Molecular Dynamics Simulations of Solids at Low Temperature

引用

Communications in computational Physics 2006年第1期1卷 135-175页

作者： Xiantao Li Weinan E Department of Mathematics Pennsylvania State UniversityUniversity ParkPA 16802USA Department of Mathematics&Program in Applied and Computational Mathematics Princeton UniversityPrincetonNJ 08544USA

Boundary conditions for molecular dynamics simulation of crystalline solids are considered with the objective of eliminating the reflection of phonons.A variational formalism is presented to construct boundary conditions that minimize total phonon *** boundary conditions that involve a few neighbors of the boundary atoms and limited number of time steps are found using the variational *** effects are studied and compared with other boundary conditions such as truncated exact boundary conditions or by appending border atoms where artificial damping forces are *** general it is found that,with the same cost or complexity,the variational boundary conditions perform much better than the truncated exact boundary conditions or by appending border atoms with empirical damping *** issues of implementation are discussed for real *** to brittle fracture dynamics is illustrated.

关键词： Boundary condition molecular dynamics phonon reflection crack propagation

来源：评论

学校读者我要写书评

暂无评论

Absolute change pivot rule for the simplex algorithm

Absolute change pivot rule for the simplex algorithm

引用

International MultiConference of Engineers and Computer Scientists, IMECS 2014

作者： Chankong, Kittiphong Intiyot, Boonyarit Sinapiromsaran, Krung Applied Mathematics and Computational Science Program Department of Mathematics and Computer Science Faculty of Science Chulalongkorn University Bangkok Thailand

ISBN: (纸本)9789881925336

The simplex algorithm is a widely used method for solving a linear programming problem (LP) which is first presented by George B. Dantzig. One of the important steps of the simplex algorithm is applying an appropriate pivot rule, the rule to select the entering variable. An effective pivot rule can lead to the optimal solution of LP with the small number of iterations. In a minimization problem, Dantzig's pivot rule selects an entering variable corresponding to the most negative reduced cost. The concept is to have the maximum improvement in the objective value per unit step of the entering variable. However, in some problems, Dantzig's rule may visit a large number of extreme points before reaching the optimal solution. In this paper, we propose a pivot rule that could reduce the number of such iterations over the Dantzig's pivot rule. The idea is to have the maximum improvement in the objective value function by trying to block a leaving variable that makes a little change in the objective function value as much as possible. Then we test and compare the efficacy of this rule with Dantzig' original rule.

关键词： Linear programming

来源：评论

学校读者我要写书评

暂无评论

Generalization and Memorization: The Bias Potential Model 2

Generalization and Memorization: The Bias Potential Model

引用

2nd Mathematical and Scientific Machine Learning Conference, MSML 2021

作者： Yang, Hongkang Weinan, E. Program in Applied and Computational Mathematics Princeton University United States Department of Mathematics and PACM Princeton University United States

Models for learning probability distributions such as generative models and density estimators behave quite differently from models for learning functions. One example is found in the memorization phenomenon, namely the ultimate convergence to the empirical distribution, that occurs in generative adversarial networks (GANs). For this reason, the issue of generalization is more subtle than that for supervised learning. For the bias potential model, we show that dimension-independent generalization accuracy is achievable if early stopping is adopted, despite that in the long term, the model either memorizes the samples or diverges. © 2021 H. Yang & W. E.

关键词： Probability distributions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：