检索结果-内蒙古大学图书馆

ENSEMBLE LEARNING FOR PHYSICS INFORMED NEURAL NETWORKS: A GRADIENT BOOSTING APPROACH

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fang, Zhiwei Wang, Sifan Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, conventional PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given partial differential equation (PDE) using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs. © 2023, CC BY-NC-SA.

关键词： Numerical methods

RESPECTING CAUSALITY IS ALL YOU NEED FOR TRAINING PHYSICS-INFORMED NEURAL NETWORKS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Sifan Sankaran, Shyam Perdikaris, Paris Graduate Group In Applied Mathematics And Computational Science University Of Pennsylvania PhiladelphiaPA19104 United States Department Of Mechanical Engineering And Applied Mechanics University Of Pennsylvania PhiladelphiaPA19104 United States

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. We provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto-Sivashinsky equation in the chaotic regime, and the Navier-Stokes equations in the turbulent regime. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity. © 2022, CC BY-NC-SA.

关键词： Partial differential equations

When and why Pinns Fail to Train: A Neural Tangent Kernel Perspective

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Sifan Yu, Xinling Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. In this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK);a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms. The data and code accompanying this manuscript are publicly available at https://***/PredictiveIntelligenceLab/ PINNsNTK. Copyright © 2020, The Authors. All rights reserved.

关键词： Gradient methods

DEEP LEARNING ALTERNATIVES OF THE KOLMOGOROV SUPERPOSITION THEOREM

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guilhoto, Leonardo Ferreira Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanical Engineering & Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

This paper explores alternative formulations of the Kolmogorov Superposition Theorem (KST) as a foundation for neural network design. The original KST formulation, while mathematically elegant, presents practical challenges due to its limited insight into the structure of inner and outer functions and the large number of unknown variables it introduces. Kolmogorov-Arnold Networks (KANs) leverage KST for function approximation, but they have faced scrutiny due to mixed results compared to traditional multilayer perceptrons (MLPs) and practical limitations imposed by the original KST formulation. To address these issues, we introduce ActNet, a scalable deep learning model that builds on the KST and overcomes many of the drawbacks of Kolmogorov’s original formulation. We evaluate ActNet in the context of Physics-Informed Neural Networks (PINNs), a framework well-suited for leveraging KST’s strengths in low-dimensional function approximation, particularly for simulating partial differential equations (PDEs). In this challenging setting, where models must learn latent functions without direct measurements, ActNet consistently outperforms KANs across multiple benchmarks and is competitive against the current best MLP-based approaches. These results present ActNet as a promising new direction for KST-based deep learning applications, particularly in scientific computing and PDE simulation *** Codes 68T07, 65M99 © 2024, CC BY-NC-SA.

关键词： Deep learning

PRIKRY-TYPE FORCING AND THE SET OF POSSIBLE COFINALITIES

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Tsukuura, Kenta Doctoral Program in Mathematics Degree Programs in Pure and Applied Sciences Graduate School of Science and Technology University of Tsukuba Tsukuba305-8571 Japan

It is known that the set of possible cofinalities pcf(A) has good properties if A is a progressive interval of regular cardinals. In this paper, we give an interval of regular cardinals A such that pcf(A) has no good properties in the presence of a measurable cardinal, or in generic extensions by Prikry-type *** Codes 03E04, 03E35, 03E55 Copyright © 2022, The Authors. All rights reserved.

关键词：

DEEP LEARNING OF FREE BOUNDARY AND STEFAN PROBLEMS

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Sifan Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Free boundary problems appear naturally in numerous areas of mathematics, science and engineering. These problems present a great computational challenge because they necessitate numerical methods that can yield an accurate approximation of free boundaries and complex dynamic interfaces. In this work, we propose a multi-network model based on physics-informed neural networks to tackle a general class of forward and inverse free boundary problems called Stefan problems. Specifically, we approximate the unknown solution as well as any moving boundaries by two deep neural networks. Besides, we formulate a new type of inverse Stefan problems that aim to reconstruct the solution and free boundaries directly from sparse and noisy measurements. We demonstrate the effectiveness of our approach in a series of benchmarks spanning different types of Stefan problems, and illustrate how the proposed framework can accurately recover solutions of partial differential equations with moving boundaries and dynamic interfaces. All code and data accompanying this manuscript are publicly available at https://***/PredictiveIntelligenceLab/DeepStefan. Copyright © 2020, The Authors. All rights reserved.

关键词： Numerical methods

Electron kinetic effects in back-stimulated Raman scattering bursts driven by broadband laser pulses

学校读者我要写书评

暂无评论

Matter and Radiation at Extremes 2024年第4期9卷 54-67页

作者： Q.K.Liu L.Deng Q.Wang X.Zhang F.Q.Meng Y.P.Wang Y.Q.Gao H.B.Cai S.P.Zhu Graduate School China Academy of Engineering PhysicsBeijing 100088China Institute of Applied Physics and Computational Mathematics Beijing 100094China School of Nuclear Science and Technology University of Science and Technology of ChinaHefei 230026China Shanghai Institute of Laser Plasma Shanghai 201800China HEDPS Center for Applied Physics and TechnologyPeking UniversityBeijing 100871China School of Physics Zhejiang UniversityHangzhou 310027China

We examine electron kinetic effects in broadband-laser-driven back-stimulated Raman scattering(BSRS)bursts using particle-in-cell *** bursts occur during the nonlinear stage,causing reflectivity spikes and generating large numbers of hot ***-duration simulations are performed to observe burst events,and a simplified model is developed to eliminate the interference of the broadband laser’s random intensity *** the simplified model,we isolate and characterize the spectrum of electron plasma *** spectrum changes from a sideband structure to a turbulence-like structure during the burst.A significant asymmetry in the spectrum is *** asymmetry is amplified and transferred to electron phase space by high-intensity broadband laser pulses,leading to violent vortex-merging and generation of hot *** proportion of hot electrons increases from 6.76%to 14.7%during a single violent burst *** demonstrate that kinetic effects profoundly influence the BSRS evolution driven by broadband lasers.

关键词： scattering kinetic simplified

COMPOSITE BAYESIAN OPTIMIZATION IN FUNCTION SPACES USING NEON - NEURAL EPISTEMIC OPERATOR NETWORKS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guilhoto, Leonardo Ferreira Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Operator learning is a rising field of scientific computing where inputs or outputs of a machine learning model are functions defined in infinite-dimensional spaces. In this paper, we introduce NEON (Neural Epistemic Operator Networks), an architecture for generating predictions with uncertainty using a single operator network backbone, which presents orders of magnitude less trainable parameters than deep ensembles of comparable performance. We showcase the utility of this method for sequential decision-making by examining the problem of composite Bayesian Optimization (BO), where we aim to optimize a function f = g ◦ h, where h : X → C(Y, ds) is an unknown map which outputs elements of a function space, and g : C(Y, ds) → is a known and cheap-to-compute functional. By comparing our approach to other state-of-the-art methods on toy and real world scenarios, we demonstrate that NEON achieves state-of-the-art performance while requiring orders of magnitude less trainable *** Codes 68T37 © 2024, CC BY-NC-SA.

关键词： Toys

On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Sifan Wang, Hanwen Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. In this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction-diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior. All code an data accompanying this manuscript will be made publicly available at https://***/PredictiveIntelligenceLab/MultiscalePINNs. Copyright © 2020, The Authors. All rights reserved.

关键词： Partial differential equations