检索结果-内蒙古大学图书馆

A statistical mechanics framework for constructing non-equilibrium thermodynamic models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Leadbetter, Travis Purohit, Prashant K. Reina, Celia Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Far-from-equilibrium phenomena are critical to all natural and engineered systems, and essential to biological processes responsible for life. For over a century and a half, since Carnot, Clausius, Maxwell, Boltzmann, and Gibbs, among many others, laid the foundation for our understanding of equilibrium processes, scientists and engineers have dreamed of an analogous treatment of non-equilibrium systems. But despite tremendous efforts, a universal theory of non-equilibrium behavior akin to equilibrium statistical mechanics and thermodynamics has evaded description. Several methodologies have proved their ability to accurately describe complex non-equilibrium systems at the macroscopic scale, but their accuracy and predictive capacity is predicated on either phenomenological kinetic equations fit to microscopic data, or on running concurrent simulations at the particle level. Instead, we provide a framework for deriving stand-alone macroscopic thermodynamics models directly from microscopic physics without fitting in overdamped Langevin systems. The only necessary ingredient is a functional form for a parameterized, approximate density of states, in analogy to the assumption of a uniform density of states in the equilibrium microcanonical ensemble. We highlight this framework's effectiveness by deriving analytical approximations for evolving mechanical and thermodynamic quantities in a model of coiled-coil proteins and double stranded DNA, thus producing, to the authors' knowledge, the first derivation of the governing equations for a phase propagating system under general loading conditions without appeal to phenomenology. The generality of our treatment allows for application to any system described by Langevin dynamics with arbitrary interaction energies and external driving, including colloidal macromolecules, hydrogels, and biopolymers. © 2023, CC BY.

关键词： Statistical mechanics

LEARNING ONLY ON BOUNDARIES: A PHYSICS-INFORMED NEURAL OPERATOR FOR SOLVING PARAMETRIC PARTIAL DIFFERENTIAL EQUATIONS IN COMPLEX GEOMETRIES

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fang, Zhiwei Wang, Sifan Perdikaris, Paris Graduate Group in Applied Mathematics Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Recently deep learning surrogates and neural operators have shown promise in solving partial differential equations (PDEs). However, they often require a large amount of training data and are limited to bounded domains. In this work, we present a novel physics-informed neural operator method to solve parametrized boundary value problems without labeled data. By reformulating the PDEs into boundary integral equations (BIEs), we can train the operator network solely on the boundary of the domain. This approach reduces the number of required sample points from O(Nd) to O(Nd−1), where d is the domain's dimension, leading to a significant acceleration of the training process. Additionally, our method can handle unbounded problems, which are unattainable for existing physics-informed neural networks (PINNs) and neural operators. Our numerical experiments show the effectiveness of parametrized complex geometries and unbounded problems. Copyright © 2023, The Authors. All rights reserved.

关键词： Boundary element method

ENSEMBLE LEARNING FOR PHYSICS INFORMED NEURAL NETWORKS: A GRADIENT BOOSTING APPROACH

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fang, Zhiwei Wang, Sifan Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, conventional PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given partial differential equation (PDE) using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs. © 2023, CC BY-NC-SA.

关键词： Numerical methods

An Expert's Guide to Training Physics-informed Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Sifan Wang, Hanwen Sankaran, Shyam Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Physics-informed neural networks (PINNs) have been popularized as a deep learning framework that can seamlessly synthesize observational data and partial differential equation (PDE) constraints. Their practical effectiveness however can be hampered by training pathologies, but also oftentimes by poor choices made by users who lack deep learning expertise. In this paper we present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs. We also put forth a series of challenging benchmark problems that highlight some of the most prominent difficulties in training PINNs, and present comprehensive and fully reproducible ablation studies that demonstrate how different architecture choices and training strategies affect the test accuracy of the resulting models. We show that the methods and guiding principles put forth in this study lead to state-of-the-art results and provide strong baselines that future studies should use for comparison purposes. To this end, we also release a highly optimized library in JAX that can be used to reproduce all results reported in this paper, enable future research studies, as well as facilitate easy adaptation to new use-case scenarios. © 2023, CC BY-NC-SA.

关键词： Deep learning

Moving from Machine Learning to Statistics: the case of Expected Points in American football

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Brill, Ryan S. Yee, Ryan Deshpande, Sameer K. Wyner, Abraham J. Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Dept. of Statistics University of Wisconsin–Madison United States Dept. of Statistics and Data Science The Wharton School University of Pennsylvania United States

Expected points is a value function fundamental to player evaluation and strategic in-game decision-making across sports analytics, particularly in American football. To estimate expected points, football analysts use machine learning tools, which are not equipped to handle certain challenges. They suffer from selection bias, display counter-intuitive artifacts of overfitting, do not quantify uncertainty in point estimates, and do not account for the strong dependence structure of observational football data. These issues are not unique to American football or even sports analytics;they are general problems analysts encounter across various statistical applications, particularly when using machine learning in lieu of traditional statistical models. We explore these issues in detail and devise expected points models that account for them. We also introduce a widely applicable novel methodological approach to mitigate overfitting, using a catalytic prior to smooth our machine learning models. © 2024, CC BY.

关键词： Adversarial machine learning

Risk-Aware Stochastic Control of a Sailboat

学校读者我要写书评

暂无评论

Risk-Aware Stochastic Control of a Sailboat

American Control Conference (ACC)

作者： MingYi Wang Natasha Patnaik Anne Somalwar Jingyi Wu Alexander Vladimirsky The Center for Applied Mathematics Cornell University Ithaca NY USA The Department of Computational Applied Mathematics and Operations Research Rice University Houston TX USA The graduate group in Applied Mathematics and Computational Science University of Pennsylvania Philadelphia PA USA The Center for Data Science New York University New York City NY USA Department of Mathematics Cornell University Ithaca NY USA

ISBN: (数字)9798350382655

ISBN: (纸本)9798350382662

Sailboat path-planning is a natural hybrid control problem (due to continuous steering and occasional “tack-switching” maneuvers), with the actual path-to-target greatly affected by stochastically evolving wind conditions. Previous studies have focused on finding risk-neutral policies that minimize the expected time of arrival. In contrast, we present a robust control approach, which maximizes the probability of arriving before a specified deadline/threshold. Our numerical method recovers the optimal risk-aware (and threshold-specific) policies for all initial sailboat positions and a broad range of thresholds simultaneously. This is accomplished by solving two quasi-variational inequalities based on second-order Hamilton-Jacobi-Bellman (HJB) PDEs with degenerate parabolicity. Monte-Carlo simulations show that risk-awareness in sailing is particularly useful when a carefully calculated bet on the evolving wind direction might yield a reduction in the number of tack-switches.

关键词： Robust control Geometry Monte Carlo methods Shape Stochastic processes Boats Differential games

Removing data heterogeneity influence enhances network topology dependence of decentralized SGD

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2023年第1期24卷 13275-13327页

作者： Kun Yuan Sulaiman A. Alghunaim Xinmeng Huang Center for Machine Learning Research Peking University AI for Science Institute Beijing P. R. China Department of Electrical Engineering Kuwait University Safat Kuwait Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania Philadelphia PA

We consider decentralized stochastic optimization problems, where a network of n nodes cooperates to find a minimizer of the globally-averaged cost. A widely studied decentralized algorithm for this problem is the decentralized SGD (D-SGD), in which each node averages only with its neighbors. D-SGD is efficient in single-iteration communication, but it is very sensitive to the network topology. For smooth objective functions, the transient stage (which measures the number of iterations the algorithm has to experience before achieving the linear speedup stage) of D-SGD is on the order of O(n/(1 - β)2) and O(n3/(1 - β)4) for strongly and generally convex cost functions, respectively, where 1 - β ∈ (0, 1) is a topology-dependent quantity that approaches 0 for a large and sparse network. Hence, D-SGD suffers from slow convergence for large and sparse *** this work, we revisit the convergence property of the D2/Exact-Diffusion algorithm. By eliminating the influence of data heterogeneity between nodes, D2/Exact-diffusion is shown to have an enhanced transient stage that is on the order of Õ(n/(1 - β)) and O(n3/(1-β)2) for strongly and generally convex cost functions (where Õ (·) hides all logarithm factors), respectively. Moreover, when D2/Exact-Diffusion is implemented with both gradient accumulation and multi-round gossip communications, its transient stage can be further improved to Õ (1/(1-β)1/2) and Õ(n/(1-β)) for strongly and generally convex cost functions, respectively. To our knowledge, these established results for D2/Exact-Diffusion have the best (i.e., weakest) dependence on network topology compared to existing decentralized algorithms. Numerical simulations are conducted to validate our theories.

关键词： decentralized optimization stochastic optimization transient stage

SCALABLE BAYESIAN OPTIMIZATION WITH HIGH-DIMENSIONAL OUTPUTS USING RANDOMIZED PRIOR NETWORKS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Bhouri, Mohamed Aziz Joly, Michael Yu, Robert Sarkar, Soumalya Perdikaris, Paris Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States Raytheon Technologies Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States

Several fundamental problems in science and engineering consist of global optimization tasks involving unknown high-dimensional (black-box) functions that map a set of controllable variables to the outcomes of an expensive experiment. Bayesian Optimization (BO) techniques are known to be effective in tackling global optimization problems using a relatively small number objective function evaluations, but their performance suffers when dealing with high-dimensional outputs. To overcome the major challenge of dimensionality, here we propose a deep learning framework for BO and sequential decision making based on bootstrapped ensembles of neural architectures with randomized priors. Using appropriate architecture choices, we show that the proposed framework can approximate functional relationships between design variables and quantities of interest, even in cases where the latter take values in high-dimensional vector spaces or even infinite-dimensional function spaces. In the context of BO, we augmented the proposed probabilistic surrogates with re-parameterized Monte Carlo approximations of multiple-point (parallel) acquisition functions, as well as methodological extensions for accommodating black-box constraints and multi-fidelity information sources. We test the proposed framework against state-of-the-art methods for BO and demonstrate superior performance across several challenging tasks with high-dimensional outputs, including a constrained multi-fidelity optimization task involving shape optimization of rotor blades in turbo-machinery. © 2023, CC BY.

关键词： Deep learning

A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Moniri, Behrad Lee, Donghwan Hassani, Hamed Dobriban, Edgar Department of Electrical and Systems Engineering University of Pennsylvania United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Department of Statistics and Data Science University of Pennsylvania United States

Feature learning is thought to be one of the fundamental reasons for the success of deep neural networks. It is rigorously known that in two-layer fully-connected neural networks under certain conditions, one step of gradient descent on the first layer can lead to feature learning;characterized by the appearance of a separated rank-one component—spike—in the spectrum of the feature matrix. However, with a constant gradient descent step size, this spike only carries information from the linear component of the target function and therefore learning non-linear components is impossible. We show that with a learning rate that grows with the sample size, such training in fact introduces multiple rank-one components, each corresponding to a specific polynomial feature. We further prove that the limiting large-dimensional and large sample training and test errors of the updated neural networks are fully characterized by these spikes. By precisely analyzing the improvement in the training and test errors, we demonstrate that these non-linear features can enhance learning. © 2023, CC BY.

关键词： Deep neural networks