检索结果-内蒙古大学图书馆

Federated scientific machine learning for approximating functions and solving differential equations with data heterogeneity

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Handi Liu, Langchen Lu, Lu Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Statistics and Data Science Yale University New HavenCT06511 United States Wu Tsai Institute Yale University New HavenCT06510 United States

By leveraging neural networks, the emerging field of scientific machine learning (SciML) offers novel approaches to address complex problems governed by partial differential equations (PDEs). In practical applications, challenges arise due to the distributed essence of data, concerns about data privacy, or the impracticality of transferring large volumes of data. Federated learning (FL), a decentralized framework that enables the collaborative training of a global model while preserving data privacy, offers a solution to the challenges posed by isolated data pools and sensitive data issues. Here, this paper explores the integration of FL and SciML to approximate complex functions and solve differential equations. We propose two novel models: federated physics-informed neural networks (FedPINN) and federated deep operator networks (FedDeepONet). We further introduce various data generation methods to control the degree of non-independent and identically distributed (non-iid) data and utilize the 1-Wasserstein distance to quantify data heterogeneity in function approximation and PDE learning. We systematically investigate the relationship between data heterogeneity and federated model performance. Additionally, we propose a measure of weight divergence and develop a theoretical framework to establish growth bounds for weight divergence in federated learning compared to traditional centralized learning. To demonstrate the effectiveness of our methods, we conducted 10 experiments, including 2 on function approximation, 5 PDE problems on FedPINN, and 3 PDE problems on FedDeepONet. These experiments demonstrate that proposed federated methods surpass the models trained only using local data and achieve competitive accuracy of centralized models trained using all data. © 2024, CC BY-NC-ND.

关键词： Federated learning

SURREAL-GAN:SEMI-SUPERVISED REPRESENTATION LEARNING VIA GAN FOR UNCOVERING HETEROGENEOUS DISEASE-RELATED IMAGING PATTERNS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Zhijian Wen, Junhao Davatzikos, Christos Center for Biomedical Image Computing and Analytics University of Pennsylvania United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States

A plethora of machine learning methods have been applied to imaging data, enabling the construction of clinically relevant imaging signatures of neurological and neuropsychiatric diseases. Oftentimes, such methods do not explicitly model the heterogeneity of disease effects, or approach it via nonlinear models that are not interpretable. Moreover, unsupervised methods may parse heterogeneity that is driven by nuisance confounding factors that affect global brain structure or function, rather than heterogeneity relevant to a pathology of interest. On the other hand, semi-supervised clustering methods seek to derive a pathology-related dichotomous subtypes, ignoring the truth that disease heterogeneity spatially and temporally extends along a continuum. To address the aforementioned limitations, herein, we propose a novel method, termed Surreal-GAN (Semi-SUpeRvised ReprEsentAtion Learning via GAN). Using cross-sectional imaging data, Surreal-GAN dissects underlying disease-related heterogeneity under the principle of semi-supervised clustering [cluster mappings from the normal control (CN) to patient (PT) domain], proposes a continuously dimensional representation, and infers the disease severity of patients at individual level along each dimension. The model first learns a transformation function from the CN domain to the PT domain with latent variables controlling transformation directions. An inverse mapping function together with regularization on function continuity, pattern orthogonality and monotonicity was also imposed to make sure that the transformation function captures necessarily meaningful imaging patterns with clinical significance. We first validated the model through semi-synthetic experiments, and then demonstrated its potential in capturing biologically plausible imaging patterns in Alzheimer's disease (AD). © 2022, CC BY.

关键词： Pathology

Broadband coplanar-waveguide-based impedance-transformed Josephson parametric amplifier

学校读者我要写书评

暂无评论

Physical Review Research 2024年第1期6卷 L012035-L012035页

作者： Bingcheng Qing Long B. Nguyen Xinyu Liu Hengjiang Ren William P. Livingston Noah Goss Ahmed Hajr Trevor Chistolini Zahra Pedramrazi David I. Santiago Jie Luo Irfan Siddiqi Department of Physics University of California Berkeley California 94720 USA Computational Research Division Lawrence Berkeley National Laboratory Berkeley California 94720 USA Anyon Computing Inc. Emeryville California 94662 USA Graduate Group in Applied Science and Technology University of California at Berkeley Berkeley California 94720 USA

Quantum-limited Josephson parametric amplifiers play a pivotal role in advancing the field of circuit quantum electrodynamics by enabling the fast and high-fidelity measurement of weak microwave signals. Therefore, it is necessary to develop robust parametric amplifiers with low noise, broad bandwidth, and reduced design complexity for microwave detection. However, current broadband parametric amplifiers either have degraded noise performance or rely on complex designs. Here, we present a device based on the broadband impedance-transformed Josephson parametric amplifier that integrates a hornlike coplanar waveguide transmission line, which significantly decreases the design and fabrication complexity while keeping comparable performance. The device shows an instantaneous bandwidth of 700 (200) MHz for 15 (20) dB gain with an average input saturation power of −110 dBm and near quantum-limited added noise. The operating frequency can be tuned over 1.4 GHz using an external flux bias. We further demonstrate the negligible backaction from our device on a transmon qubit. The amplification performance and simplicity of our device promise its wide adaptation in quantum metrology, quantum communication, and quantum information processing.

关键词： Quantum coherence & coherence measures Quantum engineering Quantum measurements Quantum metrology

RESPECTING CAUSALITY IS ALL YOU NEED FOR TRAINING PHYSICS-INFORMED NEURAL NETWORKS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Sifan Sankaran, Shyam Perdikaris, Paris Graduate Group In Applied Mathematics And Computational Science University Of Pennsylvania PhiladelphiaPA19104 United States Department Of Mechanical Engineering And Applied Mechanics University Of Pennsylvania PhiladelphiaPA19104 United States

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. We provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto-Sivashinsky equation in the chaotic regime, and the Navier-Stokes equations in the turbulent regime. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity. © 2022, CC BY-NC-SA.

关键词： Partial differential equations

Risk-aware stochastic control of a sailboat

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, MingYi Patnaik, Natasha Somalwar, Anne Wu, Jingyi Vladimirsky, Alexander The Center for Applied Mathematics Cornell University IthacaNY14853 United States The Department of Computational Applied Mathematics and Operations Research Rice University HoustonTX77005 United States The graduate group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States The Center for Data Science New York University New York CityNY10012 United States The Department of Mathematics Cornell University IthacaNY14853 United States

Sailboat path-planning is a natural hybrid control problem (due to continuous steering and occasional "tack-switching" maneuvers), with the actual path-to-target greatly affected by stochastically evolving wind conditions. Previous studies have focused on finding risk-neutral policies that minimize the expected time of arrival. In contrast, we present a robust control approach, which maximizes the probability of arriving before a specified deadline/threshold. Our numerical method recovers the optimal risk-aware (and threshold-specific) policies for all initial sailboat positions and a broad range of thresholds simultaneously. This is accomplished by solving two quasi-variational inequalities based on second-order Hamilton-Jacobi-Bellman (HJB) PDEs with degenerate parabolicity. Monte-Carlo simulations show that risk-awareness in sailing is particularly useful when a carefully calculated bet on the evolving wind direction might yield a reduction in the number of *** Codes 93E20 (Primary), 93C30, 49L20, 65K15, 91B05 (Secondary) Copyright © 2023, The Authors. All rights reserved.

关键词： Numerical methods

Collaborative learning of discrete distributions under heterogeneity and communication constraints 22

学校读者我要写书评

暂无评论

Collaborative learning of discrete distributions under heter...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Xinmeng Huang Donghwan Lee Edgar Dobriban Hamed Hassani Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania Philadelphia PA Department of Statistics and Data Science University of Pennsylvania Philadelphia PA Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia PA

ISBN: (纸本)9781713871088

In modern machine learning, users often have to collaborate to learn distributions that generate the data. Communication can be a significant bottleneck. Prior work has studied homogeneous users—i.e., whose data follow the same discrete distribution—and has provided optimal communication-efficient methods. However, these methods rely heavily on homogeneity, and are less applicable in the common case when users' discrete distributions are heterogeneous. Here we consider a natural and tractable model of heterogeneity, where users' discrete distributions only vary sparsely, on a small number of entries. We propose a novel two-stage method named SHIFT: First, the users collaborate by communicating with the server to learn a central distribution; relying on methods from robust statistics. Then, the learned central distribution is fine-tuned to estimate the individual distributions of users. We show that our method is minimax optimal in our model of heterogeneity and under communication constraints. Further, we provide experimental results using both synthetic data and n-gram frequency estimation in the text domain, which corroborate its efficiency.

关键词：

Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Xinmeng Xu, Kan Lee, Donghwan Hassani, Hamed Bastani, Hamsa Dobriban, Edgar Graduate Group in Applied Math and Computational Science Univ. of Pennsylvania United States Department of Information Systems Arizona State University United States Department of Electrical and Systems Engineering Univ. of Pennsylvania United States Department of Operations Information and Decisions Univ. of Pennsylvania United States Department of Statistics and Data Science Univ. of Pennsylvania United States

Large and complex datasets are often collected from several, possibly heterogeneous sources. Multitask learning methods improve efficiency by leveraging commonalities across datasets while accounting for possible differences among them. Here, we study multitask linear regression and contextual bandits under sparse heterogeneity, where the source/task-associated parameters are equal to a global parameter plus a sparse task-specific term. We propose a novel two-stage estimator called MOLAR that leverages this structure by first constructing a covariate-wise weighted median of the task-wise linear regression estimates and then shrinking the task-wise estimates towards the weighted median. Compared to task-wise least squares estimates, MOLAR improves the dependence of the estimation error on the data dimension. Extensions of MOLAR to generalized linear models and constructing confidence intervals are discussed in the paper. We then apply MOLAR to develop methods for sparsely heterogeneous multitask contextual bandits, obtaining improved regret guarantees over single-task bandit methods. We further show that our methods are minimax optimal by providing a number of lower bounds. Finally, we support the efficiency of our methods by performing experiments on both synthetic data and the PISA dataset on student educational outcomes from heterogeneous countries. Copyright © 2023, The Authors. All rights reserved.

关键词： Large datasets

PIRATENETS: PHYSICS-INFORMED DEEP LEARNING WITH RESIDUAL ADAPTIVE NETWORKS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Sifan Li, Bowen Perdikaris, Paris Chen, Yuhan Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mathematics Duke University DurhamNC27708 United States Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States Department of Electrical and Computer Engineering North Carolina State University RaleighNC27695 United States

While physics-informed neural networks (PINNs) have become a popular deep learning framework for tackling forward and inverse problems governed by partial differential equations (PDEs), their performance is known to degrade when larger and deeper neural network architectures are employed. Our study identifies that the root of this counter-intuitive behavior lies in the use of multi-layer perceptron (MLP) architectures with non-suitable initialization schemes, which result in poor trainablity for the network derivatives, and ultimately lead to an unstable minimization of the PDE residual loss. To address this, we introduce Physics-Informed Residual Adaptive Networks (PirateNets), a novel architecture that is designed to facilitate stable and efficient training of deep PINN models. PirateNets leverage a novel adaptive residual connection, which allows the networks to be initialized as shallow networks that progressively deepen during training. We also show that the proposed initialization scheme allows us to encode appropriate inductive biases corresponding to a given PDE system into the network architecture. We provide comprehensive empirical evidence showing that PirateNets are easier to optimize and can gain accuracy from considerably increased depth, ultimately achieving state-of-the-art results across various benchmarks. All code and data accompanying this manuscript will be made publicly available at https://***/PredictiveIntelligenceLab/jaxpi. © 2024, CC BY-NC-ND.

关键词： Inverse problems

A Bayesian analysis of the time through the order penalty in baseball

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Brill, Ryan S. Deshpande, Sameer K. Wyner, Abraham J. Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Dept. of Statistics University of Wisconsin-Madison United States Dept. of Statistics and Data Science The Wharton School University of Pennsylvania United States

As a baseball game progresses, batters appear to perform better the more times they face a particular pitcher. The apparent drop-off in pitcher performance from one time through the order to the next, known as the Time Through the Order Penalty (TTOP), is often attributed to within-game batter learning. Although the TTOP has largely been accepted within baseball and influences many managers' in-game decision making, we argue that existing approaches of estimating the size of the TTOP cannot disentangle continuous evolution in pitcher performance over the course of the game from discontinuities between successive times through the order. Using a Bayesian multinomial regression model, we find that, after adjusting for confounders like batter and pitcher quality, handedness, and home field advantage, there is little evidence of strong discontinuity in pitcher performance between times through the order. Our analysis suggests that the start of the third time through the order should not be viewed as a special cutoff point in deciding whether to pull a starting pitcher. Copyright © 2022, The Authors. All rights reserved.

关键词： Baseball