检索结果-内蒙古大学图书馆

On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Sifan Wang, Hanwen Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. In this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction-diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior. All code an data accompanying this manuscript will be made publicly available at https://***/PredictiveIntelligenceLab/MultiscalePINNs. Copyright © 2020, The Authors. All rights reserved.

关键词： Partial differential equations

Risk-Aware Stochastic Control of a Sailboat

学校读者我要写书评

暂无评论

Risk-Aware Stochastic Control of a Sailboat

American Control Conference (ACC)

作者： MingYi Wang Natasha Patnaik Anne Somalwar Jingyi Wu Alexander Vladimirsky The Center for Applied Mathematics Cornell University Ithaca NY USA The Department of Computational Applied Mathematics and Operations Research Rice University Houston TX USA The graduate group in Applied Mathematics and Computational Science University of Pennsylvania Philadelphia PA USA The Center for Data Science New York University New York City NY USA Department of Mathematics Cornell University Ithaca NY USA

ISBN: (数字)9798350382655

ISBN: (纸本)9798350382662

Sailboat path-planning is a natural hybrid control problem (due to continuous steering and occasional “tack-switching” maneuvers), with the actual path-to-target greatly affected by stochastically evolving wind conditions. Previous studies have focused on finding risk-neutral policies that minimize the expected time of arrival. In contrast, we present a robust control approach, which maximizes the probability of arriving before a specified deadline/threshold. Our numerical method recovers the optimal risk-aware (and threshold-specific) policies for all initial sailboat positions and a broad range of thresholds simultaneously. This is accomplished by solving two quasi-variational inequalities based on second-order Hamilton-Jacobi-Bellman (HJB) PDEs with degenerate parabolicity. Monte-Carlo simulations show that risk-awareness in sailing is particularly useful when a carefully calculated bet on the evolving wind direction might yield a reduction in the number of tack-switches.

关键词： Robust control Geometry Monte Carlo methods Shape Stochastic processes Boats Differential games

A subgroup-aware scoring approach to the study of effect modification in observational studies

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fan, Yijun Small, Dylan S. Graduate Group of Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Statistics and Data Science University of Pennsylvania PhiladelphiaPA19104 United States

Effect modification means the size of a treatment effect varies with an observed covariate. Generally speaking, a larger treatment effect with more stable error terms is less sensitive to bias. Thus, we might be able to conclude that a study is less sensitive to unmeasured bias by using these subgroups experiencing larger treatment effects. Lee et al. (2018) proposed the submax method that leverages the joint distribution of test statistics from subgroups to draw a firmer conclusion if effect modification occurs. However, one version of the submax method uses M-statistics as the test statistics and is implemented in the R package submax (Rosenbaum, 2017). The scaling factor in the M-statistics is computed using all observations combined across subgroups. We show that this combining can confuse effect modification with outliers. We propose a novel group M-statistic that scores the matched pairs in each subgroup to tackle the issue. We examine our novel scoring strategy in extensive settings to show the superior performance. The proposed method is applied to an observational study of the effect of a malaria prevention treatment in West Africa. Copyright © 2024, The Authors. All rights reserved.

关键词： Malaria control

Fast PDE-constrained optimization via self-supervised operator learning: A preprint

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Sifan Bhouri, Mohamed Aziz Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Design and optimal control problems are among the fundamental, ubiquitous tasks we face in science and engineering. In both cases, we aim to represent and optimize an unknown (black-box) function that associates a performance/outcome to a set of controllable variables through an experiment. In cases where the experimental dynamics can be described by partial differential equations (PDEs), such problems can be mathematically translated into PDE-constrained optimization tasks, which quickly become intractable as the number of control variables and the cost of experiments increases. In this work we leverage physics-informed deep operator networks (DeepONets) – a self-supervised framework for learning the solution operator of parametric PDEs – to build fast and differentiable surrogates for rapidly solving PDE-constrained optimization problems, even in the absence of any paired input-output training data. The effectiveness of the proposed framework will be demonstrated across different applications involving continuous functions as control or design variables, including time-dependent optimal control of heat transfer, and drag minimization of obstacles in Stokes flow. In all cases, we observe that DeepONets can minimize high-dimensional cost functionals in a matter of seconds, yielding a significant speed up compared to traditional adjoint PDE solvers that are typically costly and limited to relatively low-dimensional control/design parametrizations. © 2021, CC BY-NC-SA.

关键词： Constrained optimization

Real-time Sampling and Estimation on Random Access Channels: Age of Information and Beyond

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chen, Xingran Liao, Xinyu Bidokhti, Shirin Saeedi The Department of Electrical and System Engineering University of Pennsylvania PA19104 United States The Graduate Group of Applied Mathematics and Computational Science University of Pennsylvania PA19104 United States

Next generation multiple access channels require to provision for unprecedented massive user access in a plethora of applications in cyber-physical systems. This work proposes decentralized policies for the real-time monitoring and estimation of autoregressive processes over random access channels. Two classes of policies are investigated: (i) oblivious schemes in which sampling and transmission policies are independent of the processes that are monitored, and (ii) non-oblivious schemes in which transmitters causally observe their corresponding processes for decision making. In the class of oblivious policies, we show that minimizing the expected time-average estimation error is equivalent to minimizing the expected age of information. Consequently, we prove lower and upper bounds on the minimum achievable estimation error in this class. Next, we consider non-oblivious policies and design a threshold policy, called error-based thinning, in which each transmitter node becomes active if its instantaneous error has crossed a fixed threshold (which we optimize). Active nodes then transmit stochastically following a slotted ALOHA policy. A closed-form, approximately optimal, solution is found for the threshold as well as the resulting estimation error. It is shown that non-oblivious policies offer a multiplicative gain close to 3 compared to oblivious policies. Moreover, it is shown that oblivious policies that use age of information for decision making improve the state-of-the-art at least by the multiplicative factor 2. The performance of all discussed policies is compared using simulations. Numerical comparison shows that the performance of the proposed decentralized policy is very close to that of centralized greedy scheduling. Finally, we extend our framework to unreliable random access channels. Simulations show that the multiplicative gain offered by non-oblivious policies (compared to oblivious policies) is independent of the channel erasure probability. Copyrigh

关键词： Transmitters

Selecting the number of components in PCA via random signflips

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Hong, David Sheng, Yue Dobriban, Edgar Department of Electrical and Computer Engineering University of Delaware United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Department of Statistics and Data Science University of Pennsylvania United States

Principal component analysis (PCA) is a foundational tool in modern data analysis, and a crucial step in PCA is selecting the number of components to keep. However, classical selection methods (e.g., scree plots, parallel analysis, etc.) lack statistical guarantees in the increasingly common setting of large-dimensional data with heterogeneous noise, i.e., where each entry may have a different noise variance. Moreover, it turns out that these methods, which are highly effective for homogeneous noise, can fail dramatically for data with heterogeneous noise. This paper proposes a new method called signflip parallel analysis (FlipPA) for the setting of approximately symmetric noise: it compares the data singular values to those of "empirical null" matrices generated by flipping the sign of each entry randomly with probability one-half. We develop a rigorous theory for FlipPA, showing that it has nonasymptotic type I error control and that it consistently selects the correct rank for signals rising above the noise floor in the large-dimensional limit (even when the noise is heterogeneous). We also rigorously explain why classical permutation-based parallel analysis degrades under heterogeneous noise. Finally, we illustrate that FlipPA compares favorably to state-of-the art methods via numerical simulations and an illustration on data coming from astronomy. Copyright © 2020, The Authors. All rights reserved.

关键词： Principal component analysis

A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Moniri, Behrad Lee, Donghwan Hassani, Hamed Dobriban, Edgar Department of Electrical and Systems Engineering University of Pennsylvania United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Department of Statistics and Data Science University of Pennsylvania United States

Feature learning is thought to be one of the fundamental reasons for the success of deep neural networks. It is rigorously known that in two-layer fully-connected neural networks under certain conditions, one step of gradient descent on the first layer can lead to feature learning;characterized by the appearance of a separated rank-one component—spike—in the spectrum of the feature matrix. However, with a constant gradient descent step size, this spike only carries information from the linear component of the target function and therefore learning non-linear components is impossible. We show that with a learning rate that grows with the sample size, such training in fact introduces multiple rank-one components, each corresponding to a specific polynomial feature. We further prove that the limiting large-dimensional and large sample training and test errors of the updated neural networks are fully characterized by these spikes. By precisely analyzing the improvement in the training and test errors, we demonstrate that these non-linear features can enhance learning. © 2023, CC BY.

关键词： Deep neural networks

Demystifying Disagreement-on-the-Line in High Dimensions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lee, Donghwan Moniri, Behrad Huang, Xinmeng Dobriban, Edgar Hassani, Hamed Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania United States Department of Electrical and Systems Engineering University of Pennsylvania United States Department of Statistics and Data Science University of Pennsylvania United States

Evaluating the performance of machine learning models under distribution shift is challenging, especially when we only have unlabeled data from the shifted (target) domain, along with labeled data from the original (source) domain. Recent work suggests that the notion of disagreement, the degree to which two models trained with different randomness differ on the same input, is a key to tackle this problem. Experimentally, disagreement and prediction error have been shown to be strongly connected, which has been used to estimate model performance. Experiments have led to the discovery of the disagreement-on-the-line phenomenon, whereby the classification error under the target domain is often a linear function of the classification error under the source domain;and whenever this property holds, disagreement under the source and target domain follow the same linear relation. In this work, we develop a theoretical foundation for analyzing disagreement in high-dimensional random features regression;and study under what conditions the disagreement-on-the-line phenomenon occurs in our setting. Experiments on CIFAR-10-C, Tiny ImageNet-C, and Camelyon17 are consistent with our theory and support the universality of the theoretical findings. © 2023, CC BY.

关键词： Errors

Collaborative learning of discrete distributions under heterogeneity and communication constraints 22

学校读者我要写书评

暂无评论

Collaborative learning of discrete distributions under heter...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Xinmeng Huang Donghwan Lee Edgar Dobriban Hamed Hassani Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania Philadelphia PA Department of Statistics and Data Science University of Pennsylvania Philadelphia PA Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia PA

ISBN: (纸本)9781713871088

In modern machine learning, users often have to collaborate to learn distributions that generate the data. Communication can be a significant bottleneck. Prior work has studied homogeneous users—i.e., whose data follow the same discrete distribution—and has provided optimal communication-efficient methods. However, these methods rely heavily on homogeneity, and are less applicable in the common case when users' discrete distributions are heterogeneous. Here we consider a natural and tractable model of heterogeneity, where users' discrete distributions only vary sparsely, on a small number of entries. We propose a novel two-stage method named SHIFT: First, the users collaborate by communicating with the server to learn a central distribution; relying on methods from robust statistics. Then, the learned central distribution is fine-tuned to estimate the individual distributions of users. We show that our method is minimax optimal in our model of heterogeneity and under communication constraints. Further, we provide experimental results using both synthetic data and n-gram frequency estimation in the text domain, which corroborate its efficiency.

关键词：