检索结果-内蒙古大学图书馆

arXiv 2022年

作者： Lee, Donghwan Huang, Xinmeng Hassani, Hamed Dobriban, Edgar Graduate Group in Applied Mathematics and Computational Science Univ. of Pennsylvania United States Department of Electrical and Systems Engineering Univ. of Pennsylvania United States Department of Statistics and Data Science Univ. of Pennsylvania United States

The prediction accuracy of machine learning methods is steadily increasing, but the calibration of their uncertainty predictions poses a significant challenge. Numerous works focus on obtaining well-calibrated predictive models, but less is known about reliably assessing model calibration. This limits our ability to know when algorithms for improving calibration have a real effect, and when their improvements are merely artifacts due to random noise in finite datasets. In this work, we consider detecting mis-calibration of predictive models using a finite validation dataset as a hypothesis testing problem. The null hypothesis is that the predictive model is calibrated, while the alternative hypothesis is that the deviation from calibration is sufficiently large. We find that detecting mis-calibration is only possible when the conditional probabilities of the classes are sufficiently smooth functions of the predictions. When the conditional class probabilities are Hölder continuous, we propose T-Cal, a minimax optimal test for calibration based on a debiased plug-in estimator of the 2-Expected Calibration Error (ECE). We further propose adaptive T-Cal, a version that is adaptive to unknown smoothness. We verify our theoretical findings with a broad range of experiments, including with several popular deep neural net architectures and several standard post-hoc calibration methods. T-Cal is a practical general-purpose tool, which-combined with classical tests for discrete-valued predictors-can be used to test the calibration of virtually any probabilistic classification method. T-Cal is available at https://***/dh7401/T-Cal. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Collaborative Learning of Discrete Distributions under Heterogeneity and Communication Constraints

arXiv

引用

arXiv 2022年

作者： Huang, Xinmeng Lee, Donghwan Dobriban, Edgar Hassani, Hamed Graduate Group in Applied Mathematics and Computational Science Univ. of Pennsylvania United States Department of Statistics and Data Science Univ. of Pennsylvania United States Department of Electrical and Systems Engineering Univ. of Pennsylvania United States

In modern machine learning, users often have to collaborate to learn the distribution of the data. Communication can be a significant bottleneck. Prior work has studied homogeneous users-i.e., whose data follow the same discrete distribution-and has provided optimal communication-efficient methods for estimating that distribution. However, these methods rely heavily on homogeneity, and are less applicable in the common case when users' discrete distributions are heterogeneous. Here we consider a natural and tractable model of heterogeneity, where users' discrete distributions only vary sparsely, on a small number of entries. We propose a novel two-stage method named SHIFT: First, the users collaborate by communicating with the server to learn a central distribution;relying on methods from robust statistics. Then, the learned central distribution is fine-tuned to estimate their respective individual distribution. We show that SHIFT is minimax optimal in our model of heterogeneity and under communication constraints. Further, we provide experimental results using both synthetic data and n-gram frequency estimation in the text domain, which corroborate its efficiency. Copyright © 2022, The Authors. All rights reserved.

关键词： Frequency estimation

来源：评论

学校读者我要写书评

暂无评论

在天文学中基于机器学习的流体力学问题加速求解

在天文学中基于机器学习的流体力学问题加速求解

引用

第十三届全国流体力学学术会议

作者：毛顺元董若冰王炜琦 Kwang Moo Yi 陆路王思凡 Paris Perdikaris Department of Physics and Astronomy University of Victoria Department of Computer Science University of British Columbia Department of Statistics and Data Science Yale University Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania Department ofMechanical Engineering and Applied Mechanics University of Pennsylvania

探索行星形成的过程不仅揭示了地球及其生命的起源,还为寻找地外生命提供了关键线索。行星在原行星盘中形成。原行星盘是由气体和尘埃构成的扁平盘状系统,围绕在新生恒星的周围,其跨度可达数千亿公里。近年来,通过分析行星对原行星盘的... 详细信息

探索行星形成的过程不仅揭示了地球及其生命的起源,还为寻找地外生命提供了关键线索。行星在原行星盘中形成。原行星盘是由气体和尘埃构成的扁平盘状系统,围绕在新生恒星的周围,其跨度可达数千亿公里。近年来,通过分析行星对原行星盘的引力扰动,并比较观测和数值模拟,天文学家得以推断正在盘中形成的行星的质量和轨道。然而,这种方法依赖于复杂的流体力学数值模拟,因计算成本高昂而难以普及。为了克服这一挑战,我们开发了一款基于机器学习的工具。该工具应用数据驱动的深度算子网络(DeepONet),将系统的参数映射到流体力学偏微分方程的解,从而无需昂贵的数值模拟即可实现两项功能:1)快速求解正问题,即从行星参数预测盘中物质分布和速度场;2)高效求解反问题,即从观测到的扰动图像中推断行星参数。我们展示了该工具在求解速度、精确性和鲁棒性方面的显著优势,并讨论了其在理论研究和观测中的潜在应用。我们还探索了物理信息神经网络(PINN)在这一问题的应用,特别是针对原初行星盘中旋转吸积流的时间依赖性二维可压缩Navier-Stokes方程。行星引力扰动引入显著的时间依赖性空间特征,使问题复杂化。通过对物理定律而非标注数据的训练,PINN在捕捉系统演化的空间高频特征方面展现潜力。标准PINN在此任务中存在收敛困难、预测误差随时间累积、长时间解偏离物理定律以及捕捉空间高频特征能力不足的问题。为此,我们的研究引入了自适应激活函数、时间推进策略和网络权重继承方法,以提升PINN性能。改进后的PINN模型在与标准PINN的比较中表现出显著更低的预测误差,成功再现了大部分空间高频特征,无需标注数据。这项研究为将PINN应用于类似的时间依赖性流体力学问题提供了重要参考。

关键词：天文行星形成深度学习深度算子网络物理信息神经网络

来源：评论

学校读者我要写书评

暂无评论

One-shot distributed ridge regression in high dimensions 37

One-shot distributed ridge regression in high dimensions

引用

37th International Conference on Machine Learning, ICML 2020

作者： Dobriban, Edgar Sheng, Yue Wharton Statistics Department University of Pennsylvania PhiladelphiaPA United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA United States

ISBN: (纸本)9781713821120

To scale up data analysis, distributed and parallel computing approaches are increasingly needed. Here we study a fundamental problem in this area: How to do ridge regression in a distributed computing environment? We study one-shot methods constructing weighted combinations of ridge regression estimators computed on each machine. By analyzing the mean squared error in a high dimensional model where each predictor has a small effect, we discover several new phenomena including that the efficiency depends strongly on the signal strength, but does not degrade with many workers, the risk decouples over machines, and the unexpected consequence that the optimal weights do not sum to unity. We also propose a new optimally weighted one-shot ridge regression algorithm. Our results are supported by simulations and real data analysis. © 2020 37th International Conference on Machine Learning, ICML 2020. All rights reserved.

关键词： Mean square error

来源：评论

学校读者我要写书评

暂无评论

Learning operators with coupled attention

arXiv

引用

arXiv 2022年

作者： Kissas, Georgios Seidman, Jacob Guilhoto, Leonardo Ferreira Preciado, Victor M. Pappas, George J. Perdikaris, Paris Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Electrical and Systems Engineering University of Pennsylvania PhiladelphiaPA19104 United States

Supervised operator learning is an emerging machine learning paradigm with applications to modeling the evolution of spatio-temporal dynamical systems and approximating general black-box relationships between functional data. We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. In our architecture, the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions, enabling us to approximate nonlinear operators even when the number of output function in the training set measurements is very small. Our formulation is accompanied by rigorous approximation theoretic guarantees on the universal expressiveness of the proposed model. Empirically, we evaluate the performance of LOCA on several operator learning scenarios involving systems governed by ordinary and partial differential equations, as well as a black-box climate prediction problem. Through these scenarios we demonstrate state of the art accuracy, robustness with respect to noisy input data, and a consistently small spread of errors over testing data sets, even for out-of-distribution prediction tasks. © 2022, CC BY-NC-SA.

关键词： mathematical operators

来源：评论

学校读者我要写书评

暂无评论

Reliable Extrapolation of Deep Neural Operators Informed by Physics or Sparse Observations

SSRN

引用

SSRN 2023年

作者： Zhu, Min Zhang, Handi Jiao, Anran Karniadakis, George Em Lu, Lu Department of Chemical and Biomolecular Engineering University of Pennsylvania PhiladelphiaPA19104 United States Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Computer and Information Science University of Pennsylvania PhiladelphiaPA19104 United States Division of Applied Mathematics Brown University ProvidenceRI02912 United States School of Engineering Brown University ProvidenceRI02912 United States

Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity via the 2-Wasserstein distance between two function spaces and propose a new strategy of bias-variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information-the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed. © 2023, The Authors. All rights reserved.

关键词： Extrapolation

来源：评论

学校读者我要写书评

暂无评论

Learning the solution operator of parametric partial differential equations with physics-informed deeponets

arXiv

引用

arXiv 2021年

作者： Wang, Sifan Wang, Hanwen Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Deep operator networks (DeepONets) are receiving increased attention thanks to their demonstrated capability to approximate nonlinear operators between infinite-dimensional Banach spaces. However, despite their remarkable early promise, they typically require large training data-sets consisting of paired input-output observations which may be expensive to obtain, while their predictions may not be consistent with the underlying physical principles that generated the observed data. In this work, we propose a novel model class coined as physics-informed DeepONets, which introduces an effective regularization mechanism for biasing the outputs of DeepOnet models towards ensuring physical consistency. This is accomplished by leveraging automatic differentiation to impose the underlying physical laws via soft penalty constraints during model training. We demonstrate that this simple, yet remarkably effective extension can not only yield a significant improvement in the predictive accuracy of DeepOnets, but also greatly reduce the need for large training data-sets. To this end, a remarkable observation is that physics-informed DeepONets are capable of solving parametric partial differential equations (PDEs) without any paired input-output observations, except for a set of given initial or boundary conditions. We illustrate the effectiveness of the proposed framework through a series of comprehensive numerical studies across various types of PDEs. Strikingly, a trained physics informed DeepOnet model can predict the solution of O(103) time-dependent PDEs in a fraction of a second - up to three orders of magnitude faster compared a conventional PDE solver. The data and code accompanying this manuscript are publicly available at https://***/PredictiveIntelligenceLab/Physics-informed-DeepONets. Copyright © 2021, The Authors. All rights reserved.

关键词： mathematical operators

来源：评论

学校读者我要写书评

暂无评论

Long-time integration of parametric evolution equations with physics-informed deeponets

arXiv

引用

arXiv 2021年

作者： Wang, Sifan Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Ordinary and partial differential equations (ODEs/PDEs) play a paramount role in analyzing and simulating complex dynamic processes across all corners of science and engineering. In recent years machine learning tools are aspiring to introduce new effective ways of simulating PDEs, however existing approaches are not able to reliably return stable and accurate predictions across long temporal horizons. We aim to address this challenge by introducing an effective framework for learning infinite-dimensional operators that map random initial conditions to associated PDE solutions within a short time interval. Such latent operators can be parametrized by deep neural networks that are trained in an entirely self-supervised manner without requiring any paired input-output observations. Global long-time predictions across a range of initial conditions can be then obtained by iteratively evaluating the trained model using each prediction as the initial condition for the next evaluation step. This introduces a new approach to temporal domain decomposition that is shown to be effective in performing accurate long-time simulations for a wide range of parametric ODE and PDE systems, from wave propagation, to reaction-diffusion dynamics and stiff chemical kinetics, all at a fraction of the computational cost needed by classical numerical solvers. © 2021, CC BY-NC-SA.

关键词： Domain decomposition methods

来源：评论

学校读者我要写书评

暂无评论

Fast PDE-constrained optimization via self-supervised operator learning: A preprint

arXiv

引用

arXiv 2021年

作者： Wang, Sifan Bhouri, Mohamed Aziz Perdikaris, Paris Graduate Group in Applied Mathematics and Computational Science University of Pennsylvania PhiladelphiaPA19104 United States Department of Mechanichal Engineering and Applied Mechanics University of Pennsylvania PhiladelphiaPA19104 United States

Design and optimal control problems are among the fundamental, ubiquitous tasks we face in science and engineering. In both cases, we aim to represent and optimize an unknown (black-box) function that associates a performance/outcome to a set of controllable variables through an experiment. In cases where the experimental dynamics can be described by partial differential equations (PDEs), such problems can be mathematically translated into PDE-constrained optimization tasks, which quickly become intractable as the number of control variables and the cost of experiments increases. In this work we leverage physics-informed deep operator networks (DeepONets) – a self-supervised framework for learning the solution operator of parametric PDEs – to build fast and differentiable surrogates for rapidly solving PDE-constrained optimization problems, even in the absence of any paired input-output training data. The effectiveness of the proposed framework will be demonstrated across different applications involving continuous functions as control or design variables, including time-dependent optimal control of heat transfer, and drag minimization of obstacles in Stokes flow. In all cases, we observe that DeepONets can minimize high-dimensional cost functionals in a matter of seconds, yielding a significant speed up compared to traditional adjoint PDE solvers that are typically costly and limited to relatively low-dimensional control/design parametrizations. © 2021, CC BY-NC-SA.

关键词： Constrained optimization

来源：评论

学校读者我要写书评

暂无评论

Which Activation Function Works Best for Training Artificial Pancreas: Empirical Fact and Its Theoretical Explanation

Which Activation Function Works Best for Training Artificial...

引用

2023 IEEE Symposium Series on computational Intelligence, SSCI 2023

作者： Denes-Fazakas, Lehel Szilagyi, Laszlo Eigner, Gyorgy Kosheleva, Olga Ceberio, Martine Kreinovich, Vladik Doctoral School Óbuda University Physiological Controls Research Center Appl. Informatics & Appl. Math. Budapest Hungary Sapientia University Computational Intelligence Res. Group Tg. Mures Romania Physiological Controls Res. Center Óbuda University Budapest Hungary Biomatics and Applied Ai Institute Obuda University Physiological Controls Res. Center John von Neumann Faculty of Informatics Budapest Hungary University of Texas at El Paso Department of Teacher Education El PasoTX United States University of Texas at El Paso Department of Computer Science El PasoTX United States

ISBN: (纸本)9781665430654

One of the most effective ways to help patients at the dangerous levels of diabetes is an artificial pancreas, a device that constantly monitors the patient's blood sugar level and injects insulin based on this level. Patient's reaction to insulin is highly individualized, so the artificial pancreas needs to be trained on each patient. It turns out that the best training results are attained when instead of the usual ReLU neurons, we use their minor modification known as Exponential Linear Units (ELU). In this paper, we provide a theoretical explanation for the empirically observed effectiveness of ELUs. © 2023 IEEE.

关键词： artificial pancreas diabetes Exponential Linear Unit (ELU) Leaky ReLU neural network Rectified Linear Unit (ReLU)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：