检索结果-内蒙古大学图书馆

Solving Key Challenges in Collider Physics with Foundation Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Mikuni, Vinicius Nachman, Benjamin National Energy Research Scientific Computing Center Berkeley Lab BerkeleyCA94720 United States Physics Division Lawrence Berkeley National Laboratory BerkeleyCA94720 United States Berkeley Institute for Data Science University of California BerkeleyCA94720 United States

Foundation Models are neural networks that are capable of simultaneously solving many problems. Large Language Foundation Models like ChatGPT have revolutionized many aspects of daily life, but their impact for science is not yet clear. In this paper, we use a new Foundation Model for hadronic jets to solve three key challenges in collider physics. In particular, we show how experiments can (1) save significant computing power when developing reconstruction algorithms, (2) perform a complete uncertainty quantification for high-dimensional measurements, and (3) search for new physics with model agnostic methods using low-level inputs. In each case, there are significant computational or methodological challenges with current methods that limit the science potential of deep learning algorithms. By solving each problem, we take jet Foundation Models beyond proof-of-principle studies and into the toolkit of practitioners. © 2024, CC BY.

关键词： Colliding beam accelerators

Indirect influence in social networks as an induced percolation phenomenon (vol 119, e2100151119, 2022)

学校读者我要写书评

暂无评论

PROCEEDINGS OF THE NATIONAL ACADEMY OF scienceS OF THE UNITED STATES OF AMERICA 2022年第24期119卷 e2100151119-e2100151119页

作者： Xie, Jiarong Wang, Xiangrong Feng, Ling Zhao, Jin-Hua Liu, Wenyuan Moreno, Yamir Hu, Yanqing School of Computer Science and Engineering Sun Yat-sen University 510006 Guangzhou China Institute of Future Networks Southern University of Science and Technology 518055 Shenzhen China Peng Cheng Laboratory 518066 Shenzhen China Institute of High Performance Computing A*STAR 138632 Singapore Department of Physics National University of Singapore 117551 Singapore Guangdong Provincial Key Laboratory of Nuclear Science Institute of Quantum Matter South China Normal University 510006 Guangzhou China Guangdong-Hong Kong Joint Laboratory of Quantum Matter Southern Nuclear Science Computing Center South China Normal University 510006 Guangzhou China Division of Physics and Applied Physics School of Physical and Mathematical Sciences Nanyang Technological University 637371 Singapore Institute for Biocomputation and Physics of Complex Systems University of Zaragoza 50018 Zaragoza Spain Department of Theoretical Physics University of Zaragoza 50018 Zaragoza Spain ISI Foundation 10126 Torino Italy Department of Statistics and Data Science College of Science Southern University of Science and Technology 518055 Shenzhen China

Percolation theory has been widely used to study phase transitions in network systems. It has also successfully explained various macroscopic spreading phenomena across different fields. Yet, the theoretical frameworks have been focusing on direct interactions among nodes, while recent empirical observations have shown that indirect interactions are common in many network systems like social and ecological networks, among others. By investigating the detailed mechanism of both direct and indirect influence on scientific collaboration networks, here we show that indirect influence can play the dominant role in behavioral influence. To address the lack of theoretical understanding of such indirect influence on the macroscopic behavior of the system, we propose a percolation mechanism of indirect interactions called induced percolation. Surprisingly, our model exhibits a unique anisotropy property. Specifically, directed networks show first-order abrupt transitions as opposed to the second-order continuous transition in the same network structure but with undirected links. A mix of directed and undirected links leads to rich hybrid phase transitions. Furthermore, a unique feature of the nonmonotonic pattern is observed in network connectivities near the critical point. We also present an analytical framework to characterize the proposed induced percolation, paving the way to further understanding network dynamics with indirect interactions.

关键词： percolation indirect interactions social network phase transition behavioral contagion

Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Jerry Yao-Chieh Su, Maojiang Kuo, En-Jui Song, Zhao Liu, Han Department of Computer Science Northwestern University EvanstonIL60208 United States Department of Information and Computing Science USTC Anhui Hefei230026 China Physics Division National Center for Theoretical Sciences Taipei106319 Taiwan Adobe Research SeattleWA98103 United States Department of Statistics and Data Science Northwestern University EvanstonIL60208 United States

We study the computational limits of Low-Rank Adaptation (LoRA) update for finetuning transformer-based models using fine-grained complexity theory. Our key observation is that the existence of low-rank decompositions within the gradient computation of LoRA adaptation leads to possible algorithmic speedup. This allows us to (i) identify a phase transition behavior and (ii) prove the existence of nearly linear algorithms by controlling the LoRA update computation term by term, assuming the Strong Exponential Time Hypothesis (SETH). For the former, we identify a sharp transition in the efficiency of all possible rank-r LoRA update algorithms for transformers, based on specific norms resulting from the multiplications of the input sequence X, pretrained weights W*, and adapter matrices αBA/r. Specifically, we derive a shared upper bound threshold for such norms, and show that efficient (sub-quadratic) approximation algorithms of LoRA exist only below this threshold. For the latter, we prove the existence of nearly linear approximation algorithms for LoRA adaptation by utilizing the hierarchical low-rank structures of LoRA gradients and approximating the gradients with a series of chained low-rank approximations. To showcase our theory, we consider two practical scenarios: partial (e.g., only WV and WQ) and full adaptations (e.g., WQ, WV, and WK) of weights in attention heads. © 2024, CC BY-NC-SA.

关键词： Approximation algorithms

JARVIS-Leaderboard:a large scale benchmark of materials design methods

学校读者我要写书评

暂无评论

npj Computational Materials 2024年第1期10卷 2280-2296页

作者： Kamal Choudhary Daniel Wines Kangming Li Kevin F.Garrity Vishu Gupta Aldo H.Romero Jaron T.Krogel Kayahan Saritas Addis Fuhr Panchapakesan Ganesh Paul R.C.Kent Keqiang Yan Yuchao Lin Shuiwang Ji Ben Blaiszik Patrick Reiser Pascal Friederich Ankit Agrawal Pratyush Tiwary Eric Beyerle Peter Minch Trevor David Rhone Ichiro Takeuchi Robert B.Wexler Arun Mannodi-Kanakkithodi Elif Ertekin Avanish Mishra Nithin Mathew Mitchell Wood Andrew Dale Rohskopf Jason Hattrick-Simpers Shih-Han Wang Luke E.K.Achenie Hongliang Xin Maureen Williams Adam J.Biacchi Francesca Tavazza Material Measurement Laboratory National Institute of Standards and TechnologyGaithersburgMD20899USA Department of Materials Science and Engineering University of Toronto27 King’s College CirTorontoONCanada Department of Electrical and Computer Engineering Northwestern UniversityEvanstonIL 60208USA Lewis-Sigler Institute for Integrative Genomics Princeton UniversityPrincetonNJ 08544USA Ludwig Institute for Cancer Research Princeton UniversityPrincetonNJ 08544USA Department of Physics and Astronomy West Virginia UniversityMorgantownWV 26506USA Materials Science and Technology Division Oak Ridge National LaboratoryOak RidgeTN 37831USA Center for Nanophase Materials Science Oak Ridge National LaboratoryOak RidgeTN 37831USA Computational Sciences and Engineering Division Oak Ridge National LaboratoryOak RidgeTN 37831USA Department of Computer Science and Engineering Texas A&M UniversityCollege StationTX 77843USA Globus University of ChicagoChicagoIL 60637USA Data Science and Learning Division Argonne National LabLemontIL 60439USA Institute of Nanotechnology Karlsruhe Institute of TechnologyKaiserstraße 1276131 KarlsruheGermany Institute of Theoretical Informatics Karlsruhe Institute of TechnologyKaiserstraße 1276131 KarlsruheGermany Department of Chemistry and Biochemistry and Institute for Physical Science and Technology University of MarylandCollege ParkMD 20742USA Department of Physics Applied Physics and AstronomyRensselaer Polytechnic InstituteTroyNY12180USA Department of Materials Science and Engineering University of MarylandCollege ParkMD20742USA Department of Chemistry and Institute of Materials Science and Engineering Washington University in St.LouisSt.LouisMO63130USA School of Materials Engineering Purdue UniversityWest LafayetteIN47907USA Department of Mechanical Science and Engineering University of Illinois Urbana-ChampaignUrbanaIllinois 61801USA Materials Research Laboratory University of Illinois Urbana-ChampaignUrbanaIL 61801USA Theore

Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many *** science,in particular,encompasses a variety of experimental and theoretical approaches that require careful *** efforts have been developed previously to mitigate these ***,a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still *** work introduces JARVIS-Leaderboard,an open-source and community-driven platform that facilitates benchmarking and enhances *** platform allows users to set up benchmarks with customtasks and enables contributions in the form of dataset,code,and meta-data *** cover the following materials design categories:Artificial Intelligence(AI),Electronic Structure(ES).

关键词： rigorous perfect enable

SAGIPS: A Scalable Asynchronous Generative Inverse Problem Solver

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lersch, Daniel Schram, Malachi Dai, Zhenyu Rajput, Kishansingh Wu, Xingfu Sato, N. Childers, J. Taylor Data Science Department Thomas Jefferson National Accelerator Facility Newport NewsVA23606 United States Mathematics and Computer Science Division Argonne National Laboratory LemontIL60439 United States Theory Center Thomas Jefferson National Accelerator Facility Newport NewsVA23606 United States Leadership Computing Division Argonne National Laboratory LemontIL60439 United States

Large scale, inverse problem solving deep learning algorithms have become an essential part of modern research and industrial applications. The complexity of the underlying inverse problem often poses challenges to the algorithm and requires the proper utilization of high-performance computing systems. Most deep learning algorithms require, due to their design, custom parallelization techniques in order to be resource efficient while showing a reasonable convergence. In this paper we introduces a Scalable Asynchronous Generative workflow for solving Inverse Problems Solver (SAGIPS) on high-performance computing systems. We present a workflow that utilizes a parallelization approach where the gradients of the generator network are updated in an asynchronous ring-all-reduce fashion. Experiments with a scientific proxy application demonstrate that SAGIPS shows near linear weak scaling, together with a convergence quality that is comparable to traditional methods. The approach presented here allows leveraging GANs across multiple GPUs, promising advancements in solving complex inverse problems at scale. Copyright © 2024, The Authors. All rights reserved.

关键词： Inverse problems

Exploring the exact limits of the real-time equation-of-motion coupled cluster cumulant Green’s functions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Peng, Bo Pathak, Himadri Panyala, Ajay Vila, Fernando D. Rehr, John J. Kowalski, Karol Physical and Computational Science Directorate Pacific Northwest National Laboratory RichlandWA99354 United States Advanced Computing Mathematics and Data Division Pacific Northwest National Laboratory RichlandWA99354 United States Department of Physics University of Washington SeattleWA98195 United States

In this paper, we analyze the properties of the recently proposed real-time equation-of-motion coupled-cluster (RT-EOM-CC) cumulant Green’s function approach [J. Chem. Phys. 2020, 152, 174113]. We specifically focus on identifying the limitations of the original time-dependent coupled cluster (TDCC) ansatz and propose an enhanced double TDCC ansatz ensuring the exactness in the expansion limit. Additionally, we introduce a practical cluster-analysis-based approach for characterizing the peaks in the computed spectral function from the RT-EOM-CC cumulant Green’s function approach, which is particularly useful for the assignments of satellite peaks when many-body effects dominate the spectra. Our preliminary numerical tests focus on reproducing, approximating, and characterizing the exact impurity Green’s function of the three-site and four-site single impurity Anderson models using the RT-EOM-CC cumulant Green’s function approach. The numerical tests allow us to have a direct comparison between the RT-EOM-CC cumulant Green’s function approach and other Green’s function approaches in the numerical exact limit. © 2024, CC BY.

关键词： Green's function

State-space models are accurate and efficient neural operators for dynamical systems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Zheyuan Daryakenari, Nazanin Ahmadi Shen, Qianli Kawaguchi, Kenji Em Karniadakis, George Department of Computer Science National University of Singapore Singapore119077 Singapore Center for Biomedical Engineering School of Engineering Brown University ProvidenceRI02912 United States Division of Applied Mathematics Brown University ProvidenceRI02912 United States Advanced Computing Mathematics and Data Division Pacific Northwest National Laboratory RichlandWA United States

Physics-informed machine learning (PIML) has emerged as a promising alternative to classical methods for predicting dynamical systems, offering faster and more generalizable solutions. However, existing models, including recurrent neural networks (RNNs), transformers, and neural operators, face challenges such as long-time integration, long-range dependencies, chaotic dynamics, and extrapolation, to name a few. To this end, this paper introduces state-space models implemented in Mamba for accurate and efficient dynamical system operator learning. Mamba addresses the limitations of existing architectures by dynamically capturing long-range dependencies and enhancing computational efficiency through reparameterization techniques. To extensively test Mamba and compare against another 11 baselines, we introduce several strict extrapolation testbeds that go beyond the standard interpolation benchmarks. We demonstrate Mamba’s superior performance in both interpolation and challenging extrapolation tasks. Mamba consistently ranks among the top models while maintaining the lowest computational cost and exceptional extrapolation capabilities. Moreover, we demonstrate the good performance of Mamba for a real-world application in quantitative systems pharmacology for assessing the efficacy of drugs in tumor growth under limited data scenarios. Taken together, our findings highlight Mamba’s potential as a powerful tool for advancing scientific machine learning in dynamical systems modeling. (The code will be available at https://***/zheyuanhu01/State_ Space_Model_Neural_Operator upon acceptance.) © 2024, CC BY.

关键词： Recurrent neural networks

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Zheyuan Zhang, Zhongqiang Em Karniadakis, George Kawaguchi, Kenji Department of Computer Science National University of Singapore Singapore119077 Singapore Department of Mathematical Sciences Worcester Polytechnic Institute WorcesterMA01609 United States Division of Applied Mathematics Brown University USA ProvidenceRI02912 United States Advanced Computing Mathematics and Data Division Pacific Northwest National Laboratory RichlandWA United States

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. Moreover, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function;and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Browni

关键词： Fokker Planck equation

ANFIS Model for Cost Analysis in a Dual Source Multi-Destination System

学校读者我要写书评

暂无评论

Procedia Computer science 2023年 217卷 1266-1279页

作者： M.O. Okwu L.K. Tartibu E.O. Ojo S. Adume J.O. Gidiagba J. Fadeyi Department of Mechanical and Industrial Engineering Tech. University of Johannesburg South Africa Department of Mechanical Engineering Federal University of Petroleum Resources Effurun Nigeria Mathematics and Data Science Dept. Computing Science and Mathematics Division Faculty of Natural Science University of Stirling UK Faculty of Engineering and Applied Sc. Memorial University Newfoundland Canada Department of Industrial and Systems Engineering University of Buffalo USA

Managers face uncertainties while making allocation decisions, especially in dual or multi-source multi-destination inventory systems. Regrettably, many studies focus on classical methods as a technique for product distribution, which has never guaranteed a satisfactory solution. Real-life problems are non-deterministic polynomial-time hard (NP-hard), and solving such problems is relatively challenging. Such complicated problems need an efficient and robust computational hybrid algorithm. This study emphasises the need for a hybrid intelligent technique for effective product distribution. Soft computing hybrid algorithm, ANFIS was applied to product distribution in a double source multi-destination system. Rules were developed from available input datasets. Distributing products from dual manufacturing plants to fifteen available depots using the creative algorithm resulted in an overall 13.5% decrease in cost compared to the existing method adopted by the company. The result showed that the proposed method is relatively satisfactory and adequate for cost modelling. In addition, it is easy to use and outperforms the classical approach.

关键词： Adaptive Neuro-Fuzzy Inference System Dual Source Multi-Echelon Multi-Destinations