检索结果-内蒙古大学图书馆

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Soumyadip Ghosh Vijay Gupta University of Notre Dame

Communication in parallel systems consumes significant amount of time and energy which often turns out to be a bottleneck in distributed machine learning. In this paper, we present EventGraD - an algorithm with event-triggered communication in parallel stochastic gradient descent. The main idea of this algorithm is to modify the requirement of communication at every epoch to communicating only in certain epochs when necessary. In particular, the parameters are communicated only in the event when the change in their values exceed a threshold. The threshold for a parameter is chosen adaptively based on the rate of change of the parameter. The adaptive threshold ensures that the algorithm can be applied to different models on different datasets without any change. We focus on data-parallel training of a popular convolutional neural network used for training the MNIST dataset and show that EventGraD can reduce the communication load by up to 70% while retaining the same level of accuracy.

关键词： Training Adaptation models machine learning algorithms Conferences high performance computing Neural networks Stochastic processes

来源：评论

学校读者我要写书评

暂无评论

Metaoptimization on a Distributed System for Deep Reinforcement learning 5

Metaoptimization on a Distributed System for Deep Reinforcem...

引用

5th IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Heinrich, Greg Frosio, Iuri NVIDIA Nice France NVIDIA Santa Clara CA USA

ISBN: (纸本)9781728159850

Training intelligent agents through reinforcement learning (RL) is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experiences and consequently reduce instabilities, but the success of training remains strongly influenced by the choice of the hyperparameters. To overcome this issue, we introduce HyperTrick, a new metaoptimization algorithm, and show its effective application to tune hyperparameters in the case of deep RL, while learning to play different Atari games on a distributed system. Our analysis provides evidence of the interaction between the identification of the optimal hyperparameters and the learned policy, that is peculiar of the case of metaoptimization for deep RL. When compared with state-of-the-art metaoptimization algorithms, HyperTrick is characterized by a simpler implementation and it allows learning similar policies, while making a more effective

关键词： Reinforcement learning distributed systems metaoptimization

来源：评论

学校读者我要写书评

暂无评论

How Good Is Your Scientific Data Generative Model?

How Good Is Your Scientific Data Generative Model?

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Yuxin Yang Ben Gremillion Xitong Zhang Youzuo Lin Brendt Wohlberg Qiang Guan Los Alamos National Laboratory Kent State University The University of Texas at Austin Science and Engineering Michigan State University

ISBN: (纸本)9781665422918;9781665446556

Nowadays, leveraging data augmentation methods on helping resolving scientific problems becomes prevailing. And many scientific problems benefit from data augmentation methods build with deep generative models. Yet due to the complexity of the scientific data, commonly used evaluation methods of generative models appear not so suitable for generated scientific data. In this paper, we explore how do we effectively evaluate data augmentation methods for scientific data generative models? To answer this question, we use one example of real world scientific problem to show how we evaluate the quality of the generated data from two domain specific deep generative models. We observe that most existing state-of-art evaluation metrics are incompetent. They either show completely contradicting results or provide inaccurate insight from real data.

关键词： Measurement Conferences high performance computing Computational modeling Imaging machine learning Data models

来源：评论

学校读者我要写书评

暂无评论

GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks 5

GradVis: Visualization and Second Order Analysis of Optimiza...

引用

5th IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Chatzimichailidis, Avraam Pfreundt, Franz-Josef Gauger, Nicolas R. Keuper, Janis Fraunhofer ITWM Competence Ctr High Performance Comp Kaiserslautern Germany Fraunhofer Ctr Machine Learning Berlin Germany TU Kaiserslautern Chair Sci Comp Kaiserslautern Germany Offenburg Univ Inst Machine Learning & Analyt Offenburg Germany

ISBN: (纸本)9781728159850

Current training methods for deep neural networks boil down to very high dimensional and non-convex optimization problems which are usually solved by a wide range of stochastic gradient descent methods. While these approaches tend to work in practice, there are still many gaps in the theoretical understanding of key aspects like convergence and generalization guarantees, which are induced by the properties of the optimization surface (loss landscape). In order to gain deeper insights, a number of recent publications proposed methods to visualize and analyze the optimization surfaces. However, the computational cost of these methods are very high, making it hardly possible to use them on larger networks. In this paper, we present the GradVis Toolbox, an open source library for efficient and scalable visualization and analysis of deep neural network loss landscapes in Tensorflow and PyTorch. Introducing more efficient mathematical formulations and a novel parallelization scheme, GradVis allows to plot 2d and 3d projections of optimization surfaces and trajectories, as well as high resolution second order gradient information for large networks.

关键词： visualization eigenvalues parallelization deeplearning second-order

来源：评论

学校读者我要写书评

暂无评论

A Benders Decomposition Approach to Correlation Clustering

A Benders Decomposition Approach to Correlation Clustering

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Jovita Lukasik Margret Keuper Maneesh Singh Julian Yarkony Data and Web Science Group University of Mannheim Germany Verisk Jersey City New Jersey USA

We tackle the problem of graph partitioning for image segmentation using correlation clustering (CC), which we treat as an integer linear program (ILP). We reformulate optimization in the ILP so as to admit efficient optimization via Benders decomposition, a classic technique from operations research. Our Benders decomposition formulation has many subproblems, each associated with a node in the CC instance's graph, which can be solved in parallel. Each Benders subproblem enforces the cycle inequalities corresponding to edges with negative (repulsive) weights attached to its corresponding node in the CC instance. We generate Magnanti-Wong Benders rows in addition to standard Benders rows to accelerate optimization. Our Benders decomposition approach provides a promising new avenue to accelerate optimization for CC, and, in contrast to previous cutting plane approaches, theoretically allows for massive parallelization.

关键词： Correlation Operations research Conferences machine learning Pricing Parallel processing Linear programming

来源：评论

学校读者我要写书评

暂无评论

Reinforcement learning-Based Solution to Power Grid Planning and Operation Under Uncertainties

Reinforcement Learning-Based Solution to Power Grid Planning...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Xiumin Shang Lin Ye Jing Zhang Jingping Yang Jianping Xu Qin Lyu Ruisheng Diao GEIRI North America San Jose CA USA Zhejiang Electric Power Co. Hangzhou China Jinhua Electric Power Co. Jinhua China

ISBN: (纸本)9781665422918;9781665446556

With the ever-increasing stochastic and dynamic behavior observed in today's bulk power systems, securely and economically planning future operational scenarios that meet all reliability standards under uncertainties becomes a challenging computational task, which typically involves searching feasible and suboptimal solutions in a highly dimensional space via massive numerical simulations. This paper presents a novel approach to achieving this goal by adopting the state-of-the-art reinforcement learning algorithm, Soft Actor Critic (SAC). First, the optimization problem of finding feasible solutions under uncertainties is formulated as Markov Decision Process (MDP). Second, a general and flexible framework is developed to train SAC agent by adjusting generator active power outputs for searching feasible operating conditions. A software prototype is developed that verifies the effectiveness of the proposed approach via numerical studies conducted on the planning cases of the SGCC Zhejiang Electric Power Company.

关键词： Training Uncertainty Conferences Power system dynamics Software algorithms Power grids Software

来源：评论

学校读者我要写书评

暂无评论

Scalable Hyperparameter Optimization with Lazy Gaussian Processes 5

Scalable Hyperparameter Optimization with Lazy Gaussian Proc...

引用

5th IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Ram, Raju Mueller, Sabine Pfreundt, Franz-Josef Gauger, Nicolas R. Keuper, Janis Fraunhofer ITWM Competence Ctr High Performance Comp Kaiserslautern Germany Fraunhofer Ctr Machine Learning Berlin Germany TU Kaiserslautern Sci Comp Grp Kaiserslautern Germany Offenburg Univ Inst Machine Learning & Analyt Offenburg Germany

ISBN: (纸本)9781728159850

Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample efficiency, Bayesian Optimization over a Gaussian Processes modeling of the parameter space has become the method of choice. Unfortunately, this approach suffers from a cubic compute complexity due to underlying Cholesky factorization, which makes it very hard to be scaled beyond a small number of sampling steps. In this paper, we present a novel, highly accurate approximation of the underlying Gaussian Process. Reducing its computational complexity from cubic to quadratic allows an efficient strong scaling of Bayesian Optimization while outperforming the previous approach regarding optimization accuracy. First experiments show speedups of a factor of 162 in single node and further speed up by a factor of 5 in a parallel environment.

关键词： Hyper-Parameter Optimization Neural Architecture Search Bayesian Optimization Gaussian Processes Scalability

来源：评论

学校读者我要写书评

暂无评论

Automatic Particle Trajectory Classification in Plasma Simulations

Automatic Particle Trajectory Classification in Plasma Simul...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Stefano Markidis Ivy Peng Artur Podobas Itthinat Jongsuebchoke Gabriel Bengtsson Pawel Herman KTH Royal Institute of Technology Stockholm Sweden Lawrence Livermore National Laboratory Livermore CA USA

Numerical simulations of plasma flows are crucial for advancing our understanding of microscopic processes that drive the global plasma dynamics in fusion devices, space, and astrophysical systems. Identifying and classifying particle trajectories allows us to determine specific on-going acceleration mechanisms, shedding light on essential plasma *** overall goal is to provide a general workflow for exploring particle trajectory space and automatically classifying particle trajectories from plasma simulations in an unsupervised manner. We combine pre-processing techniques, such as Fast Fourier Transform (FFT), with machine learning methods, such as Principal Component Analysis (PCA), k-means clustering algorithms, and silhouette analysis. We demonstrate our workflow by classifying electron trajectories during magnetic reconnection problem. Our method successfully recovers existing results from previous literature without a priori knowledge of the underlying *** workflow can be applied to analyzing particle trajectories in different phenomena, from magnetic reconnection, shocks to magnetospheric flows. The workflow has no dependence on any physics model and can identify particle trajectories and acceleration mechanisms that were not detected before.

关键词： Plasma simulation Magnetic reconnection Conferences machine learning Numerical simulation Trajectory Space exploration

来源：评论

学校读者我要写书评

暂无评论

The Cell Tracking Challenge: 10 years of objective benchmarking

引用

NATURE METHODS 2023年第7期20卷 1010-+页

作者： Maska, Martin Ulman, Vladimir Delgado-Rodriguez, Pablo Gomez-de-Mariscal, Estibaliz Necasova, Tereza Pena, Fidel A. Guerrero Ren, Tsang Ing Meyerowitz, Elliot M. Scherr, Tim Loeffler, Katharina Mikut, Ralf Guo, Tianqi Wang, Yin Allebach, Jan P. Bao, Rina Al-Shakarji, Noor M. Rahmon, Gani Toubal, Imad Eddine Palaniappan, Kannappan Lux, Filip Matula, Petr Sugawara, Ko Magnusson, Klas E. G. Aho, Layton Cohen, Andrew R. Arbelle, Assaf Ben-Haim, Tal Raviv, Tammy Riklin Isensee, Fabian Jaeger, Paul F. Maier-Hein, Klaus H. Zhu, Yanming Ederra, Cristina Urbiola, Ainhoa Meijering, Erik Cunha, Alexandre Munoz-Barrutia, Arrate Kozubek, Michal Ortiz-de-Solorzano, Carlos Masaryk Univ Fac Informat Ctr Biomed Image Anal Brno Czech Republic VSB Tech Univ Ostrava IT4Innovat Natl Supercomp Ctr Ostrava Czech Republic Univ Carlos III Madrid Bioengn Dept Madrid Spain Inst Invest Sanitaria Gregorio Maranon Madrid Spain Inst Gulbenkian Ciencias Opt Cell Biol Oeiras Portugal Univ Fed Pernambuco Ctr Informat Recife PE Brazil CALTECH Beckman Inst Ctr Adv Methods Biol Image Anal Pasadena CA USA CALTECH Div Biol & Biol Engn Pasadena CA USA CALTECH Howard Hughes Med Inst Pasadena CA USA Karlsruhe Inst Technol Inst Automat & Appl Informat Eggenstein Leopoldshafen Germany Purdue Univ Elmore Family Sch Elect & Comp Engn W Lafayette IN USA Boston Childrens Hosp Boston MA USA Harvard Med Sch Boston MA USA Univ Missouri Dept Elect Engn & Comp Sci CIVA Lab Columbia MO USA Ecole Normale Super Lyon Inst Genom Fonct Lyon IGFL Lyon France Ctr Natl Rech Sci CNRS Paris France Raysearch Labs AB Stockholm Sweden Drexel Univ Dept Elect & Comp Engn Philadelphia PA USA Ben Gurion Univ Negev Sch Elect & Comp Engn Beer Sheva Israel German Canc Res Ctr Div Med Image Comp Heidelberg Germany German Canc Res Ctr Helmholtz Imaging Heidelberg Germany German Canc Res Ctr Interact Machine Learning Grp Heidelberg Germany Heidelberg Univ Hosp Dept Radiat Oncol Pattern Anal & Learning Grp Heidelberg Germany Univ New South Wales Sch Comp Sci & Engn Sydney NSW Australia Griffith Univ Nathan Qld Australia Univ Navarra Ctr Appl Med Res Biomed Engn Program Pamplona Spain Univ Navarra Ctr Appl Med Res Ciberonc Pamplona Spain

The Cell Tracking Challenge is an ongoing benchmarking initiative that has become a reference in cell segmentation and tracking algorithm development. Here, we present a significant number of improvements introduced in the challenge since our 2017 report. These include the creation of a new segmentation-only benchmark, the enrichment of the dataset repository with new datasets that increase its diversity and complexity, and the creation of a silver standard reference corpus based on the most competitive results, which will be of particular interest for data-hungry deep learning-based strategies. Furthermore, we present the up-to-date cell segmentation and tracking leaderboards, an in-depth analysis of the relationship between the performance of the state-of-the-art methods and the properties of the datasets and annotations, and two novel, insightful studies about the generalizability and the reusability of top-performing methods. These studies provide critical practical conclusions for both developers and users of traditional and machine learning-based cell segmentation and tracking algorithms. This updated analysis of the Cell Tracking Challenge explores how algorithms for cell segmentation and tracking in both 2D and 3D have advanced in recent years, pointing users to high-performing tools and developers to open challenges.

关键词： Computational platforms and environments Image processing

来源：评论

学校读者我要写书评

暂无评论

Predictions of Steady and Unsteady Flows using machine-learned Surrogate Models

Predictions of Steady and Unsteady Flows using Machine-learn...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Shanti Bhushan Greg W. Burgreen Joshua L. Bowman Ian D. Dettwiller Wesley Brewer Mississippi State University Starkville MS USA Engineer Research and Development Center (ERDC) Vicksburg MS USA DoD HPCMP PET/GDIT Vicksburg MS USA

The applicability of computational fluid dynamics (CFD) based design tools depend on the accuracy and complexity of the physical models, for example turbulence models, which remains an unsolved problem in physics, and rotor models that dictates the computational cost of rotorcraft and wind/hydro turbine farm simulations. The research focuses on investigation of the ability of neural networks to learn correlation between desired modeling variables and flow parameters, thereby providing surrogate models. For the turbulence modeling, the machine learned turbulence model is developed for unsteady boundary layer flow, and the predictions are validated against DNS data and compared with one-equation unsteady Reynolds Averaged Navier-Stokes (URANS) predictions. The machine-learned model performs much better than the URANS model due to its ability to incorporate the non-linear correlation between turbulent stresses and rate-of-strain. The development of the surrogate rotor model builds on the hypothesis that if a model can mimic the axial and tangential momentum deficit generated by a blade resolved model, then it should produce a qualitatively and quantitatively similar wake recovery. An initial validation of the hypothesis was performed, which showed encouraging results.

关键词： Correlation Computational modeling Conferences Computational fluid dynamics Rotors machine learning Predictive models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：