检索结果-内蒙古大学图书馆

IEEE/ACM 5th International workshop on Extreme Scale Programming Models and Middleware (ESPM2)

作者： Imes, Connor Colin, Alexei Zhang, Naifeng Srivastava, Ajitesh Prasanna, Viktor Walters, John Paul USC Informat Sci Inst Los Angeles CA 90007 USA Univ Southern Calif Los Angeles CA 90007 USA

ISBN: (纸本)9781665422840

As HPC progresses toward exascale, writing applications that are highly efficient, portable, and support programmer productivity is becoming more challenging than ever. The growing scale, diversity, and heterogeneity in compute platforms increases the burden on software to efficiently use available distributed parallel resources. This burden has fallen on developers who, increasingly, are experts in application domains rather than traditional computer scientists and engineers. We propose CASPER-Compiler Abstractions Supporting high performance on Extreme-scale Resources-a novel domain-specific compiler and runtime framework to enable domain scientists to achieve high performance and scalability on complex HPC systems. CASPER extends domain-specific languages with machine learning to map software tasks to distributed, heterogeneous resources, and provides a runtime framework to support a variety of adaptive optimizations in dynamic environments. This paper presents an initial design and analysis of CASPER for synthetic aperture radar and computational fluid dynamics domains.

关键词： parallel programming runtime dynamic compiler high performance computing synthetic aperture radar fluid dynamics

来源：评论

学校读者我要写书评

暂无评论

3rd workshop on Education for high performance computing (EduHiPC - 2021)

3rd Workshop on Education for High Performance Computing (Ed...

引用

IEEE International Conference on high performance computing workshops (HiPCW)

作者： Sheikh Ghafoor Sushil K. Prasad

ISBN: (纸本)9781665410380

high performance computing (HPC) and, in general, Parallel and Distributed computing (PDC) is ubiquitous. Every computing device, from a smartphone to a supercomputer, relies on parallel processing. Compute clusters of multicore and manycore processors (CPUs and GPUs) are routinely used in many subdomains of computer science, such as data science, parallel machine learning and high performance computing. Therefore, it is important for every computing professional (and especially every programmer) to understand how parallelism and distributed computing affect problem solving. It is essential for educators to impart a range of PDC and HPC skills and knowledge at multiple levels within the curriculum of Computer Science (CS), Computer Engineering (CE), and related disciplines such as computational data science. Software industry and research laboratories require people with these skills, more so now. Therefore, they now engage in extensive on-the-job training. Additionally, rapid changes in hardware platforms, languages, and programming environments increasingly challenge educators to decide what to teach and how to teach it, in order to prepare students for careers that are increasingly likely to involve PDC and HPC. EduHiPC aims to provide a forum that brings together academia, industry, government, and non-profit organizations - especially from India, its vicinity, and Asia - for exploring and exchanging experiences and ideas about the inclusion of high-performance, parallel, and distributed computing into undergraduate and graduate curriculum of Computer Science, Computer Engineering, Computational Science, Computational Engineering, and computational courses for STEM and business and other non-STEM disciplines.

关键词： high performance computing Distributed computing PDC gene computer science Curriculum Nonprofit Organizations Education educators supercomputer Smartphone COMPUTER ENGINEERING Laboratory Research Multi-core processors high-performance career

来源：评论

学校读者我要写书评

暂无评论

Sequence-to-sequence models for workload interference prediction on batch processing datacenters

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2020年 110卷 155-166页

作者： Buchaca, David Marcual, Joan LLuis Berral, Josep Carrera, David BSC C Jordi Girona 1-3 Barcelona 08034 Spain UPC BarcelonaTECH Barcelona Spain

Co-scheduling of jobs in data centers is a challenging scenario where jobs can compete for resources, leading to severe slowdowns or failed executions. Efficient job placement on environments where resources are shared requires awareness on how jobs interfere during execution, to go far beyond ineffective resource overbooking techniques. Current techniques, most of which already involve machine learning and job modeling, are based on workload behavior summarization over time, rather than focusing on effective job requirements at each instant of the execution. In this work, we propose a methodology for modeling co-scheduling of jobs on data centers, based on their behavior towards resources and execution time and using sequence-to-sequence models based on recurrent neural networks. The goal is to forecast co-executed jobs footprint on resources throughout their execution time, from the profile shown by the individual jobs, in order to enhance resource manager and scheduler placement decisions. The methods presented herein are validated by using high performance computing benchmarks based on different frameworks (such as Hadoop and Spark) and applications (CPU bound, IO bound, machine learning, SQL queries...). Experiments show that the model can correctly identify the resource usage trends from previously seen and even unseen co-scheduled jobs. (C) 2020 Elsevier B.V. All rights reserved.

关键词： Resource management Sequence-to-sequence Workload interference Deep learning Workload placement

来源：评论

学校读者我要写书评

暂无评论

Message from the workshop Chair

Message from the Workshop Chair

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Metaoptimization on a Distributed System for Deep Reinforcement learning

Metaoptimization on a Distributed System for Deep Reinforcem...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Greg Heinrich Iuri Frosio NVIDIA Nice France NVIDIA Santa Clara USA

ISBN: (纸本)9781728159867

Training intelligent agents through reinforcement learning (RL) is a notoriously unstable procedure. Massive parallelization on GPUs and distributed systems has been exploited to generate a large amount of training experiences and consequently reduce instabilities, but the success of training remains strongly influenced by the choice of the hyperparameters. To overcome this issue, we introduce HyperTrick, a new metaoptimization algorithm, and show its effective application to tune hyperparameters in the case of deep RL, while learning to play different Atari games on a distributed system. Our analysis provides evidence of the interaction between the identification of the optimal hyperparameters and the learned policy, that is peculiar of the case of metaoptimization for deep RL. When compared with state-of-the-art metaoptimization algorithms, HyperTrick is characterized by a simpler implementation and it allows learning similar policies, while making a more effective use of the computational resources in a distributed system.

关键词： Reinforcement learning Distributed systems Metaoptimization Distributed Systems learning learning (artificial intelligence) Intelligent agents Training

来源：评论

学校读者我要写书评

暂无评论

Message from the workshop chair

Proceedings of MLHPC 2019: 5th Workshop on Machine Learning ...

引用

Proceedings of mlhpc 2019: 5th workshop on machine learning in HPC environments - Held in conjunction with SC 2019: The International Conference for high performance computing, Networking, Storage and Analysis 2019年 V页

作者： Lim, Seung-Hwan Oak Ridge National Laboratory United States

来源：评论

学校读者我要写书评

暂无评论

Scalable Hyperparameter Optimization with Lazy Gaussian Processes

Scalable Hyperparameter Optimization with Lazy Gaussian Proc...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Raju Ram Sabine Müller Franz-Josef Pfreundt Nicolas R. Gauger Janis Keuper Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Scientific Computing Group TU Kaiserslautern Germany Fraunhofer ITWM Kaiserslautern Germany Fraunhofer Center Machine Learning Germany Institute for Machine Learning and Analytics Offenburg University Germany

ISBN: (纸本)9781728159867

Most machine learning methods require careful selection of hyper-parameters in order to train a high performing model with good generalization abilities. Hence, several automatic selection algorithms have been introduced to overcome tedious manual (try and error) tuning of these parameters. Due to its very high sample efficiency, Bayesian Optimization over a Gaussian Processes modeling of the parameter space has become the method of choice. Unfortunately, this approach suffers from a cubic compute complexity due to underlying Cholesky factorization, which makes it very hard to be scaled beyond a small number of sampling steps. In this paper, we present a novel, highly accurate approximation of the underlying Gaussian Process. Reducing its computational complexity from cubic to quadratic allows an efficient strong scaling of Bayesian Optimization while outperforming the previous approach regarding optimization accuracy. First experiments show speedups of a factor of 162 in single node and further speed up by a factor of 5 in a parallel environment.

关键词： Hyper-Parameter optimization Neural architecture search Bayesian optimization Gaussian processes Scalability Gaussian processes Scalability Hyperparameter parameter space methods Cholesky factorisation Cubic Speeding

来源：评论

学校读者我要写书评

暂无评论

GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks

GradVis: Visualization and Second Order Analysis of Optimiza...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Avraam Chatzimichailidis Janis Keuper Franz-Josef Pfreundt Nicolas R. Gauger Competence Center High Performance Computing Fraunhofer ITWM Kaiserslautern Germany Chair for Scientific Computing TU Kaiserslautern Kaiserslautern Germany Institute for Machine Learning and Analytics Offenburg University Germany Fraunhofer Center Machine Learning Germany

ISBN: (纸本)9781728159867

Current training methods for deep neural networks boil down to very high dimensional and non-convex optimization problems which are usually solved by a wide range of stochastic gradient descent methods. While these approaches tend to work in practice, there are still many gaps in the theoretical understanding of key aspects like convergence and generalization guarantees, which are induced by the properties of the optimization surface (loss landscape). In order to gain deeper insights, a number of recent publications proposed methods to visualize and analyze the otimization surfaces. However, the computational cost of these methods are very high, making it hardly possible to use them on larger networks. In this paper, we present the GradVis Toolbox, an open source library for efficient and scalable visualization and analysis of deep neural network loss landscapes in Tesorflow and PyTorch. Introducing more efficient mathematical formulations and a novel parallelization scheme, GradVis allows to plot 2d and 3d projections of optimization surfaces and trajectories, as well as high resolution second order gradient information for large networks.

关键词： Visualization Eigenvalues Parallelization Deep-learning Second-order parallelization Visualization eigenvalues Neural network source program library Training Functional training Surfaces Analysis

来源：评论

学校读者我要写书评

暂无评论

Python Workflows on HPC Systems

Python Workflows on HPC Systems

引用

workshop on Python for high-performance and Scientific computing (PyHPC)

作者： Dominik Straßel Philipp Reusch Janis Keuper Fraunhofer ITWM Kaiserslautern Germany Offenburg University Germany

ISBN: (数字)9780738110868

ISBN: (纸本)9781665422864

The recent successes and wide spread application of compute intensive machine learning and data analytics methods have been boosting the usage of the Python programming language on HPC systems. While Python provides many advantages for the users, it has not been designed with a focus on multi-user environments or parallel programming - making it quite challenging to maintain stable and secure Python workflows on a HPC system. In this paper, we analyze the key problems induced by the usage of Python on HPC clusters and sketch appropriate workarounds for efficiently maintaining multi-user Python software environments, securing and restricting resources of Python jobs and containing Python processes, while focusing on Deep learning applications running on GPU clusters.

关键词： Python Graphics processing units machine learning Kernel Linux Standards Software

来源：评论

学校读者我要写书评

暂无评论

Understanding Scalability and Fine-Grain Parallelism of Synchronous Data Parallel Training

Understanding Scalability and Fine-Grain Parallelism of Sync...

引用

IEEE/ACM workshop on machine learning in high performance computing environments (mlhpc)

作者： Jiali Li Bogdan Nicolae Justin Wozniak George Bosilca The University of Tennessee Knoxville USA Argonne National Laboratory USA

ISBN: (纸本)9781728159867

In the age of big data, deep learning has emerged as a powerful tool to extract insight and exploit its value, both in industry and scientific applications. With increasing complexity of learning models and amounts of training data, data-parallel approaches based on frequent all-reduce synchronization steps are increasingly popular. Despite the fact that high-performance computing (HPC) technologies have been designed to address such patterns efficiently, the behavior of data-parallel approaches on HPC platforms is not well understood. To address this issue, in this paper we study the behavior of Horovod, a popular data-parallel approach that relies on MPI, on Theta, a pre-Exascale machine at Argonne National Laboratory. Using two representative applications, we explore two aspects: (1) how performance and scalability is affected by important parameters such as number of nodes, number of workers, threads per node, batch size; (2) how computational phases are interleaved withall-reduce communication phases at fine granularity and what consequences this interleaving has in terms of potential bottlenecks. Our findings show that pipelining of back-propagation, gradient reduction and weight updates mitigate the effects of stragglers during all-reduce only partially. Furthermore, there can be significant delays between weights update, which can be leveraged to mask the overhead of additional background operations that are coupled with the training.

关键词： Deep learning Data-parallel training Behavior analysis behavioral research data parallel Scalability high performance computing Fine Grain Size ARGONNE NATIONAL LABORATORY Pipeline processing microparticles Training message passing mannose phosphate isomerase remote procedure calls

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：