检索结果-内蒙古大学图书馆

2018 ieee/acm machine learning in hpc environments, mlhpc 2018

作者： Li, Zhenyu Davis, James Jarvis, Stephen A. University of Warwick Department of Computer Science Coventry United Kingdom

ISBN: (纸本)9781728101804

machine learning has established itself as a powerful tool for the construction of decision making models and algorithms through the use of statistical techniques on training data. However, a significant impediment to its progress is the time spent training and improving the accuracy of these models - this is a data and compute intensive process, which can often take days, weeks or even months to complete. A common approach to accelerate this process is to employ the use of multiple machines simultaneously, a trait shared with the field of High Performance Computing (hpc) and its clusters. However, existing distributed frameworks for data analytics and machine learning are designed for commodity servers, which do not realize the full potential of a hpc cluster, and thus denies the effective use of a readily available and potentially useful resource. In this work we adapt the application of Apache Spark, a distributed data-flow framework, to support the use of machine learning in hpc environments for the purposes of machine learning. There are inherent challenges to using Spark in this context;memory management, communication costs and synchronization overheads all pose challenges to its efficiency. To this end we introduce: (i) the application of MapRDD, a fine grained distributed data representation;(ii) a task-based all-reduce implementation;and (iii) a new asynchronous Stochastic Gradient Descent (SGD) algorithm using non-blocking all-reduce. We demonstrate up to a 2.6x overall speedup (or a 11.2x theoretical speedup with a Nvidia K80 graphics card), a 82-91 % compute ratio, and a 80% reduction in the memory usage, when training the GoogLeNet model to classify 10% of the ImageNet dataset on a 32-node cluster. We also demonstrate a comparable convergence rate using the new asynchronous SGD with respect to the synchronous method. With increasing use of accelerator cards, larger cluster computers and deeper neural network models, we predict a 2x further speedup (i.e

关键词： machine learning

来源：评论

学校读者我要写书评

暂无评论

Deep learning Evolutionary Optimization for Regression of Rotorcraft Vibrational Spectra

Deep Learning Evolutionary Optimization for Regression of Ro...

引用

2018 ieee/acm machine learning in hpc environments, mlhpc 2018

作者： Martinez, Daniel Brewer, Wesley Behm, Gregory Strelzoff, Andrew Wilson, Andrew Wade, Daniel United States HPCMP PETTT/Engility Corporation United States United States

ISBN: (纸本)9781728101804

A method for Deep Neural Network (DNN) hyperparameter search using evolutionary optimization is proposed for nonlinear high-dimensional multivariate regression problems. Deep networks often lead to extensive hyperparameter searches which can become an ambiguous process due to network complexity. Therefore, we propose a user-friendly method that integrates Dakota optimization library, TensorFlow, and Galaxy hpc workflow management tool to deploy massively parallel function evaluations in a Genetic Algorithm (GA). Deep learning Evolutionary Optimization (DLEO) is the current GA implementation being presented. Compared with random generated and hand-tuned models, DLEO proved to be significantly faster and better searching for optimal architecture hyperparameter configurations. Implementing DLEO allowed us to find models with higher validation accuracies at lower computational costs in less than 72 hours, as compared with weeks of random and manual search. Moreover, DLEO parallel coordinate plots provided valuable insights about network architecture designs and their regression capabilities. © 2018 ieee.

关键词： Genetic algorithms

来源：评论

学校读者我要写书评

暂无评论

Proceedings of mlhpc 2018: machine learning in hpc environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis

Proceedings of MLHPC 2018: Machine Learning in HPC Environme...

引用

2018 ieee/acm machine learning in hpc environments, mlhpc 2018

ISBN: (纸本)9781728101804

The proceedings contain 13 papers. The topics discussed include: large-scale clustering using MPI-based canopy;automated labeling of electron microscopy images using deep learning;large minibatch training on supercomputers with improved accuracy and reduced time to train;communication-efficient parallelization strategy for deep convolutional neural network training;auto-tuning TensorFlow threading model for CPU backend;training speech recognition models on hpc infrastructure;automated parallel data processing engine with application to large-scale feature extraction;aluminum: an asynchronous, GPU-aware communication library optimized for large-scale training of deep neural networks on hpc systems;and on adam trained models and a parallel method to improve the generalization performance.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Understanding Scalability and Fine-Grain Parallelism of Synchronous Data Parallel Training

Understanding Scalability and Fine-Grain Parallelism of Sync...

引用

ieee/acm Workshop on machine learning in High Performance Computing environments (mlhpc)

作者： Jiali Li Bogdan Nicolae Justin Wozniak George Bosilca The University of Tennessee Knoxville USA Argonne National Laboratory USA

ISBN: (纸本)9781728159867

In the age of big data, deep learning has emerged as a powerful tool to extract insight and exploit its value, both in industry and scientific applications. With increasing complexity of learning models and amounts of training data, data-parallel approaches based on frequent all-reduce synchronization steps are increasingly popular. Despite the fact that high-performance computing (hpc) technologies have been designed to address such patterns efficiently, the behavior of data-parallel approaches on hpc platforms is not well understood. To address this issue, in this paper we study the behavior of Horovod, a popular data-parallel approach that relies on MPI, on Theta, a pre-Exascale machine at Argonne National Laboratory. Using two representative applications, we explore two aspects: (1) how performance and scalability is affected by important parameters such as number of nodes, number of workers, threads per node, batch size; (2) how computational phases are interleaved withall-reduce communication phases at fine granularity and what consequences this interleaving has in terms of potential bottlenecks. Our findings show that pipelining of back-propagation, gradient reduction and weight updates mitigate the effects of stragglers during all-reduce only partially. Furthermore, there can be significant delays between weights update, which can be leveraged to mask the overhead of additional background operations that are coupled with the training.

关键词： Deep learning Data-parallel training Behavior analysis behavioral research data parallel Scalability High Performance Computing Fine Grain Size ARGONNE NATIONAL LABORATORY Pipeline processing microparticles Training message passing mannose phosphate isomerase remote procedure calls

来源：评论

学校读者我要写书评

暂无评论

Automated Labeling of Electron Microscopy Images Using Deep learning

Automated Labeling of Electron Microscopy Images Using Deep ...

引用

2018 ieee/acm machine learning in hpc environments, mlhpc 2018

作者： Weber, Gunther H. Ophus, Colin Ramakrishnan, Lavanya Lawrence Berkeley National Laboratory Computational Research Division One Cyclotron Road BerkeleyCA94720 United States Department of Computer Science University of California Davis One Shields Avenue DavisCA95616 United States National Center for Electron Microscopy Molecular Foundry Lawrence Berkeley National Laboratory One Cyclotron Road BerkeleyCA94720 United States

ISBN: (纸本)9781728101804

Searching for scientific data requires metadata providing a relevant context. Today, generating metadata is a time and labor intensive manual process that is often neglected, and important datasets are not accessible through search. We investigate the use of machine learning to generalize metadata from a subset of labeled data, thus increasing the availability of meaningful metadata for search. Specifically, we consider electron microscopy images collected at the National Center for Electron Microscopy (NCEM) at the Lawrence Berkeley National Laboratory (LBNL) and use deep learning to discern characteristics from a small subset of labeled images and transfer labels to the entire image corpus. Relatively small training set sizes and a minimum resolution of 512\times 512 pixels required by the application domain pose unique challenges. We overcome these challenges by using a simple yet powerful convolutional network architecture that limits the number of free parameters to lower the required amount of computational power and reduce the risk of overfitting. We achieve a classification accuracy of approximately 80% in discerning between images recorded in two operating modes of the electron microscope-transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM). We use transfer learning-i.e., re-using the pre-trained convolution layers from the TEM vs. STEM classification problem-to generalize labels and achieve an accuracy of approximately 70 % despite current experiments being limited to small training data sets. We present these predictions as suggestions to domain scientists through a web-based tool to accelerate the labeling process with the goal of further validating our approach and improving the accuracy of automatically created labels. © 2018 ieee.

关键词： Metadata

来源：评论

学校读者我要写书评

暂无评论

Scheduling Optimization of Parallel Linear Algebra Algorithms Using Supervised learning

Scheduling Optimization of Parallel Linear Algebra Algorithm...

引用

ieee/acm Workshop on machine learning in High Performance Computing environments (mlhpc)

作者： gabriel laberge Shahrzad Shirzad Patrick Diehl Hartmut Kaiser Serge Prudhomme Adrian S. Lemoine Department of Mathematics and Industrial Engineering Polytechnique Montréal Montréal QC Canada Center for Computation and Technology Louisiana State University Baton-Rouge LA USA

ISBN: (纸本)9781728159867

Linear algebra algorithms are used widely in a variety of domains, e.g machine learning, numerical physics and video games graphics. For all these applications, loop-level parallelism is required to achieve high performance. However, finding the optimal way to schedule the workload between threads is a non-trivial problem because it depends on the structure of the algorithm being parallelized and the hardware the executable is run on. In the realm of Asynchronous Many Task runtime systems, a key aspect of the scheduling problem is predicting the proper chunk-size, where the chunk-size is defined as the number of iterations of a for-loop are assigned to a thread as one task. In this paper, we study the applications of supervised learning models to predict the chunk-size which yields maximum performance on multiple parallel linear algebra operations using the HPX backend of Blaze's linear algebra library. More precisely, we generate our training and tests sets by measuring performance of the application with different chunk-sizes for multiple linear algebra operations; vector-addition, matrix-vector-multiplication, matrix-matrix addition and matrix-matrix-multiplication. We compare the use of logistic regression, neural networks and decision trees with a newly developed decision tree based model in order to predict the optimal value for chunk-size. Our results show that classical decision trees and our custom decision tree model are able to forecast a chunk-size which results in good performance for the linear algebra operations.

关键词： hpc Linear algebra Loop-level parallelism Supervised learning HPX Blaze AMT Linear algebra Semi-supervised learning HPX gene AMT gene Decision Trees High Performance Computing

来源：评论

学校读者我要写书评

暂无评论

A systematic literature review of machine learning in online personal health data

引用

JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION 2019年第6期26卷 561-576页

作者： Yin, Zhijun Sulieman, Lina M. Malin, Bradley A. Vanderbilt Univ Med Ctr Dept Biomed Informat 2525 West End AveSuite 1412A Nashville TN 37203 USA Vanderbilt Univ Med Ctr Dept Biostat Nashville TN USA Vanderbilt Univ Dept Elect Engn & Comp Sci 221 Kirkland Hall Nashville TN 37235 USA

Objective: User-generated content (UGC) in online environments provides opportunities to learn an individual's health status outside of clinical settings. However, the nature of UGC brings challenges in both data collecting and processing. The purpose of this study is to systematically review the effectiveness of applying machine learning (ML) methodologies to UGC for personal health investigations. Materials and Methods: We searched PubMed, Web of Science, ieee Library, acm library, AAAI library, and the ACL anthology. We focused on research articles that were published in English and in peer-reviewed journals or conference proceedings between 2010 and 2018. Publications that applied ML to UGC with a focus on personal health were identified for further systematic review. Results: We identified 103 eligible studies which we summarized with respect to 5 research categories, 3 data collection strategies, 3 gold standard dataset creation methods, and 4 types of features applied in ML models. Popular off-the-shelf ML models were logistic regression (n=22), support vector machines (n=18), naive Bayes (n=17), ensemble learning (n=12), and deep learning (n=11). The most investigated problems were mental health (n=39) and cancer (n=15). Common health-related aspects extracted from UGC were treatment experience, sentiments and emotions, coping strategies, and social support. Conclusions: The systematic review indicated that ML can be effectively applied to UGC in facilitating the description and inference of personal health. Future research needs to focus on mitigating bias introduced when building study cohorts, creating features from free text, improving clinical creditability of UGC, and model interpretability.

关键词： systematic review machine learning online environment online health community social media patient portal personal health

来源：评论

学校读者我要写书评

暂无评论

Evaluating the Wide Area Classroom After 10,500 hpc Students

Evaluating the Wide Area Classroom After 10,500 HPC Students

引用

ieee/acm Workshop on Education for High-Performance Computing (Eduhpc)

作者： Urbanic, John Maiden, Thomas Pittsburgh Supercomp Ctr Pittsburgh PA 15213 USA

ISBN: (纸本)9781728101903

As of mid-2018 we have taught over 10,500 students in the course of 58 events using the Wide Area Classroom, a novel distributed teaching platform. This has been a successful effort gauged by several important metrics. We describe both the technical and logistical structure of these events as well as the specific hpc curriculums which have proven to be most popular.

关键词： hpc training education supercomputing parallel computing MPI OpenMP OpenACC Big Data machine learning

来源：评论

学校读者我要写书评

暂无评论

Beyond the Memory Wall: A Case for Memory-centric hpc System for Deep learning 51

Beyond the Memory Wall: A Case for Memory-centric HPC System...

引用

51st Annual ieee/acm International Symposium on Microarchitecture (MICRO)

作者： Kwon, Youngeun Rhu, Minsoo Korea Adv Inst Sci & Technol Sch Elect Engn Seoul South Korea

ISBN: (纸本)9781538662403

As the models and the datasets to train deep learning (DL) models scale, system architects are faced with new challenges, one of which is the memory capacity bottleneck, where the limited physical memory inside the accelerator device constrains the algorithm that can be studied. We propose a memory-centric deep learning system that can transparently expand the memory capacity available to the accelerators while also providing fast inter-device communication for parallel training. Our proposal aggregates a pool of memory modules locally within the device side interconnect, which are decoupled from the host interface and function as a vehicle for transparent memory capacity expansion. Compared to conventional systems, our proposal achieves an average 2.8x speedup on eight DL applications and increases the system-wide memory capacity to tens of TBs.

关键词： System architecture hpc machine learning

来源：评论

学校读者我要写书评

暂无评论

Enhancing machine learning optimization algorithms by leveraging memory caching 16

Enhancing machine learning optimization algorithms by levera...

引用

International Conference on High Performance Computing & Simulation (hpcS)

作者： Chakroun, Imen Vander Aa, Tom Ashby, Tom IMEC Exasci Life Lab Leuven Belgium

ISBN: (纸本)9781538678794

Searching a solution space using Stochastic Gradient Descent (SGD) depends on the examples picked at each iteration of the algorithm. Therefore, best practices suggest randomizing the order of training points to visit after every epoch. This random selection is typically implemented as a random shuffling of the order of the training vectors rather than a genuine random training point selection. The shuffling is usually performed after every epoch which results in an extremely low temporal locality of access to the training set. Indeed, each training point is used once, and not before all the other training points have been visited. This means that a cache layer in the memory hierarchy of a modern hpc computer system will have little benefit for the algorithm unless all the training points fit inside that cache.

关键词： Training machine learning Optimization Convergence Computational modeling machine learning algorithms Stochastic processes

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：