检索结果-内蒙古大学图书馆

Proceedings of mlhpc 2016: machine learning in hpc environments - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

Proceedings of MLHPC 2016: Machine Learning in HPC Environme...

引用

2016 machine learning in hpc environments, mlhpc 2016

ISBN: (纸本)9781509038824

The proceedings contain 7 papers. The topics discussed include: communication quantization for data-parallel training of deep neural networks;performance-portable autotuning of OpenCL kernels for convolutional layers of deep neural networks;distributed training of deep neural networks: theoretical and practical limits of parallel scalability;a scalable parallel q-learning algorithm for resource constrained decentralized computing environments;parallel evolutionary optimization for neuromorphic network training;a study of complex deep learning networks on high performance, neuromorphic, and quantum computers;and practical efficiency of asynchronous stochastic gradient descent.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Scalable Parallel Q-learning Algorithm for Resource Constrained Decentralized Computing environments

A Scalable Parallel Q-Learning Algorithm for Resource Constr...

引用

2nd Workshop on machine learning in hpc environments (mlhpc)

作者： Camelo, Miguel Famaey, Jeroen Latre, Steven Univ Antwerp IMEC Dept Math & Comp Sci Middelheimlaan 1 B-2020 Antwerp Belgium

ISBN: (纸本)9781509038824

The Internet of Things (IoT) is more and more becoming a platform for mission critical applications with stringent requirements in terms of response time and mobility. Therefore, a centralized High Performance Computing (hpc) environment is often not suitable or simply non-existing. Instead, there is a need for a scalable hpc model that supports the deployment of applications on the decentralized but resource constrained devices of the IoT. Recently, Reinforcement learning (RL) algorithms have been used for decision making within applications by directly interacting with the environment. However, most RL algorithms are designed for centralized environments and are time and resource consuming. Therefore, they are not applicable to such constrained decentralized computing environments. In this paper, we propose a scalable Parallel Q-learning (PQL) algorithm for resource constrained environments. By combining a table partition strategy together with a co-allocation of both processing and storage, we can significantly reduce the individual resource cost and, at the same time, guarantee convergence and minimize the communication cost. Experimental results show that our algorithm reduces the required training in proportion of the number of Q-learning agents and, in terms of execution time, it is up to 24 times faster than several well-known PQL algorithms.

关键词： Internet of things

来源：评论

学校读者我要写书评

暂无评论

A Study of Complex Deep learning Networks on High Performance, Neuromorphic, and Quantum Computers

A Study of Complex Deep Learning Networks on High Performanc...

引用

2nd Workshop on machine learning in hpc environments (mlhpc)

作者： Potok, Thomas E. Schuman, Catherine D. Young, Steven R. Patton, Robert M. Spedalieri, Federico Liu, Jeremy Yao, Ke-Thia Rose, Garrett Chakma, Gangotree Oak Ridge Natl Lab 1 Bethel Valley Rd Oak Ridge TN 37830 USA USC Informat Sci Inst Marina Del Rey CA USA Univ Tennessee Knoxville TN USA

ISBN: (纸本)9781509038824

Current Deep learning models use highly optimized convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers with a fairly simple layered network topology, i.e., highly connected layers, without intra-layer connections. Complex topologies have been proposed, but are intractable to train on current systems. Building the topologies of the deep learning network requires hand tuning, and implementing the network in hardware is expensive in both cost and power. In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (hpc) to automatically determine network topology, and neuromorphic computing for a lowpower hardware implementation. Due to input size limitations of current quantum computers we use the MNIST dataset for our evaluation. The results show the possibility of using the three architectures in tandem to explore complex deep learning networks that are untrainable using a von Neumann architecture. We show that a quantum computer can find high quality values of intra-layer connections and weights, while yielding a tractable time result as the complexity of the network increases;a high performance computer can find optimal layer-based topologies;and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware. This represents a new capability that is not feasible with current von Neumann architecture. It potentially enables the ability to solve very complicated problems unsolvable with current computing technologies.

关键词： Quantum computers

来源：评论

学校读者我要写书评

暂无评论

A Scalable Parallel Q-learning Algorithm for Resource Constrained Decentralized Computing environments

A Scalable Parallel Q-Learning Algorithm for Resource Constr...

引用

Workshop on machine learning in hpc environments (mlhpc)

作者： Miguel Camelo Jeroen Famaey Steven Latré University of Antwerp-imec Antwerp Belgium

关键词： Convergence Training Partitioning algorithms Heuristic algorithms Algorithm design and analysis Scalability machine learning algorithms

来源：评论

学校读者我要写书评

暂无评论

A Study of Complex Deep learning Networks on High Performance, Neuromorphic, and Quantum Computers

A Study of Complex Deep Learning Networks on High Performanc...

引用

Workshop on machine learning in hpc environments (mlhpc)

作者： Thomas E. Potok Catherine D. Schuman Steven R. Young Robert M. Patton Federico Spedalieri Jeremy Liu Ke-Thia Yao Garrett Rose Gangotree Chakma Oak Ridge National Laboratory USC Information Sciences Institute Marina del Rey CA USA University of Tennessee Knoxville TN USA

ISBN: (纸本)9781509038831

Current Deep learning models use highly optimized convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers with a fairly simple layered network topology, i.e., highly connected layers, without intra-layer connections. Complex topologies have been proposed, but are intractable to train on current systems. Building the topologies of the deep learning network requires hand tuning, and implementing the network in hardware is expensive in both cost and *** this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (hpc) to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. Due to input size limitations of current quantum computers we use the MNIST dataset for our *** results show the possibility of using the three architectures in tandem to explore complex deep learning networks that are untrainable using a von Neumann architecture. We show that a quantum computer can find high quality values of intra-layer connections and weights, while yielding a tractable time result as the complexity of the network increases; a high performance computer can find optimal layer-based topologies; and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive *** represents a new capability that is not feasible with current von Neumann architecture. It potentially enables the ability to solve very complicated problems unsolvable with current computing technologies.

关键词： machine learning Quantum computing Computers Neuromorphics Network topology Topology Computer architecture

来源：评论

学校读者我要写书评

暂无评论

Practical Efficiency of Asynchronous Stochastic Gradient Descent

Practical Efficiency of Asynchronous Stochastic Gradient Des...

引用

2nd Workshop on machine learning in hpc environments (mlhpc)

作者： Bhardwaj, Onkar Cong, Guojing IBM TJ Watson Res Yorktown Hts NY 10598 USA

ISBN: (纸本)9781509038824

Stochastic gradient descent (SGD) and its distributed variants are essential to leverage modern computing resources for large-scale machine learning tasks. ASGD [1] is one of the most popular asynchronous distributed variant of SGD. Recent mathematical analyses have shown that with certain assumptions on the learning task (and ignoring communication cost), ASGD exhibits linear speed-up asymptotically. However, as practically observed, ASGD does not lead linear speed-up as we increase the number of learners. Motivated by this, we investigate finite time convergence properties of ASGD. We observe that the learning rate used by mathematical analyses to guarantee linear speed-up can be very small (and practically sub-optimal with respect to convergence speed) as opposed to practically chosen learning rates (for quick convergence) which exhibit sub-linear speed-up. We show that such an observation can in fact be supported by mathematical analysis, i.e., in the finite time regime, better convergence rate guarantees can be proven for ASGD with small number of learners, thus indicating lack of linear speed up as we increase the number of learners. Thus we conclude that even with ignoring communication cost, there is an inherent inefficiency in ASGD with respect to increasing the number of learners.

关键词： learning systems

来源：评论

学校读者我要写书评

暂无评论

Evaluation of Deep learning Frameworks over Different hpc Architectures 37

Evaluation of Deep Learning Frameworks over Different HPC Ar...

引用

37th IEEE International Conference on Distributed Computing Systems (ICDCS)

作者： Shams, Shayan Platania, Richard Lee, Kisung Park, Seung-Jong Ctr Computat & Technol Div Comp Sci & Engn Baton Rouge LA 70803 USA

ISBN: (纸本)9781538617915

Recent advances in deep learning have enabled researchers across many disciplines to uncover new insights about large datasets. Deep neural networks have shown applicability to image, time-series, textual, and other data, all of which are available in a plethora of research fields. However, their computational complexity and large memory overhead requires advanced software and hardware technologies to train neural networks in a reasonable amount of time. To make this possible, there has been an influx in development of deep learning software that aim to leverage advanced hardware resources. In order to better understand the performance implications of deep learning frameworks over these different resources, we analyze the performance of three different frameworks, Caffe, TensorFlow, and Apache SINGA, over several hardware environments. This includes scaling up and out with single-and multi-node setups using different CPU and GPU technologies. Notably, we investigate the performance characteristics of NVIDIA's state-of-the-art hardware technology, NVLink, and also Intel's Knights Landing, the most advanced Intel product for deep learning, with respect to training time and utilization. To our best knowledge, this is the first work concerning deep learning bench-marking with NVLink and Knights Landing. Through these experiments, we provide analysis of the frameworks' performance over different hardware environments in terms of speed and scaling. As a result of this work, better insight is given towards both using and developing deep learning tools that cater to current and upcoming hardware technologies.

关键词： machine learning Hardware Graphics processing units Training Tools Benchmark testing

来源：评论

学校读者我要写书评

暂无评论

A Comprehensive View of Personalized Federated learning on Heterogeneous Clinical Datasets 9

A Comprehensive View of Personalized Federated Learning on H...

引用

9th machine learning for Healthcare Conference, MLHC 2024

作者： Tavakoli, Fatemeh Emerson, D.B. Ayromlou, Sana Jewell, John Krishnan, Amrit Zhang, Yuchong Verma, Amol Razak, Fahad Vector Institute Toronto Canada Unity Health Toronto Toronto Canada

Federated learning (FL) is increasingly being recognized as a key approach to overcoming the data silos that so frequently obstruct the training and deployment of machine-learning models in clinical settings. This work contributes to a growing body of FL research specifically focused on clinical applications along three important directions. First, we expand the FLamby benchmark (du Terrail et al., 2022a) to include a comprehensive evaluation of personalized FL methods and demonstrate substantive performance improvements over the original results. Next, we advocate for a comprehensive checkpointing and evaluation framework for FL to reflect practical settings and provide multiple comparison baselines. To this end, an open-source library aimed at making FL experimentation simpler and more reproducible is released. Finally, we propose an important ablation of PerFCL (Zhang et al., 2022). This ablation results in a natural extension of FENDA (Kim et al., 2016) to the FL setting. Experiments conducted on the FLamby benchmark and GEMINI datasets (Verma et al., 2017) show that the proposed approach is robust to heterogeneous clinical data and often outperforms existing global and personalized FL techniques, including PerFCL. © 2024 F. Tavakoli, D.B. Emerson, S. Ayromlou, J. Jewell, A. Krishnan, Y. Zhang, A. Verma & F. Razak.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

OpenCL caffe: Accelerating and enabling a cross platform machine learning framework 16

OpenCL caffe: Accelerating and enabling a cross platform mac...

引用

4th International Workshop on OpenCL, IWOCL 2016

作者： Gu, Junli Liu, Yibing Gao, Yuan Zhu, Maohua

ISBN: (纸本)9781450343381

Deep neural networks (DNN) achieved significant breakthrough in vision recognition in 2012 and quickly became the leading machine learning algorithm in Big Data based large scale object recognition applications. The successful deployment of DNN based applications pose challenges for a cross platform software framework that enable multiple user scenarios, including offline model training on hpc clusters and online recognition in embedded environments. Existing DNN frameworks are mostly focused on a closed format CUDA implementations, which is limiting of deploy breadth of DNN hardware systems. This paper presents OpenCL™ caffe, which targets in transforming the popular CUDA based framework caffe [1] into open standard OpenCL backend. The goal is to enable a heterogeneous platform compatible DNN framework and achieve competitive performance based on OpenCL tool chain. Due to DNN models' high complexity, we use a two-phase strategy. First we introduce the OpenCL porting strategies that guarantee algorithm convergence;then we analyze OpenCL's performance bottlenecks in DNN domain and propose a few optimization techniques including batched manner data layout and multiple command queues to better map the problem size into existing BLAS library, improve hardware resources utilization and boost OpenCL runtime efficiency. We verify OpenCL caffe's successful offline training and online recognition on both server-end and consumer-end GPUs. Experimental results show that the phase-two's optimized OpenCL caffe achieved a 4.5x speedup without modifying BLAS library. The user can directly run mainstream DNN models and achieves the best performance for a specific processors by choosing the optimal batch number depending on H/W properties and input data size.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Job placement advisor based on turnaround predictions for hpc hybrid clouds

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2017年 67卷 35-46页

作者： Cunha, Renato L. F. Rodrigues, Eduardo R. Tizzei, Leonardo P. Netto, Marco A. S. IBM Res Yorktown Hts NY 10598 USA

Several companies and research institutes are moving their CPU-intensive applications to hybrid High Performance Computing (hpc) cloud environments. Such a shift depends on the creation of software systems that help users decide where a job should be placed considering execution time and queue wait time to access on-premise clusters. Relying blindly on turnaround prediction techniques will affect negatively response times inside hpc cloud environments. This paper introduces a tool to make job placement decisions in hpc hybrid cloud environments taking into account the inaccuracy of execution and waiting time predictions. We used job traces from real supercomputing centers to run our experiments, and compared the performance between environments using real speedup curves. We also extended a state-of-the-art machine learning based predictor to work with data from the cluster scheduler. Our main findings are as follows: (i) depending on workload characteristics, there is a turning point where predictions should be disregarded in favor of a more conservative decision to minimize job turnaround times and (ii) scheduler data plays a key role in improving predictions generated with machine learning using job trace data our experiments showed around 20% prediction accuracy improvements. (C) 2016 Elsevier B.V. All rights reserved.

关键词： hpc Cloud computing Hybrid cloud machine learning Advisory system Runtime and wait time predictions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：