检索结果-内蒙古大学图书馆

Multiple data parallelism with HPF and KeLP

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 1999年第3期15卷 393-405页

作者： Merlin, J Baden, S Fink, S Chapman, B Univ Vienna Inst Software Technol & Parallel Syst VCPC A-1090 Vienna Austria Univ Calif San Diego Dept Comp Sci & Engn La Jolla CA 92093 USA

High Performance Fortran (HPF) is an effective language for implementing regular data parallel applications on distributed memory architectures, but it is not well suited to irregular, block-structured applications such as multiblock and adaptive mesh methods. A solution to this problem is to use an SPMD program to coordinate multiple concurrent HPF tasks, each operating on a regular subgrid of the multiblock domain. This paper presents such a system, in which the coordination layer is provided by the C++ class library KeLP. We describe the KeLP-HPF implementation and programming model, and show an example KeLP-HPF multiblock solver together with performance results. (C) 1999 Elsevier Science B.V. All rights reserved.

关键词： C++ data parallelism distributed memory high performance fortran multiblock applications

来源：评论

学校读者我要写书评

暂无评论

Measuring the Effects of data parallelism on Neural Network Training

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2019年第1期20卷 1-49页

作者： Shallue, Christopher J. Lee, Jaehoon Antognini, Joseph Sohl-Dickstein, Jascha Frostig, Roy Dahl, George E. Google Brain 1600 Amphiteatre Pkwy Mountain View CA 94043 USA Google AI Residency Program Mountain View CA USA

Recent hardware developments have dramatically increased the scale of data parallelism available for neural network training. Among the simplest ways to harness next-generation hardware is to increase the batch size in standard mini-batch neural network training algorithms. In this work, we aim to experimentally characterize the effects of increasing the batch size on training time, as measured by the number of steps necessary to reach a goal out-of-sample error. We study how this relationship varies with the training algorithm, model, and data set, and find extremely large variation between workloads. Along the way, we show that disagreements in the literature on how batch size affects model quality can largely be explained by differences in metaparameter tuning and compute budgets at different batch sizes. We find no evidence that larger batch sizes degrade out-of-sample performance. Finally, we discuss the implications of our results on efforts to train neural networks much faster in the future. Our experimental data is publicly available as a database of 71,638,836 loss measurements taken over the course of training for 168,160 individual models across 35 workloads.

关键词： neural networks stochastic gradient descent data parallelism batch size deep learning

来源：评论

学校读者我要写书评

暂无评论

AsynGraph: Maximizing data parallelism for Efficient Iterative Graph Processing on GPUs

引用

ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION 2020年第4期17卷 1–21页

作者： Zhang, Yu Liao, Xiaofei Gu, Lin Jin, Hai Hu, Kan Liu, Haikun He, Bingsheng Huazhong Univ Sci & Technol Natl Engn Res Ctr Big Data Technol & Syst Sch Comp Sci & Technol Cluster & Grid Comp LabServ Comp Technol & Syst Wuhan 430074 Hubei Peoples R China Natl Univ Singapore Singapore Singapore

Recently, iterative graph algorithms are proposed to be handled by GPU-accelerated systems. However, in iterative graph processing, the parallelism of GPU is still underutilized by existing GPU-based solutions. In fact, because of the power-law property of the natural graphs, the paths between a small set of important vertices (e.g., high-degree vertices) play a more important role in iterative graph processing's convergence speed. Based on this fact, for faster iterative graph processing on GPUs, this article develops a novel system, called AsynGraph, to maximize its data parallelism. It first proposes an efficient structure-aware asynchronous processing way. It enables the state propagations of most vertices to be effectively conducted on the GPUs in a concurrent way to get a higher GPU utilization ratio through efficiently handling the paths between the important vertices. Specifically, a graph sketch (consisting of the paths between the important vertices) is extracted from the original graph to serve as a fast bridge for most state propagations. Through efficiently processing this sketch more times within each round of graph processing, higher parallelism of GPU can be utilized to accelerate most state propagations. In addition, a forward-backward antra-path processing way is also adopted to asynchronously handle the vertices on each path, aiming to further boost propagations along paths and also ensure smaller data access cost. In comparison with existing GPU-based systems, i.e., Gunrock, Groute, Tigr, and DiGraph, AsynGraph can speed up iterative graph processing by 3.06-11.52, 2.47-5.40, 2.23-9.65, and 1.41-4.05 times, respectively.

关键词： GPU graph processing data parallelism convergence speed

来源：评论

学校读者我要写书评

暂无评论

Profiling and optimization of Python-based social sciences applications on HPC systems by means of task and data parallelism

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2023年第1期148卷 623-635页

作者： Szustak, Lukasz Lawenda, Marcin Arming, Sebastian Bankhamer, Gregor Schweimer, Christoph Elsaesser, Robert Czestochowa Tech Univ Dabrowskiego 69 PL-42201 Czestochowa Poland Poznan Supercomp & Networking Ctr Jana Pawla II 10 PL-61139 Poznan Poland Paris Lodron Univ Salzburg PLUS Jakob Haringer Str 2 A-5020 Salzburg Austria Know Ctr GmbH Inffeldgasse 13-6 A-8010 Graz Austria

The article presents optimization techniques for two Python-based large-scale social sciences applications: SN (Social Network) Simulator and KPM (Kernel Polynomial Method). These applications use MPI technology to transfer data between computing processes, which in the regular implementation leads to load imbalance and performance degradation. To avoid this effect, we propose a 2-stage optimization. In the first step, the order of tasks is changed, and in the second step, the tasks are divided into smaller ones for easier allocation. In addition, we focus on mitigating performance and memory bottlenecks using modern ccNUMA systems with multiple NUMA domains. As part of the performance analysis, the limitations of communication in data traffic between and within the processor were revealed and resolved through appropriate data allocation. Benchmarking was carried out, examining various environments, including vendors of traditional x86-64 and ARM-based processors for HPC.& COPY;2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://***/licenses/by/4.0/).

关键词： Social sciences applications Profiling Optimization Task parallelism data parallelism HPC Co-design CcNUMA

来源：评论

学校读者我要写书评

暂无评论

Distributed Model Training Based on data parallelism in Edge Computing-Enabled Elastic Optical Networks

引用

IEEE COMMUNICATIONS LETTERS 2021年第4期25卷 1241-1244页

作者： Li, Yajie Zeng, Zebin Li, Jun Yan, Boyuan Zhao, Yongli Zhang, Jie Beijing Univ Posts & Telecommun Sch Elect Engn Beijing 100876 Peoples R China Chalmers Univ Technol Dept Elect Engn S-41296 Gothenburg Sweden

The emergence of edge computing provides an effective solution to execute distributed model training (DMT). The deployment of training data among edge nodes affects the training efficiency and network resource usage. This letter aims for the efficient provisioning of DMT services by optimizing the partition and distribution of training data in edge computing-enabled optical networks. An integer linear programming (ILP) model and a data parallelism deployment algorithm (DPDA) are proposed to solve this problem. The performance of the proposed approaches is evaluated through simulation. Simulation results show that the proposed algorithm can deploy more DMT services compared with benchmark.

关键词： Training Training data Task analysis Computational modeling data models Parallel processing Optical fiber networks data parallelism distributed model training edge computing optical networks

来源：评论

学校读者我要写书评

暂无评论

Quantifying the benefit of thread and data parallelism for fast motion estimation in MPEG-2

引用

ELECTRONICS LETTERS 2006年第13期42卷 747-748页

作者： Chouliaras, V. A. Agha, S. Jacobs, T. R. Dwyer, V. M. Loughborough Univ Technol Dept Elect Engn Loughborough LE11 3TU Leics England

Preliminary results of a concise investigation of the performance benefits obtained by exploiting thread and data parallelism in fast motion estimation algorithms in MPEG-2 are presented. Thirteen such fast ME algorithms were implemented using both thread-pamllel and data-parallel schemes to determine their computational requirements in an embedded environment. The results are then compared to both the default (nonparallelised, full-search) as well as their respective (non-parallelised, fast) versions. Results conclusively demonstrate that both thread and data level parallelism should be exploited for cases where full-search motion estimation is a requirement. By contrast, all fast methods demonstrate that wide data-parallel hardware provides little performance improvement over a conservative, 4-byte single-instruction, multiple-data (SIMD) sum-of-absolute-differences (SAD) coprocessor. In the context of portable, consumer applications, both sets of results strongly suggest a multi-core approach with moderate data-parallel infrastructure.

关键词： coprocessors motion estimation parallel processing video coding MPEG-2 data parallelism data-parallel hardware data-parallel scheme fast motion estimation sum-of-absolute-differences coprocessor thread parallelism thread-parallel scheme

来源：评论

学校读者我要写书评

暂无评论

AI Services-Oriented Dynamic Computing Resource Scheduling Algorithm Based on Distributed data parallelism in Edge Computing Network of Smart Grid

引用

FUTURE INTERNET 2024年第9期16卷 312页

作者： Zou, Jing Xin, Peizhe Wang, Chang Zhang, Heli Wei, Lei Wang, Ying State Grid Econ & Technol Res Inst Ltd Beijing 221005 Peoples R China Beijing Univ Posts & Telecommun Sch Informat & Commun Engn Beijing 100876 Peoples R China State Grid Jiangsu Elect Power Ltd Informat & Telecommun Branch Nanjing 211103 Peoples R China

Massive computational resources are required by a booming number of artificial intelligence (AI) services in the communication network of the smart grid. To alleviate the computational pressure on data centers, edge computing first network (ECFN) can serve as an effective solution to realize distributed model training based on data parallelism for AI services in smart grid. Due to AI services with diversified types, an edge data center has a changing workload in different time periods. Selfish edge data centers from different edge suppliers are reluctant to share their computing resources without a rule for fair competition. AI services-oriented dynamic computational resource scheduling of edge data centers affects both the economic profit of AI service providers and computational resource utilization. This letter mainly discusses the partition and distribution of AI data based on distributed model training and dynamic computational resource scheduling problems among multiple edge data centers for AI services. To this end, a mixed integer linear programming (MILP) model and a Deep Reinforcement Learning (DRL)-based algorithm are proposed. Simulation results show that the proposed DRL-based algorithm outperforms the benchmark in terms of profit of AI service provider, backlog of distributed model training tasks, running time and multi-objective optimization.

关键词： AI services computational resource scheduling dynamic scheduling distributed model training data parallelism edge computing first network smart grid

来源：评论

学校读者我要写书评

暂无评论

Distributed Hybrid DL Cyber-Attacks Detection using data parallelism in Cloud-Dew Computing

Distributed Hybrid DL Cyber-Attacks Detection using Data Par...

引用

IEEE Transportation Electrification Conference and Expo (ITEC)

作者： Moussa, Mohamed Mounir Alazzawi, Lubna Wayne State Univ Dept Elect & Comp Engn Detroit MI 48202 USA

ISBN: (纸本)9798350397420

Mobility is a synonym for ubiquitous devices on the ground and is more related to secure and distributed connectivity. Research in Machine Learning (ML) has developed techniques and systems that parallelize Deep Learning (DL) training over numerous devices or compute nodes as the pace of innovation in these domains has quickened recently. Many systems have failed to deliver overall performance on a variety of DL models as the models' structural complexity increases. Particularly, the knowledge and time needed to map from a suitable distribution strategy to the model are frequently underestimated when it comes to ML scale-up. More specifically, when Autonomous and Connected Vehicles (CAVs) are used as dew devices in cloud-dew computing systems and are planning a safe route in specific road circumstances. Applying parallel training systems to complicated models leads to lower-than-expected performance and nontrivial development overheads in addition to model prototyping. This article identifies and addresses research difficulties in both usability and performance in parallel ML approaches and system implementations, when a CAV generates the dataset for path planning and communicates it on the cloud-dew network, we use a hybrid DL model that uses an LSTM-AE built-in a cyber-attacks detection system. We emulate the situation of numerous CAVs using the same data profile for motion planning on the same road environment. We examine the configuration in a distributed architecture employing synchronous data parallelism, which enables an optimized speedup and improved training runtime by comparing various processing units.

关键词： Mobility CAV data parallelism distributed architecture cloud-dew computing learning

来源：评论

学校读者我要写书评

暂无评论

A Dual-mode Scheduling Algorithm for Task Graphs with data parallelism

A Dual-mode Scheduling Algorithm for Task Graphs with Data P...

引用

IEEE Asia Pacific Conference on Circuits and Systems (APCCAS)

作者： Liu, Yang Meng, Lin Taniguchi, Ittetsu Tomiyama, Hiroyuki Ritsumeikan Univ Dept Elect & Comp Engn Kusatsu Shiga 5258577 Japan

ISBN: (纸本)9781479952304

This paper proposes a task scheduling algorithm for multi/many-core systems. To increase the quality of results on the low computational complexity, our algorithm uses two static priorities which are switched during task scheduling. In our experiments, we compared the proposed algorithm with a state-of-the-art algorithm. The experimental results show that the proposed algorithm yields the better schedule results than the existing algorithm.

关键词： task scheduling multicore task parallelism data parallelism

来源：评论

学校读者我要写书评

暂无评论

Enhancing data parallelism of Fully Homomorphic Encryption 19th

Enhancing Data Parallelism of Fully Homomorphic Encryption

引用

19th International Conference on Information Security and Cryptology (ICISC)

作者： Martins, Paulo Sousa, Leonel Univ Lisbon INESC ID Inst Super Tecn Rua Alves Redol9 P-1000029 Lisbon Portugal

ISBN: (纸本)9783319531762;9783319531779

With Fully Homomorphic Encryption (FHE), it is possible to produce encryptions of the addition and multiplication of encrypted values without access to the private-key. Since homomorphic multiplication is the most burdensome operation of FHE, every possible improvement to it has a significant impact on the performance of the homomorphic evaluation of arbitrary functions. In this paper, we propose an optimized homomorphic multiplication algorithm and apply it to the NTT-based Fast Lattice library (NFLlib), which is a library designed for the implementation of Lattice-based Cryptography (LBC). When implemented with AVX2 Single Instruction Multiple data (SIMD) extensions on a i7-4770k CPU, the proposed algorithm produces a normalized speed-up of 1.93 when compared with the fastest AVX2 implementation of the state of the art. Furthermore, when extended to decryption, the new method achieves a normalized speed-up of 2.0 when compared with related art.

关键词： Homomorphic Encryption data parallelism Ring learning with errors

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：