检索结果-内蒙古大学图书馆

international Conference on parallel and distributed systems (ICPADS)

作者： Hanchi Shen Jun Li Kang Wei Pengcheng Xia Sirui Tian Ming Ding Zengxiang Li Nanjing University of Science and Technology China Data61 Australia Digital Research Institute ENN Group China

Federated learning (FL) has become a promising machine learning (ML) paradigm for training machine learning models over distributed datasets, owing to its low communication costs and privacy preserving property. To date, the most commonly adopted model fusion mechanism in FL is average aggregation. However, it has been shown that this average aggregation mechanism performs poorly in heterogeneous systems, especially for non-independent and identically distributed (NonIID) data. In order to address this challenge, we propose a weighted FL model aggregation strategy for each client based on clustering, termed CluFL. Specifically, CluFL first measures the similarities among uploaded models from clients through their parameters using a spectral clustering algorithm. Then, CluFL assigns aggregation weights according to the similarity of the intra-cluster global model for each cluster and the average model across the clusters. Further, we derive a convergence bound on the CluFL algorithm considering a practical nonconvex setting of neural network training. This bound reveals that the proposed CluFL algorithm can achieve a convergence speed in the order of O(1/T). Extensive experiments have been conducted on both FashionMNIST and CIFAR-10 datasets and show that CluFL outperforms the state-of-the-art FL algorithms in terms of accuracy and communication efficiency.

关键词： Training Data privacy Costs Federated learning Neural networks Clustering algorithms distributed databases

来源：评论

学校读者我要写书评

暂无评论

Coded parallelism for distributed Deep Learning

Coded Parallelism for Distributed Deep Learning

引用

IEEE international symposium on Information Theory

作者： Songting Ji Zhaoyang Zhang Zhaohui Yang Richeng Jin Qianqian Yang College of Information Science and Electronic Engineering Zhejiang University Hangzhou China Zhejiang Provincial Key Laboratory of Info. Proc. Commun. & Netw. (IPCAN) Hangzhou China International Joint Innovation Center Zhejiang University Haining China

With the rapid development of deep learning, the parameters of modern neural network models, especially in the field of Natural Language Processing (NLP) are extremely huge. When the parameters of the model are larger even than the storage memory of a single device, it is necessary to split the original big learning model into different parts with each part assigned to one device, thus realizing joint model training over different devices (i.e., distributed training). In this paper, we aim to introduce the advanced coding scheme into the distributed parallel framework, which leads to the perfect combination of coding and the underlying calculation of neural networks. The proposed scheme is not only able to avoid the impact of poor computing power or low bandwidth and even dropped devices (stragglers) on system performance but also reduce the communication load between different devices, thereby greatly improving the performance of distributed parallel systems.

关键词：

来源：评论

学校读者我要写书评

暂无评论

A Study on Integrating HRRC Network for Robust CVR Solutions

A Study on Integrating HRRC Network for Robust CVR Solutions

引用

international Conference on Ubiquitous and Future Networks, ICUFN

作者： Sungsoo Choi Jimyoung Kang Power ICT Research Center Korea Electrotechnology Research Institute (KERI) Sa-dong Ansan S. Korea

ISBN: (数字)9798350385298

ISBN: (纸本)9798350385304

Conservation Voltage Reduction (CVR) technology optimizes distribution grid voltage to reduce power demand and losses, improving power system efficiency and stability. Effective real-time monitoring and control of power system voltage and load need a reliable communication network. Ensuring time-sensitive stability in measuring and sharing extensive data from distributed power resources is crucial, especially in unpredictable electrical environments. This paper proposes a solution in the form of a High Reliable Redundancy Communication (HRRC) network. HRRC aims to enhance the network performance and reliability of CVR technology for future power grid systems. The design and implementation of HRRC involves Ethernet-based High Availability Seamless Redundancy (HSR), parallel Redundancy Protocol (PRP), and Merging Unit (MU), offering robust solutions for CVR.

关键词： Protocols Voltage measurement Power measurement Termination of employment distributed databases Power system stability Real-time systems

来源：评论

学校读者我要写书评

暂无评论

Improving Confidence of Uncertain Knowledge Graphs by Crowdsourcing with Limited Budget

Improving Confidence of Uncertain Knowledge Graphs by Crowds...

引用

international Conference on parallel and distributed systems (ICPADS)

作者： Haodi Zhang Wenxi Huang Chenyu Xu Zhengbang Yang Hongxin Zhou Fengtian Qi Chen Zhang Kaishun Wu School of Computer Science and Software Engineering Shenzhen University Department of Computing Hong Kong Polytechnic University

Knowledge graphs (KGs), either constructed automatically from texts or collected manually from crowdsourcing workers, may contain uncertainty. The uncertainty may propagate into the knowledge graph embedding and downstream tasks, which is potentially harmful, especially for those confidencesensitive applications such as medical diagnostic suggestion. Crowdsourcing workers with domain knowledge can help improve the data quality of knowledge graphs, by knowledge checking. However, due to the large scale of knowledge graphs and the limitation of adequate crowdsourcing workers, it is unrealistic to check all triplets in a knowledge graph to improve the data quality. Therefore, in this paper, we propose a crowdsourcing framework that efficiently improves the confidence of knowledge graphs with limited budget. We instantiate the framework in the medical domain and conduct a series of experiments with realworld medical data. We deploy the framework for knowledge graph embedding UKGE and corresponding downstream tasks. The experimental results show that the proposed method efficiently improves the quality of the knowledge graphs, and hence improves the performance of probabilistic knowledge graph embedding in the downstream tasks.

关键词： Crowdsourcing Uncertainty Data integrity distributed databases Knowledge graphs Probabilistic logic Medical diagnosis

来源：评论

学校读者我要写书评

暂无评论

Modelling the behavioural component of the emergent parallel processes of working with graph databases using Petri net-tools

引用

international JOURNAL OF parallel EMERGENT AND distributed systems 2021年第6期36卷 498-515页

作者： Velykodniy, Stanislav Burlachenko, Zhanna Zaitseva-Velykodna, Svitlana Odessa State Environm Univ Automated Environm Monitoring Syst Dept Lvivska StrB 15Off 130 UA-65016 Odessa Ukraine Odessa State Environm Univ Informat Dept Odessa Ukraine

The article focuses on the transformation of the behavioural UML diagrams modelled by the authors, which reflect the emergent manifestations of the life cycles of working with graph databases (GDB), to models in the form of Petri nets. Such a transformation process was carried out for the diagrams of objects, states, sequences, and activities. The extended UML 2.5 notation and Enterprise Architect 14.0 CASE tool were used in the formation of the project architecture. Modelling of the emergent parallel processes of working with GDBs in the form of a Petri net might initially seem to make the sequence more complicated for the designer to perceive than while using UML modelling, in which the relationships between entities on a diagram are obvious to the user. Conversely, the representation of behavioural models in the form of Petri nets facilitates the reflection of other important emergent points, in particular, by demonstrating objects relationship, dynamic information updates, components, or other necessary information concerning parallel GDB design.

关键词： Graph database Petri net UML diagram behavioural modelling structure rendering

来源：评论

学校读者我要写书评

暂无评论

Improving Real-Time Service Quality Through parallel Monte Carlo Tree Search

Improving Real-Time Service Quality Through Parallel Monte C...

引用

international Workshop on Quality of Service

作者： Jiajia Zhang Shuman Zhuang Yulin Wu School of Computer Science Harbin Institite of Technology(ShenZhen) Shenzhen China

ISBN: (数字)9798350350128

ISBN: (纸本)9798350350135

In this paper, we present a new algorithm named distributed Multi-root Pipeline MCTS (DMP-MCTS), to improve real-time search efficiency in multi-machine scenarios. By utilizing root parallel technology with significance detection and pipeline pattern for parallel MCTS (3PMCTS), we not only reduce the repetitive computing tasks among subtrees but also allow for flexible operation time based resource allocation. The experiment shows this work achieves better computational performance under linear acceleration conditions, compared with other existing works.

关键词： Monte Carlo methods Pipelines Quality of service parallel processing Real-time systems Resource management

来源：评论

学校读者我要写书评

暂无评论

Scalable Data parallel distributed Training for Graph Neural Networks

Scalable Data Parallel Distributed Training for Graph Neural...

引用

IEEE international symposium on parallel and distributed Processing Workshops and Phd Forum (IPDPSW)

作者： Sohei Koyama Osamu Tatebe University of Tsukuba Tsukuba Ibaraki Japan

ISBN: (数字)9781665497473

ISBN: (纸本)9781665497480

Graph neural networks (GNNs) operate on data represented as graphs, and are useful for a wide variety of tasks from chemical reaction and protein structure prediction to content recommendation systems. However, training for large graphs and improving training performance remain significant challenges. Existing distributed training systems partition a graph among all compute nodes to train for large graphs; however, this results in a communication overhead to degrade training performance. In this study, to solve these two problems, we propose a scalable data-parallel distributed GNN training system designed to partition a graph redundantly. It is implemented using remote direct memory access (RDMA) and nonblocking active messages to efficiently utilize network performance and hide communication overhead by overlapping with the training computation. Experimental results are presented to show the strong scalability of the proposed approach, which achieved parallel efficiencies of 0.93 using eight compute nodes for the ogbn-products dataset in the Open Graph Benchmark (OGB) and 0.95 based on two compute nodes using 32 compute nodes for the ogbn-papers100M dataset. The proposed system exhibited a training performance 18.9% better than the state-of-the-art DistDGL, even with only a single compute node. The results demonstrate that the proposed approach may be considered a promising method to achieve scalable training performance for large graphs.

关键词： Training Proteins distributed processing Scalability Conferences Memory management distributed databases

来源：评论

学校读者我要写书评

暂无评论

SE-LeNet: A Data Reconstruction Method for Dissolved Oxygen in Tropical Pacific with Deep Learning

SE-LeNet: A Data Reconstruction Method for Dissolved Oxygen ...

引用

international symposium on parallel and distributed Processing with Applications, ISPA

作者： Ruixin Cao Shuchang Wang Senliang Bao Xiaoyong Li Jiaming Tan Chengcheng Shao College of Meteorology and Oceanology National University of Defense Technology China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Dissolved Oxygen (DO) is an important indicator of water quality and sea-air interactions, and the formation mechanism and dramatic change of Oxygen Minimum Zones (OMZs) are closely related to the activities of marine organisms and marine ecological events. However, due to the sparsity and scarcity of DO data, the studies on OMZs get a lot of limits. This paper proposes a deep learning method called SE-LeNet to reconstruct underwater multilayer DO, which integrates the channel attention module called SE-Block into the LeNet model. Simultaneously, the relationship between sea surface elements (e.g. spCO 2 , PhyC) and DO are created based on CEMES reanalysis data, which has never been considered in the reconstructed model before. Extensive experiments are conducted, demonstrating that the deep learning methods notably outperform the traditional statistical and other machine learning methods when processing high-resolution multi-dimensional data with spatial information. Moreover, the result accuracy is improved after adding the SEBlock module, and the R 2 and RMSE of SE-LeNet are as low as 0.97 and 12.00 μmol/kg, respectively. SE-LeNet can be successfully used in reconstructing DO data and then promoting the studies on OMZs in the ocean field.

关键词： Deep learning Surface reconstruction Sea surface Oxygen Biological system modeling Water quality Nonhomogeneous media Spatial databases Surface roughness Surface treatment

来源：评论

学校读者我要写书评

暂无评论

RLRP: High-Efficient Data Placement with Reinforcement Learning for Modern distributed Storage systems

RLRP: High-Efficient Data Placement with Reinforcement Learn...

引用

international symposium on parallel and distributed Processing (IPDPS)

作者： Kai Lu Nannan Zhao Jiguang Wan Changhong Fei Wei Zhao Tongliang Deng Wuhan National Laboratory for Optoelectronics Huazhong University of Science and Technology Wuhan China The School of Computer Science Northwestern Polytechnical University Xi&#x0027 an China SenseTime Research Shenzhen China

Modern distributed storage systems with massive data and storage nodes pose higher requirements to the data placement strategy. Furthermore, with emerged new storage devices, heterogeneous storage architecture has become increasingly common and popular. However, traditional strategies expose great limitations in the face of these requirements, especially do not well consider distinct characteristics of heterogeneous storage nodes yet, which will lead to suboptimal performance. In this paper, we present and evaluate the RLRP, a deep reinforcement learning (RL) based replica placement strategy. RLRP constructs placement and migration agents through the Deep-Q-Network (DQN) model to achieve fair distribution and adaptive data migration. Besides, RLRP provides optimal performance for heterogeneous environment by an attentional Long Short-term Memory (LSTM) model. Finally, RLRP adopts Stagewise Training and Model fine-tuning to accelerate the training of RL models with large-scale state and action space. RLRP is implemented on Park and the evaluation results indicate RLRP is a highly efficient data placement strategy for modern distributed storage systems. RLRP can reduce read latency by 10%∼50% in heterogeneous environment compared with existing strategies. In addition, RLRP is used in the real-world system Ceph, which improves the read performance of Ceph by 30%∼40%.

关键词： Training Performance evaluation Adaptation models distributed processing distributed databases Reinforcement learning Data models

来源：评论

学校读者我要写书评

暂无评论

Understanding Recursive Divide-and-Conquer Dynamic Programs in Fork-Join and Data-Flow Execution Models

Understanding Recursive Divide-and-Conquer Dynamic Programs ...

引用

35th IEEE international parallel and distributed Processing symposium (IPDPS)

作者： Nookala, Poornima Ahmad, Zafar Javanmard, Mohammad Mahdi Kong, Martin Chowdhury, Rezaul Harrison, Robert IIT Chicago IL 60616 USA SUNY Stony Brook Stony Brook NY 11794 USA Univ Oklahoma Norman OK 73019 USA

ISBN: (纸本)9781665435772

On shared-memory multicore machines, classic two-way recursive divide-and-conquer algorithms are implemented using common fork-join based parallel programming paradigms such as Intel Cilk+ or OpenMP. However, in such parallel paradigms, the use of joins for synchronization may lead to artificial dependencies among function calls which are not implied by the underlying DP recurrence. These artificial dependencies can increase the span asymptotically and thus reduce parallelism. From a practical perspective, they can lead to resource underutilization, i.e., threads becoming idle. To eliminate such artificial dependencies, task-based runtime systems and data-flow parallel paradigms, such as Concurrent Collections (CnC), PaRSEC, and Legion have been introduced. Such parallel paradigms and runtime systems overcome the limitations of fork-join parallelism by specifying data dependencies at a finer granularity and allowing tasks to execute as soon as dependencies are satisfied. In this paper, we investigate how the performance of data-flow implementations of recursive divide-and-conquer based DP algorithms compare with fork-join implementations. We have designed and implemented data-flow versions of DP algorithms in Intel CnC and compared the performance with fork-join based implementations in OpenMP. Considering different execution parameters (e.g., algorithmic properties such as recursive base size as well as machine configuration such as the number of physical cores, etc), our results confirm that a data-flow based implementation outperforms its fork-join based counterpart when due to artificial dependencies, the fork-join implementation fails to generate enough subtasks to keep all processors busy and does not have enough data locality to compensate for the lost performance. This phenomena happens when the input size of the DP algorithm is small or we have a huge number of compute cores in the system. As a result, with a fixed computation resource, moving from small in

关键词： Dynamic Programming Recursive Divide-and-Conquer Data-Flow parallel Model Fork-Join parallel Model Concurrent Collections OpenMP

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：