检索结果-内蒙古大学图书馆

OffStreamNG: Partial Stream Hybrid Graph Edge Partitioning Based on Neighborhood Expansion and Greedy Heuristic 1

24th East-European Conference on Advances in Databases and Information systems/24th International Conference on Theory and Practice of Digital Libraries/16th workshop on Business Intelligence and Big Data (ADBIS/TPDL/EDA)

作者： Ayalew, Tewodros Duan, Hancong Liu, Changhong Gereme, Fantahun Delele, Mesay Univ Elect Sci & Technol China Sch Comp Sci & Engn Chengdu Peoples R China Univ Elect Sci & Technol China Inst Fundamental & Frontier Sci Chengdu Peoples R China Univ Elect Sci & Technol China Sch Informat Sci & Engn Chengdu Peoples R China

ISBN: (数字)9783030546236

ISBN: (纸本)9783030546229;9783030546236

Recently, graph edge partitioning has shown better partitioning quality than the vertex graph partitioning for the skewed degree distribution of real-world graph data. Graph edge partitioning can be classified as stream and offline. The stream edge partitioning approach supports a big graph partitioning;however, it has lower partitioning quality, is affected by stream order, and it has taken much time to make partitioning compared with the offline edge partitioning. Conversely, the offline edge partitioning approach has better partitioning quality than stream edge partitioning;however, it does not support big graph partitioning. In this study, we propose partial stream hybrid graph edge partitioning OffStreamNG, which leverages the advantage of both offline and stream edge partitioning approaches by interconnecting via saved partition state layer. The OffStreamNG holds vertex and load states as partition state, while the offline component is partitioning using neighborhood expansion heuristic. And it is transferring this partition state to the online component of Greedy heuristic with minor modification of both algorithms. Experimental results show that OffStreamNG achieves attractive results in terms of replication factor, load balance, and total partitioning time.

关键词： Edge partitioning Stream approach Offline approach distributed graph computing Hybrid edge partitioning Saved partition state

来源：评论

学校读者我要写书评

暂无评论

CORE - An optimal data placement strategy in Hadoop for data intensive applications based on cohesion relation

引用

COMPUTER systems SCIENCE AND ENGINEERING 2019年第1期34卷 47-60页

作者： Vengadeswaran Balasundaram Natl Inst Technol Dept Comp Applicat Tiruchirappalli 620015 Tamil Nadu India

The tremendous growth of data being generated today is making storage and computing a mammoth task. With its distributed processing capability Hadoop gives an efficient solution for such large data. Hadoop's default data placement strategy places the data blocks randomly across the nodes without considering the execution parameters resulting in several lacunas such as increased execution time, query latency etc., Also, most of the data required for a task execution may not be locally available which creates data-locality problem. Hence we propose an innovative data placement strategy based on dependency of data blocks across the nodes. Our strategy dynamically analyses the history log and establishes relationship between various tasks and blocks required for each task through Block Dependency Graph (BDG). Then Our CORE-Algorithm re-organizes the HDFS layout by redistributing the data blocks to give an optimal data placement, resulting in improved performance for Big Data sets in distributed environment. This strategy is tested in 20-node cluster with different real-world MR applications. The results conclude that proposed strategy reduces the query execution time by 23%, improves the data locality by 50.7%, compared to default.

关键词： Big Data distributed Storage parallel Processing Hadoop Data Placement Cohesion

来源：评论

学校读者我要写书评

暂无评论

real-time Deployment and Resource Allocation for distributed UAV systems in Disaster Relief 20

Real-Time Deployment and Resource Allocation for Distributed...

引用

20th IEEE International workshop on Signal Processing Advances in Wireless Communications (SPAWC)

作者： Nguyen, Long D. Nguyen, Khoi K. Kortun, Ayse Duong, Trung Q. Duy Tan Univ Da Nang Vietnam Queens Univ Belfast Belfast Antrim North Ireland

ISBN: (纸本)9781538665282

In this work, we propose a robust and efficient resource allocation scheme for UAV-enabled cellular networks that aid in disaster communications. To recover the network within a disaster area, a fast user clustering model based on K-means procedure and distributed control power coefficient will be proposed and can be embedded in the real system by using UAV-assisted relaying for real-time recovering and maintaining network during and after disasters. Algorithms of low computational complexity and fast convergence are also proposed. Numerical examples are provided to demonstrate the benefit of the proposed computational approach.

关键词： aircraft communication autonomous aerial vehicles cellular radio computational complexity convergence disasters distributed control pattern clustering relay networks (telecommunication) resource allocation telecommunication control Resource utilization AIRCRAFT COMMUNICATION Communication and control decentralised control cellular radio complexity classes Disaster relief Cluster Analysis management of resources Disasters Converge disaster area

来源：评论

学校读者我要写书评

暂无评论

分布式流处理系统的时间感知分组算法

引用

清华大学学报（自然科学版） 2020年第10期60卷 822-828页

作者：杨李杨陈思远张展朱和一左德承哈尔滨工业大学计算机科学与技术学院哈尔滨150001

随着对实时数据流处理需求的增加,分布式流处理系统的发展也越来越受到关注。大量的倾斜的数据流以及复杂分布式系统的异构性对当前的分布式流处理系统的分组策略提出了挑战。目前已有的分布式流处理分组策略通常关注并行实例之间元组... 详细信息

随着对实时数据流处理需求的增加,分布式流处理系统的发展也越来越受到关注。大量的倾斜的数据流以及复杂分布式系统的异构性对当前的分布式流处理系统的分组策略提出了挑战。目前已有的分布式流处理分组策略通常关注并行实例之间元组数量的均衡性,而忽视了系统异构性对分组策略造成的影响。该文提出了一种时间感知分组算法,通过对分布式流处理系统存在的网络异构性和处理能力异构性的分析,综合考虑流处理系统中各下游算子实例的处理时间以及上游算子与下游算子之间的通信时间,并根据键值的频率不同制定不同的路由策略,在较小的开销下使系统达到负载均衡。在Apache Flink分布式流处理系统上进行的实验结果表明:时间感知分组算法比已有的分组算法在系统吞吐量上提高了10%,在平均处理延迟上降低了33%。

关键词：流处理分组算法时间感知

来源：评论

学校读者我要写书评

暂无评论

Dynamic Stale Synchronous parallel distributed Training for Deep Learning 39

Dynamic Stale Synchronous Parallel Distributed Training for ...

引用

39th IEEE International Conference on distributed Computing systems (ICDCS)

作者： Zhao, Xing An, Aijun Liu, Junfeng Chen, Bao Xin York Univ Dept Elect Engn & Comp Sci Toronto ON Canada IBM Canada Platform Comp Markham ON Canada

ISBN: (纸本)9781728125190

Deep learning is a popular machine learning technique and has been applied to many real-world problems, ranging from computer vision to natural language processing. However, training a deep neural network is very time-consuming, especially on big data. It has become difficult for a single machine to train a large model over large datasets. A popular solution is to distribute and parallelize the training process across multiple machines using the parameter server framework. In this paper, we present a distributed paradigm on the parameter server framework called Dynamic Stale Synchronous parallel (DSSP) which improves the state-of-the-art Stale Synchronous parallel (SSP) paradigm by dynamically determining the staleness threshold at the run time. Conventionally to run distributed training in SSP, the user needs to specify a particular stalenes threshold as a hyper-parameter. However, a user does not usually know how to set the threshold and thus often finds a threshold value through trial and error, which is time-consuming. Based on workers' recent processing time, our approach DSSP adaptively adjusts the threshold per iteration at running time to reduce the waiting time of faster workers for synchronization of the globally shared parameters (the weights of the model), and consequently increases the frequency of parameters updates (increases iteration throughput), which speedups the convergence rate. We compare DSSP with other paradigms such as Bulk Synchronous parallel (BSP), Asynchronous parallel (ASP), and SSP by running deep neural networks (DNN) models over GPU clusters in both homogeneous and heterogeneous environments. The results show that in a heterogeneous environment where the cluster consists of mixed models of GPUs, DSSP converges to a higher accuracy much earlier than SSP and BSP and performs similarly to ASP. In a homogeneous distributed cluster, DSSP has more stable and slightly better performance than SSP and ASP, and converges much faster than BSP.

关键词： distributed deep learning parameter server BSP ASP SSP GPU cluster

来源：评论

学校读者我要写书评

暂无评论

A New Julia-Based parallel time-Domain Simulation Algorithm for Analysis of Power System Dynamics

A New Julia-Based Parallel Time-Domain Simulation Algorithm ...

引用

International workshop/Symposium on distributed Simulation and real-time Applications (DS-RT)

作者： Michael Kyesswa Philipp Schmurr Hüseyin K. Çakmak Uwe Kühnapfel Veit Hagenmeyer Institute for Automation and Applied Informatics Karlsruhe Institute of Technology Karlsruhe Germany

ISBN: (数字)9781728173436

ISBN: (纸本)9781728173443

The present paper describes a new parallel time-domain simulation algorithm using a high performance computing environment - Julia - for the analysis of power system dynamics in large networks. The parallel algorithm adapts a parallel-in-space decomposition scheme to a previously sequential algorithm in order to develop a new parallelizable numerical solution of the power system equations. The parallel-in-space decomposition is based on the block bordered diagonal form, which reformulates the network admittance matrix into sub-blocks that can be solved in parallel. For the optimal spatial decomposition of the network, a new extended graph partitioning strategy is developed for load balancing and minimizing the communication between subnetworks. The new parallel simulation algorithm is tested using standard test networks of varying complexity. The simulation results are compared to those obtained from a sequential implementation in order to validate the solution accuracy and to determine the performance improvement in terms of computational speedup. Test simulations are conducted using the ForHLR II supercomputing cluster and show a huge potential in computational speedup with increasing network complexity.

关键词： Mathematical model Computational modeling Power system stability Power system dynamics Heuristic algorithms Partitioning algorithms Analytical models

来源：评论

学校读者我要写书评

暂无评论

Predictive Control of parallel Three-Phase Grid-tied Inverters: A Simple and Effective Solution to Mitigate Circulating Current

Predictive Control of Parallel Three-Phase Grid-tied Inverte...

引用

workshop on Predictive Control of Electrical Drives and Power Electronics (PRECEDE)

作者： Yu Li Han He Xiaodong Liu Zhenbin Zhang Shuo Shi Jose Rodriguez

ISBN: (纸本)9781665425582

High power grid-tied inverters have attracted increasing attentions in energy conversion and electrical drives applications. parallel configuration is a promising solution to enlarge the total power rating. However, zero-sequence loop arises in paralleled grid-tied inverters, hence introducing zero-sequence circulating current. This leads to the deceases of the effective power volume and additional power losses. An effective solution based on predictive control framework is proposed in this work, and it successfully eliminating the circulating current. The proposed method decomposes the voltage vector into two parts, i.e., the former is used for reference current tracking, and the latter is responsible to suppress the circulating current. It is straightforward to be implemented in low-cost processors, and flexible to balance the importance of control performance and bus voltage utilization. We validate the controller on a lab-constructed test bench. The experimental results confirm the effectiveness and robustness of our proposed solution.

关键词： Electric potential Program processors Energy conversion Predictive models Inverters Robustness real-time systems

来源：评论

学校读者我要写书评

暂无评论

Optimal power allocation and load balancing for non-dedicated heterogeneous distributed embedded computing systems

引用

JOURNAL OF parallel AND distributed COMPUTING 2019年 130卷 24-36页

作者： Huang, Jing Liu, Yan Li, Renfa Li, Keqin An, JiYao Bai, Yang Yang, Fan Xie, Guoqi Hunan Univ Coll Comp Sci & Elect Engn Changsha 410082 Hunan Peoples R China Hunan Univ Key Lab Embedded & Network Comp Hunan Prov Changsha 410082 Hunan Peoples R China SUNY Coll New Paltz Dept Comp Sci New Paltz NY 12561 USA Cent South Univ Forestry & Technol Coll Comp & Informat Engn Changsha Hunan Peoples R China

This paper investigates on the optimal power allocation and load balancing problem encountered by heterogeneous and distributed embedded systems with mixed tasks. Given that each node has real and different urgent tasks in the majority of practical heterogeneous embedded systems, three priority disciplines are considered: dedicated jobs without priority, prioritized dedicated jobs without preemption, and prioritized dedicated jobs with preemption. A model is established for heterogeneous embedded processors with dedicated-task-dependent dynamic power and load balancing management;each processor is considered as an M/M/1 queueing sub-model with mixed generic and dedicated tasks. The processors have different levels of power consumption, and each one can employ any of the three disciplines. The objective of this study is to find an optimal load balancing (for generic tasks) and power allocation strategy for heterogeneous processors preloaded by different amounts of dedicated tasks such that the average response time of generic tasks is minimized. Considering that this problem is a multi-constrained, multi-variable optimization problem for which a closed-form solution is unlikely to be obtained, we propose an optimal power allocation and load balancing scheme by employing Lagrange method and binary search approach, which are completed by utilizing two new rules established by observing numerical variations of parameters. Several numerical examples are presented to demonstrate the effectiveness of our solution. To the best of our knowledge, this is the first work on analytical study that combines load balancing, energy efficiency, and priority of tasks in heterogeneous and distributed embedded systems. (C) 2019 Elsevier Inc. All rights reserved.

关键词： Embedded and distributed system Load distribution Power allocation Queueing model Response time

来源：评论

学校读者我要写书评

暂无评论

FPO tree and DP3 algorithm for distributed parallel Frequent Itemsets Mining

引用

EXPERT systems WITH APPLICATIONS 2020年 140卷 112874-112874页

作者： Van Quoc Phuong Huynh Kueng, Josef Johannes Kepler Univ Linz Inst Applicat Oriented Knowledge Proc FAW Sci Pk 3Altenberger Str 66b A-4040 Linz Austria

Frequent Itemsets Mining is a fundamental mining model in Data Mining. It supports a vast range of application fields and can be employed as a key calculation phase in many other mining models such as Association Rules, Correlations, Classifications, etc. Many distributed parallel algorithms have been introduced to confront with very large-scale datasets of Big Data. However, the problems of running time and memory scalability still have not had adequate solutions for very large and "hard-to-mined" datasets. In this paper, we propose a distributed parallel algorithm named DP3 (distributed PrePostPlus) which parallelizes the state-of-the-art algorithm PrePost(+) and operates in Master-Slaves model. Slave machines mine and send local frequent itemsets and support counts to the Master for aggregations. In the case of tremendous numbers of itemsets transferred between the Slaves and Master, the computational load at the Master, therefore, is extremely heavy if there is not the support from our complete FPO tree (Frequent Patterns Organization) which can provide optimal compactness for light data transfers and highly efficient aggregations with pruning ability. Processing phases of the Slaves and Master are designed for memory scalability and shared-memory parallel in Work-Pool model so as to utilize the computational power of multi-core CPUs. We conducted experiments on both synthetic and real datasets, and the empirical results have shown that our algorithm far outperforms the well-known PFP and other three recently high-performance ones Dist-Eclat, BigFIM, and MapFIM. (C) 2019 Elsevier Ltd. All rights reserved.

关键词： Frequent Itemsets Mining parallel distributed Data Mining Big Data Prefix tree

来源：评论

学校读者我要写书评

暂无评论

带相关噪声多传感器系统的事件触发贯序和分布式融合估计

引用

空军工程大学学报（自然科学版） 2021年第2期22卷 60-67页

作者：王妮孙书利黑龙江大学电子工程学院哈尔滨150080

对带相关噪声的多传感器系统,研究了事件触发的贯序和分布式融合估计算法。不同传感器之间的观测噪声同时刻相关,并与过程噪声一步相关。为了节省通信能耗,采用了事件触发传输机制,该机制依赖于每个传感器当前的观测值和上一个触发时刻... 详细信息

对带相关噪声的多传感器系统,研究了事件触发的贯序和分布式融合估计算法。不同传感器之间的观测噪声同时刻相关,并与过程噪声一步相关。为了节省通信能耗,采用了事件触发传输机制,该机制依赖于每个传感器当前的观测值和上一个触发时刻的观测值。在事件触发条件下,提出了在线性最小方差意义上的最优贯序融合和分布式融合估计算法。所提出的贯序融合算法可以根据传感器观测数据到达滤波器的顺序进行实时处理,具有较小的计算负担。所提出的分布式融合算法可以对传感器观测数据进行并行处理,具有更好的可靠性。两种算法与事件触发集中融合算法具有相同的估计精度。仿真结果验证了算法的有效性。

关键词：事件触发机制贯序融合分布式融合多传感器系统噪声相关

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：