检索结果-内蒙古大学图书馆

An Optimized Straggler Mitigation Framework for Large-Scale distributed computing Systems

IEEE ACCESS 2022年 10卷 97075-97088页

作者： Said, Samar A. Habashy, Shahira M. Salem, Sameh A. Saad, Elsayed M. Helwan Univ Fac Engn Dept Comp & Syst Engn Cairo 11792 Egypt Natl Telecom Regulatory Author NTRA Egyptian Comp Emergency Readiness Team EG CERT Cairo 12577 Egypt

Nowadays, Big Data becomes a research focus in industrial, banking, social network, and other fields. In addition, the explosive increase of data and information require efficient processing solutions. Therefore, Spark is considered as a promising candidate of Large-Scale distributed computing Systems for big data processing. One primary challenge is the straggler problem that occurred due to the presence of heterogeneity where a machine takes an extra-long time to finish execution of a task, which decreases the system throughput. To mitigate straggler tasks, Spark adopts speculative execution mechanism, in which the scheduler launches additional backup to avoid slow task processing and achieve acceleration. In this paper, a new Optimized Straggler Mitigation Framework is proposed. The proposed framework uses a dynamic criterion to determine the closest straggler tasks. This criterion is based on multiple coefficients to achieve a reliable straggler decision. Also, it integrates the historical data analysis and online adaptation for intelligent straggler judgment. This guarantees the effectiveness of speculative tasks by improving cluster performance. Experimental results on various benchmarks and applications show that the proposed framework achieves 23.5% to 30.7% execution time reductions, and 25.4 to 46.3% increase of the cluster throughputs compared with spark engine.

关键词： Clustering algorithms Large-scale systems Throughput Social networking (online) distributed computing Heuristic algorithms Cloning Big Data Data analysis Spark straggler speculative execution cluster throughput

来源：评论

学校读者我要写书评

暂无评论

DistDLB: Improving cosmology SAMR simulations on distributed computing systems through hierarchical load balancing

引用

JOURNAL OF PARALLEL AND distributed computing 2006年第5期66卷 716-731页

作者： Lan, ZL Taylor, VE Li, YW IIT Chicago IL 60616 USA Texas A&M Univ College Stn TX 77843 USA

Cosmology SAMR simulations have played a prominent role in the field of astrophysics. The emerging distributed computing systems provide an economic alternative to the traditional parallel machines, and enable scientists to conduct cosmological simulations that require vast computing power. An important issue of conducting distributed cosmological simulations is about performance and efficiency. In this paper, we present a dynamic load balancing scheme called DistDLB that is designed to improve the performance of distributed cosmology simulations Computation Grid, usually consist of heterogeneous resources connected with shared networks. By considering distributed systems. e.g. the these features of distributed systems and unique characteristics of cosmology SAMR simulations, DistDLB focuses on reducing the redistribution cost through a hierarchical load balancing approach and a run-time decision making mechanism. Heuristic methods have been proposed to adaptively adjust load balancing strategies based on the observation of the current system and application state. Our experiments with real-world cosmology simulations on production systems indicate that the proposed DistDLB scheme can effectively improve the performance of cosmology simulations by 2.56-79.14% as compared to the scheme that does not consider the heterogeneous and dynamic features of distributed systems. (c) 2006 Elsevier Inc. All rights reserved.

关键词： load balancing performance structured adaptive mesh refinement cosmology grid computing distributed computing

来源：评论

学校读者我要写书评

暂无评论

Coded distributed computing Over Unreliable and Insecure D2D Networks

引用

IEEE TRANSACTIONS ON COMMUNICATIONS 2024年第5期72卷 2563-2576页

作者： Chen, Jiajun Sung, Chi Wan City Univ Hong Kong Dept Elect Engn Hong Kong Peoples R China

With the development of wireless computing devices, extending distributed computing to wireless networks deserves a closer look. This paper considers distributed computing over unreliable and insecure device-to-device (D2D) networks, in which each device is not always available to perform computation. The process of distributed devices exchanging calculated results with each other is vulnerable to eavesdropping in wireless environments. To handle the unreliable devices, we adopt repetition codes to build a novel system that supports general computations, called rho-replication system, where each device has rho -1 replicas with duplicate data. A coded computation scheme for the $\rho $ -replication system is proposed, which not only achieves the minimum communication load of the system but also ensures weak security of wireless transmissions during data exchange. Furthermore, the replication nature of the system can be exploited for beamforming transmissions, naturally leading to the idea of energy optimization. Simulation results show that increasing rho does not necessarily improve energy efficiency, as the benefit of increased beamforming gain may be outweighed by the drawback of heavier communication load.

关键词： distributed computing Wireless communication Array signal processing Task analysis Security Communication system security Encoding Coded distributed computing D2D networks computation replication weak security computation-communication tradeoff energy efficiency

来源：评论

学校读者我要写书评

暂无评论

A parking space allocation algorithm based on distributed computing

引用

INTERNATIONAL JOURNAL OF SENSOR NETWORKS 2020年第4期33卷 250-258页

作者： Chen, Guanlin Pang, Huajian Xu, Huang Yang, Wujian Chen, Yong Zhejiang Univ City Coll Sch Comp & Comp Sci Hangzhou 310015 Peoples R China Changzhou Univ Sch Informat Sci & Engn Changzhou 213164 Jiangsu Peoples R China Zhejiang Univ City Coll Sch Business Hangzhou 310015 Peoples R China

In order to make it easier for drivers to find a parking slot, optimise the resources of urban parking slots, and alleviate the problem of parking slot shortage, a distributed parking allocation algorithm was proposed. The algorithm collects the parking requests of user, this parking requests including the current position coordinate information of the users and destination coordinate information, the algorithm allocates parking spaces to users by analysing the available state of parking spaces, then return the parking route planning to the client. Compared with the traditional algorithm, the distributed parking algorithm has a higher ability to withstand pressure and global search capability, and it can ensure the real-time and validity of the parking information, so it can reduce the problem of the parking space shortage and unavailable parking space. The simulation results show that this algorithm can find the solution set more quickly and accurately under the circumstance of high demand. It also has application value and practicality.

关键词： city traffic parking allocation distributed computing matching algorithm smart city genetic algorithm parking slot urban management parking requests greedy algorithm

来源：评论

学校读者我要写书评

暂无评论

Asymptotically Optimal Coded distributed computing via Combinatorial Designs

引用

IEEE-ACM TRANSACTIONS ON NETWORKING 2024年第4期32卷 3018-3033页

作者： Cheng, Minquan Wu, Youlong Li, Xianxian Wu, Dianhua Guangxi Normal Univ Minist Educ Key Lab Educ Blockchain & Intelligent Technol Guilin 541004 Peoples R China Guangxi Normal Univ Guangxi Key Lab Multisource Informat Miningand Sec Guilin 541004 Peoples R China ShanghaiTech Univ Sch Informat Sci & Technol Shanghai 201210 Peoples R China Guangxi Normal Univ Sch Math & Stat Guilin 541006 Peoples R China

Coded distributed computing (CDC) introduced by Li et al. can greatly reduce the communication load for MapReduce computing systems. In the cascaded CDC with K workers, N input files and Q output functions, each input file will be mapped by r workers and each output function will be computed by s workers such that coding techniques can be applied to create multicast opportunities. The main drawback of most existing CDC schemes is that they require the original data to be split into a large number of input files that grows exponentially with K , which would significantly increase the coding complexity and degrade the system performance. In this paper, we first use a classical combinatorial structure t -design, for any integer t >= 2 >= 2 , to develop a low-complexity and communication-efficient CDC with r=s. Our scheme has much smaller N and Q than the existing schemes under the same parameters K , r , and s;and achieves smaller communication loads compared with the state-of-the-art schemes when K is relatively large. Remarkably, unlike the previous schemes that realize on large operation fields, our scheme operates in one-shot communication on the minimum binary field F-2 . With a derived lower bound on the communication load under one-shot linear delivery, we show that the t -design scheme is asymptotically optimal. Furthermore, we show that our construction method can incorporate the other combinatorial structures that have a similar property to t -design. For instance, we use t -GDD to obtain another one-shot asymptotically optimal CDC scheme over F-2 that has different parameters from t -design. Finally, we show that our construction method can also be used to construct CDC schemes with r not equal s that have small file number and output function number. Like what you're reading?

关键词： distributed computing Costs Task analysis Encoding Vectors Technological innovation Symbols Coded distributed computing asymptotically optimal t-design t-GDD

来源：评论

学校读者我要写书评

暂无评论

Handoff-Aware distributed computing in High Altitude Platform Station (HAPS)--Assisted Vehicular Networks

引用

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS 2023年第12期22卷 8814-8827页

作者： Ren, Qiqi Abbasi, Omid Kurt, Gunes Karabulut Yanikomeroglu, Halim Chen, Jian Xidian Univ State Key Lab Integrated Serv Networks Xian 710071 Shaanxi Peoples R China Carleton Univ Dept Syst & Comp Engn Nonterr Networks NTN Lab Ottawa ON K1S 5B6 Canada Polytech Montreal Dept Elect Engn Montreal PQ H3T 1J4 Canada

distributed computing enables Internet of vehicle (IoV) services by collaboratively utilizing the computing resources from the network edge and the vehicles. However, the computing interruption issue caused by frequent edge network handoffs, and a severe shortage of computing resources are two problems in providing IoV services. High altitude platform station (HAPS) computing can be a promising addition to existing distributed computing frameworks because of its wide coverage and strong computational capabilities. In this regard, this paper proposes an adaptive scheme in a new distributed computing framework that involves HAPS computing to deal with the two problems of the IoV. Based on the diverse demands of vehicles, network dynamics, and the time-sensitivity of handoffs, the proposed scheme flexibly divides each task into three parts and assigns them to the vehicle, roadside units (RSUs), and a HAPS to perform synchronous computing. The scheme also constrains the computing of tasks at RSUs such that they are completed before handoffs to avoid the risk of computing interruptions. On this basis, we formulate a delay minimization problem that considers task-splitting ratio, transmit power, bandwidth allocation, and computing resource allocation. To solve the problem, variable replacement and successive convex approximation-based method are proposed. The simulation results show that this scheme not only avoids the negative effects caused by handoffs in a flexible manner, it also takes delay performance into account and maintains the delay stability.

关键词： Task analysis Servers Delays Computational modeling Wireless communication distributed databases Autonomous aerial vehicles Internet of Vehicles (IoV) high altitude platform station (HAPS) distributed computing edge network handoff

来源：评论

学校读者我要写书评

暂无评论

Incentive-Based Coded distributed computing Management for Latency Reduction in IoT Services-A Game Theoretic Approach

引用

IEEE INTERNET OF THINGS JOURNAL 2021年第10期8卷 8259-8278页

作者： Kim, Nakyoung Kim, Daejin Lee, Joohyung Niyato, Dusit Choi, Jun Kyun Korea Adv Inst Sci & Technol Sch Elect Engn Daejeon 34141 South Korea Samsung Elect Dept Network Business Suwon 16677 South Korea Gachon Univ Sch Comp Seongnam 13120 South Korea Nanyang Technol Univ Sch Comp Sci & Engn Singapore 639798 Singapore

This article studies distributed computing (DC) mechanisms on heterogeneous mobile devices (MDs) for latency reduction (LR) in Internet-of-Things (IoT) services by mitigating the effect of straggling MDs. We propose novel coded DC mechanisms with two different incentive distributions that consider the time-discounting value of processed results and the amount of the workload computed by MDs. Specifically, we consider distributed gradient descent computing with coding when a task publisher (TP) with a limited amount of budget offers incentives to encourage MDs' participation in the computation. To analyze a hierarchical decision-making structure of the TP and MDs, we formulate a strategic competition between them as a Stackelberg game. In the case that the MDs are the leaders, we design a CPU-cycle frequency control scheme to balance each MD's computing speed and energy consumption for obtaining its maximum utility with the incentive mechanisms. As the follower, the TP aims at minimizing latency of the DC, and it follows the MDs' decisions to determine the load allocation for each MD. Then, we design an algorithm achieving the Stackelberg equilibrium, which is shown to be a unique Nash equilibrium of the game. The performance evaluation results show that the proposed mechanisms achieve 39% of LR on average compared to the benchmark mechanism. Furthermore, the results corroborate the efficiency of the proposed mechanisms in terms of the MDs' social welfare.

关键词： distributed computing Encoding Task analysis Internet of Things Games Resource management distributed databases Coded distributed computing (DC) incentive mechanism load management Stackelberg game

来源：评论

学校读者我要写书评

暂无评论

The evolution of distributed computing systems: from fundamental to new frontiers

引用

computing 2021年第8期103卷 1859-1878页

作者： Lindsay, Dominic Gill, Sukhpal Singh Smirnova, Daria Garraghan, Peter Univ Lancaster Sch Comp & Commun Lancaster England Queen Mary Univ London Sch Elect Engn & Comp Sci London England

distributed systems have been an active field of research for over 60 years, and has played a crucial role in computer science, enabling the invention of the Internet that underpins all facets of modern life. Through technological advancements and their changing role in society, distributed systems have undergone a perpetual evolution, with each change resulting in the formation of a new paradigm. Each new distributed system paradigm-of which modern prominence include cloud computing, Fog computing, and the Internet of Things (IoT)-allows for new forms of commercial and artistic value, yet also ushers in new research challenges that must be addressed in order to realize and enhance their operation. However, it is necessary to precisely identify what factors drive the formation and growth of a paradigm, and how unique are the research challenges within modern distributed systems in comparison to prior generations of systems. The objective of this work is to study and evaluate the key factors that have influenced and driven the evolution of distributed system paradigms, from early mainframes, inception of the global inter-network, and to present contemporary systems such as edge computing, Fog computing and IoT. Our analysis highlights assumptions that have driven distributed systems appear to be changing, including (1) an accelerated fragmentation of paradigms driven by commercial interests and physical limitations imposed by the end of Moore's law, (2) a transition away from generalized architectures and frameworks towards increasing specialization, and (3) each paradigm architecture results in some form of pivoting between centralization and decentralization coordination. Finally, we discuss present day and future challenges of distributed research pertaining to studying complex phenomena at scale and the role of distributed systems research in the context of climate change.

关键词： distributed computing computing systems Evolution Green computing

来源：评论

学校读者我要写书评

暂无评论

Sunburn-using excess energy of small-scale production for distributed computing

引用

ENERGY EFFICIENCY 2018年第1期11卷 97-119页

作者： Nurminen, J. K. Niemi, T. Strandman, J. Ruokosuo, K. Helsinki Inst Phys Helsinki Finland Aalto Univ Helsinki Finland CERN Helsinki Inst Phys Meyrin Switzerland

The Sunburn solar computer system is based on the idea of consuming the excess electricity of photovoltaic energy systems for useful computing. In this way, the proposed approach allows us to move a part of computing capacity from dedicated data centers to individual solar energy systems and thereby reduce the energy consumption of the data centers. In principle, the proposed mechanism converts the excess electrical energy into valuable computation. The results of computation or data processing are easier to store and transfer than produced electricity. In small-scale systems, the benefit from storing or selling the excess electricity is small, whereas the benefit from selling the generated data is potentially larger, as we demonstrate in this study. Finally, the technical feasibility of the solution is illustrated by constructing and evaluating a prototype implementation using excess solar energy for distributed BOINC computing.

关键词： distributed computing Solar energy Green computing Energy storage Scientific computing distributed data center

来源：评论

学校读者我要写书评

暂无评论

DESIGN OF ELECTRICAL MACHINES BY THE FINITE-ELEMENT METHOD USING distributed computing

引用

COMPUTERS IN INDUSTRY 1991年第4期17卷 367-374页

作者： CHAN, CC CHAU, KT UNIV HONG KONG DEPT ELECT & ELECTR ENGNHONG KONGHONG KONG HONG KONG POLYTECH DEPT ELECT ENGNHONG KONGHONG KONG

Electrical machines play an essential part in the lives of all of us: in power generation, in transport, in industry and in our homes. They have been in a recognizable form for about a century. Yet the advances in new materials, computation methods and computer facilities have contributed to an explosion of new electrical machines. To keep up with the more stringent design requirements of industry, the design of electrical machines turns to computer aided design. In this paper, a visual and interactive approach for electrical machine design is described which incorporates both electromagnetic calculation and visual evaluation. Moreover, a distributed computing environment for the design of electrical machines is proposed and implemented. Applications of the proposed approach for synchronous generators, induction motors and permanent magnet motors are described.

关键词： FINITE ELEMENT METHOD ELECTRICAL MACHINE DESIGN distributed computing COMPUTER GRAPHICS

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：