检索结果-内蒙古大学图书馆

New Results on the Computation-Communication Tradeoff for Heterogeneous Coded distributed computing

IEEE TRANSACTIONS ON COMMUNICATIONS 2021年第4期69卷 2254-2270页

作者： Xu, Fan Shao, Shuo Tao, Meixia Shanghai Jiao Tong Univ Dept Elect Engn Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai 200240 Peoples R China

Coded distributed computing (CDC) can alleviate the communication load in distributed computing systems by leveraging coding opportunities via redundant computation. While the optimal computation-communication tradeoff has been well studied for homogeneous systems, it remains largely unknown for heterogeneous systems where workers have different computation capabilities. This paper characterizes the upper and lower bounds of the optimal communication load as two linear programming problems for a general heterogeneous CDC system using the MapReduce framework. Our achievable scheme first designs a parametric data shuffling strategy for any given mapping strategy, and then jointly optimizes the mapping strategy and the data shuffling strategy to obtain the upper bound. The parametric data shuffling strategy allows adjusting the size of the multicast message intended for each worker set, so that it can largely decrease the number of unicast messages and hence increase the communication efficiency. Numerical results show that our achievable communication load is lower than those achieved in existing works. Our lower bound is established by unifying an improved cut-set bound and a peeling method. The obtained upper and lower bounds degenerate to the existing result in homogeneous systems, and coincide with each other when the system is approximately homogeneous or grouped homogeneous.

关键词： distributed computing Multicast communication Linear programming Upper bound Optimization Wireless communication Unicast Coded distributed computing heterogeneous system computation-communication tradeoff

来源：评论

学校读者我要写书评

暂无评论

FLCD: A Flexible Low Complexity Design of Coded distributed computing

引用

IEEE TRANSACTIONS ON CLOUD computing 2023年第1期11卷 470-483页

作者： Woolsey, Nicholas Wang, Xingyue Chen, Rong-Rong Ji, Mingyue Univ Utah Dept Elect Engn Salt Lake City UT 84112 USA

We propose a flexible low complexity design (FLCD) of coded distributed computing (CDC) with empirical evaluation on Amazon Elastic Compute Cloud (Amazon EC2). CDC can expedite MapReduce like computation by trading increased map computations to reduce communication load and shuffle time. A main novelty of FLCD is to utilize the design freedom in defining map and reduce functions to develop asymptotic homogeneous systems to support varying intermediate values (IV) sizes under a general MapReduce framework. Compared to existing designs with constant IV sizes, FLCD offers greater flexibility in adapting to network parameters and significantly reduces the implementation complexity by requiring fewer input files and shuffle groups. The FLCD scheme is the first proposed low-complexity CDC design that can operate on a network with an arbitrary number of nodes and computation load. We perform empirical evaluations of the FLCD by executing the TeraSort algorithm on an Amazon EC2 cluster. This is the first time that theoretical predictions of the CDC shuffle time are validated by empirical evaluations. The evaluations demonstrate a 2.0 to 4.24x speedup compared to conventional uncoded MapReduce, a 12 to 52 percent reduction in total time, and a wider range of operating network parameters compared to existing CDC schemes.

关键词： Complexity theory Computational modeling Cloud computing distributed computing Indexes Benchmark testing Urban areas Coded distributed computing communication load computation load coded multicasting heterogeneity low-complexity implementation Amazon EC2

来源：评论

学校读者我要写书评

暂无评论

Optimizing distributed computing systems for k-nearest neighbours classifiers-evolutionary approach

引用

LOGIC JOURNAL OF THE IGPL 2011年第2期19卷 357-372页

作者： Przewozniczek, Michal Walkowiak, Krzysztof Wozniak, Michal Wroclaw Univ Technol PL-50370 Wroclaw Poland

Since the amount of information is rapidly growing, there is an overwhelming interest in efficient network computing. In this article, we take a detailed look at the problem of modelling and optimization of aforementioned systems for k-nearest neighbour classifier. First, we present a comprehensive discussion on considered classification methods with a special focus on improving classification accuracy or response time through the use of partitions of original data set for the nearest neighbour rule. Next, we propose a generic optimization model of a network computing system that can be used for distributed implementation of aforementioned recognition methods. The objective is to minimize the response time of the computing system applied for tasks related to k-nearest neighbours classifiers. We solve the problem using traditional branch and cut method and original algorithm GReTiMA based on a genetic approach as well. To illustrate our work, we provide results of numerical experiments showing the performance of the evolutionary approach compared against optimal results. Moreover, we show that the distributed approach enables significant improvement of the system response time.

关键词： k-nearest neighbour classifier pattern recognition distributed computing grid computing genetic algorithm

来源：评论

学校读者我要写书评

暂无评论

Fast Access to Remote Objects 2.0 a renewed gateway to ENEAGRID distributed computing resources

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2019年 94卷 920-928页

作者： Mariano, Angelo D'Amato, Giulio Ambrosino, Fiorenzo Aprea, Giuseppe Buonocore, Francesco Celino, Massimo Colavincenzo, Antonio Fina, Marco Funel, Agostino Giusepponi, Simone Guarnieri, Guido Palombi, Filippo Pierattini, Samuele Ponti, Giovanni Santomauro, Giuseppe Bracco, Giovanni Migliori, Silvio ENEA Apulia Local Off I-70126 Bari Italy Sysman Progetti & Serv Srl I-00143 Rome Italy ENEA Portici Res Ctr I-80055 Portici NA Italy ENEA Frascati Res Ctr I-00123 Rome Italy ENEA Frascati Res Ctr I-00144 Rome Italy ENEA Tuscany Local Off I-50019 Florence Italy Consortium GARR I-00196 Rome Italy ENEA Headquarters I-00196 Rome Italy

This paper introduces a renewed gateway to ENEAGRID distributed computing resources named Fast Access to Remote Objects 2.0 (FARO 2.0). FARO 2.0 is a tool for application and desktop virtualization with a focus towards user experience (UX), providing trained as well as untrained users with a collection of centralized services that can be seamlessly used on their client through a remote desktop protocol. FARO 2.0 is a javaFX application whose graphical user interface (GUI) and whose main logics have been implemented through the well-known Web technologies (HTML5, CSS3, Javascript) for easier maintainability and customizability, taking full advantage of the WebView component. The FARO 2.0 framework has been deployed both as a general purpose GUI for remote user access to ENEAGRID resources and as a specialized application or workflow-oriented GUI. They are applied in a set of applicative domains, ranging from materials science to technologies for energy and industry, environmental modeling, and nuclear fusion. Some examples and results are also presented. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Graphical user interface Remote desktop distributed computing Virtual labs

来源：评论

学校读者我要写书评

暂无评论

Coded distributed computing Over Unreliable and Insecure D2D Networks

引用

IEEE TRANSACTIONS ON COMMUNICATIONS 2024年第5期72卷 2563-2576页

作者： Chen, Jiajun Sung, Chi Wan City Univ Hong Kong Dept Elect Engn Hong Kong Peoples R China

With the development of wireless computing devices, extending distributed computing to wireless networks deserves a closer look. This paper considers distributed computing over unreliable and insecure device-to-device (D2D) networks, in which each device is not always available to perform computation. The process of distributed devices exchanging calculated results with each other is vulnerable to eavesdropping in wireless environments. To handle the unreliable devices, we adopt repetition codes to build a novel system that supports general computations, called rho-replication system, where each device has rho -1 replicas with duplicate data. A coded computation scheme for the $\rho $ -replication system is proposed, which not only achieves the minimum communication load of the system but also ensures weak security of wireless transmissions during data exchange. Furthermore, the replication nature of the system can be exploited for beamforming transmissions, naturally leading to the idea of energy optimization. Simulation results show that increasing rho does not necessarily improve energy efficiency, as the benefit of increased beamforming gain may be outweighed by the drawback of heavier communication load.

关键词： distributed computing Wireless communication Array signal processing Task analysis Security Communication system security Encoding Coded distributed computing D2D networks computation replication weak security computation-communication tradeoff energy efficiency

来源：评论

学校读者我要写书评

暂无评论

MetaLAFFA: a flexible, end-to-end, distributed computing-compatible metagenomic functional annotation pipeline

引用

BMC BIOINFORMATICS 2020年第1期21卷 1-9页

作者： Eng, Alexander Verster, Adrian J. Borenstein, Elhanan Univ Washington Dept Genome Sci Seattle WA 98195 USA Hlth Canada Food Directorate Bur Food Surveillance & Sci Integrat Ottawa ON K1A 0K9 Canada Tel Aviv Univ Blavatnik Sch Comp Sci IL-6997801 Tel Aviv Israel Tel Aviv Univ Sackler Fac Med IL-6997801 Tel Aviv Israel Santa Fe Inst Santa Fe NM 87501 USA

BackgroundMicrobial communities have become an important subject of research across multiple disciplines in recent years. These communities are often examined via shotgun metagenomic sequencing, a technology which can offer unique insights into the genomic content of a microbial community. Functional annotation of shotgun metagenomic data has become an increasingly popular method for identifying the aggregate functional capacities encoded by the community's constituent microbes. Currently available metagenomic functional annotation pipelines, however, suffer from several shortcomings, including limited pipeline customization options, lack of standard raw sequence data pre-processing, and insufficient capabilities for integration with distributed computing *** we introduce MetaLAFFA, a functional annotation pipeline designed to take unfiltered shotgun metagenomic data as input and generate functional profiles. MetaLAFFA is implemented as a Snakemake pipeline, which enables convenient integration with distributed computing clusters, allowing users to take full advantage of available computing resources. Default pipeline settings allow new users to run MetaLAFFA according to common practices while a Python module-based configuration system provides advanced users with a flexible interface for pipeline customization. MetaLAFFA also generates summary statistics for each step in the pipeline so that users can better understand pre-processing and annotation *** is a new end-to-end metagenomic functional annotation pipeline with distributed computing compatibility and flexible customization options. MetaLAFFA source code is available at https://***/borenstein-lab/MetaLAFFA and can be installed via Conda as described in the accompanying documentation.

关键词： Metagenomics Functional annotation Pipeline distributed computing

来源：评论

学校读者我要写书评

暂无评论

Bargaining Game Based Offloading Service Algorithm for Edge-Assisted distributed computing Model

引用

IEEE ACCESS 2022年 10卷 63648-63657页

作者： Kim, Sungwook Sogang Univ Dept Comp Sci Seoul 04107 South Korea

Computation offloading is a new paradigm to provide cloud computing capabilities at the edge of pervasive radio access networks in close proximity to mobile users. In this paper, we propose a new computation offloading model for the 5G networks and beyond. Based on the edge computing platform, intensive computing tasks can be partially offloaded from local devices to edge clouds to supplement the computation capability of resource-limited devices. This approach leverages the edge server's idle computing power to assist individual devices in model training. To implement control decision algorithms for the distributed computing process, we adopt the concepts of different bargaining solutions for the dynamic offloading services. According to the cooperative game theory, the proposed method can maximize the full synergy that gives mutual advantages for devices and edge clouds while improving the system efficiency. Therefore, we can take various benefits to reach a fair-efficient consensus under the edge-assisted distributed computing system environment. Finally, experimental results demonstrate the effectiveness of our bargaining based computation offloading scheme by comparing with the existing state-of-the-art distributed computing protocols;we can accelerate training process thanks to our efficient bargaining approach.

关键词： Internet of Things Cloud computing Games Task analysis Edge computing Computational modeling Game theory distributed computing edge offloading computing cooperative game theory loss aversion bargaining solutions utilitarian bargaining solution

来源：评论

学校读者我要写书评

暂无评论

distributed Mobile computing for Deep Learning Applications

Distributed Mobile Computing for Deep Learning Applications

引用

International Conference on Information Networking

作者： Seunghyun Lee Haesung Jo Jihyeon Yun Changhee Joo Department of Computer Science and Engineering Korea University Seoul Korea

ISBN: (数字)9798331506940

ISBN: (纸本)9798331506957

distributed computation is the widely used methodology to overcome challenges that the application covering multiple mobile devices mostly experiences, such as the high complexity of the computation and the resource limitation. By splitting the required computation and distribute the computation across the multiple devices, it can achieve lowered computation time and resource required per device and effective utilization in terms of the total resource management. This is risen as an appropriate approach to manage problems that recent applications with deep learning process have. Followed by the generalization of Internet of Things (IoT) and the development of data collecting technology, the deep learning process has to handle much larger dataset which makes it hard to be transferred through the network. This also leads to more complex computation that a single device may not be able to operate itself. In this paper, we consider the distributed computation applied in various fields, and how it is applied to distribute the deep learning process through observing researches studying about it.

关键词： Deep learning Data collection Market research Mobile handsets Complexity theory Internet of Things Resource management distributed computing Mobile computing

来源：评论

学校读者我要写书评

暂无评论

Asymptotically Optimal Coded distributed computing via Combinatorial Designs

引用

IEEE-ACM TRANSACTIONS ON NETWORKING 2024年第4期32卷 3018-3033页

作者： Cheng, Minquan Wu, Youlong Li, Xianxian Wu, Dianhua Guangxi Normal Univ Minist Educ Key Lab Educ Blockchain & Intelligent Technol Guilin 541004 Peoples R China Guangxi Normal Univ Guangxi Key Lab Multisource Informat Miningand Sec Guilin 541004 Peoples R China ShanghaiTech Univ Sch Informat Sci & Technol Shanghai 201210 Peoples R China Guangxi Normal Univ Sch Math & Stat Guilin 541006 Peoples R China

Coded distributed computing (CDC) introduced by Li et al. can greatly reduce the communication load for MapReduce computing systems. In the cascaded CDC with K workers, N input files and Q output functions, each input file will be mapped by r workers and each output function will be computed by s workers such that coding techniques can be applied to create multicast opportunities. The main drawback of most existing CDC schemes is that they require the original data to be split into a large number of input files that grows exponentially with K , which would significantly increase the coding complexity and degrade the system performance. In this paper, we first use a classical combinatorial structure t -design, for any integer t >= 2 >= 2 , to develop a low-complexity and communication-efficient CDC with r=s. Our scheme has much smaller N and Q than the existing schemes under the same parameters K , r , and s;and achieves smaller communication loads compared with the state-of-the-art schemes when K is relatively large. Remarkably, unlike the previous schemes that realize on large operation fields, our scheme operates in one-shot communication on the minimum binary field F-2 . With a derived lower bound on the communication load under one-shot linear delivery, we show that the t -design scheme is asymptotically optimal. Furthermore, we show that our construction method can incorporate the other combinatorial structures that have a similar property to t -design. For instance, we use t -GDD to obtain another one-shot asymptotically optimal CDC scheme over F-2 that has different parameters from t -design. Finally, we show that our construction method can also be used to construct CDC schemes with r not equal s that have small file number and output function number. Like what you're reading?

关键词： distributed computing Costs Task analysis Encoding Vectors Technological innovation Symbols Coded distributed computing asymptotically optimal t-design t-GDD

来源：评论

学校读者我要写书评

暂无评论

Handoff-Aware distributed computing in High Altitude Platform Station (HAPS)--Assisted Vehicular Networks

引用

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS 2023年第12期22卷 8814-8827页

作者： Ren, Qiqi Abbasi, Omid Kurt, Gunes Karabulut Yanikomeroglu, Halim Chen, Jian Xidian Univ State Key Lab Integrated Serv Networks Xian 710071 Shaanxi Peoples R China Carleton Univ Dept Syst & Comp Engn Nonterr Networks NTN Lab Ottawa ON K1S 5B6 Canada Polytech Montreal Dept Elect Engn Montreal PQ H3T 1J4 Canada

distributed computing enables Internet of vehicle (IoV) services by collaboratively utilizing the computing resources from the network edge and the vehicles. However, the computing interruption issue caused by frequent edge network handoffs, and a severe shortage of computing resources are two problems in providing IoV services. High altitude platform station (HAPS) computing can be a promising addition to existing distributed computing frameworks because of its wide coverage and strong computational capabilities. In this regard, this paper proposes an adaptive scheme in a new distributed computing framework that involves HAPS computing to deal with the two problems of the IoV. Based on the diverse demands of vehicles, network dynamics, and the time-sensitivity of handoffs, the proposed scheme flexibly divides each task into three parts and assigns them to the vehicle, roadside units (RSUs), and a HAPS to perform synchronous computing. The scheme also constrains the computing of tasks at RSUs such that they are completed before handoffs to avoid the risk of computing interruptions. On this basis, we formulate a delay minimization problem that considers task-splitting ratio, transmit power, bandwidth allocation, and computing resource allocation. To solve the problem, variable replacement and successive convex approximation-based method are proposed. The simulation results show that this scheme not only avoids the negative effects caused by handoffs in a flexible manner, it also takes delay performance into account and maintains the delay stability.

关键词： Task analysis Servers Delays Computational modeling Wireless communication distributed databases Autonomous aerial vehicles Internet of Vehicles (IoV) high altitude platform station (HAPS) distributed computing edge network handoff

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：