distributed computing became very popular in the last two decades and in many cases such projects are based on a donated calculation power. With the expansion of the mobile devices in the last decade it become relevan...
详细信息
ISBN:
(纸本)9783030366254;9783030366247
distributed computing became very popular in the last two decades and in many cases such projects are based on a donated calculation power. With the expansion of the mobile devices in the last decade it become relevant some donated distributed computing solutions to be developed as mobile applications. Such a solution was developed at IICT-BAS [2], which is based on an Android Live Wallpaper technology. This research proposes an extension of the work done at IICT-BAS in the direction of human-computer based distributed computing by providing software capabilities of the users to vote for future financial changes. At the beginning, a brief overview of the topic with special emphasis on ANNs/GAs and their strengths/weaknesses when applied for the problem's solution is given. After that the study treats the aspect of extension of a distributed computing system based on mobile devices to human-computer distributed computing. Experiments and results are presented in Sect. 3 and the final Sect. 4 concludes and provides some further work suggestions.
The optimal storage-computation tradeoff is characterized for a MapReduce-like distributed computing system with straggling nodes, where only a part of the nodes can be utilized to compute the desired output functions...
详细信息
ISBN:
(纸本)9781538692912
The optimal storage-computation tradeoff is characterized for a MapReduce-like distributed computing system with straggling nodes, where only a part of the nodes can be utilized to compute the desired output functions. The result holds for arbitrary output functions and thus generalizes previous results that restricted to linear functions. Specifically, in this work, we propose a new information-theoretical converse and a new matching coded computing scheme, that we call coded computing for straggling systems (CCS).
This article describes the study results of semi-structured data processing and analysis of the Russian court decisions (almost 30 million) using distributed cluster-computing framework and machine learning. Spark was...
详细信息
This article describes the study results of semi-structured data processing and analysis of the Russian court decisions (almost 30 million) using distributed cluster-computing framework and machine learning. Spark was used for data processing and decisions trees were used for analysis. The results of the automation of data collection and structuring of court decisions are presented. The methods for extracting and structuring knowledge from semi-structured data for the field ofjustice, taking into account the specifics of the Russian Federation legislation, have been developed. On the example of the fire safety law, the machine learning method for identify the effectiveness of changes in the law and predictions of the consequences of changing the law is demonstrated. It is also shown an association on the impact of lawmaking on law enforcement. The regularities in law enforcement change associate by changes in the law. The connections of law enforcement with economic and social indicators between the regions are identified. The judicial interpretations of the observations are also described in this article what proves the compliance of the results. (C) 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC -ND license (https://***/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science.
Coded distributed computing (CDC) introduced by Li et al. in 2015 offers an efficient approach to trade computing power to reduce the communication load in general distributed computing frameworks such as MapReduce. F...
详细信息
ISBN:
(纸本)9781538692912
Coded distributed computing (CDC) introduced by Li et al. in 2015 offers an efficient approach to trade computing power to reduce the communication load in general distributed computing frameworks such as MapReduce. For the more general cascaded CDC, Map computations are repeated at r nodes to significantly reduce the communication load among nodes tasked with computing Q Reduce functions s times. While an achievable cascaded CDC scheme was proposed, it only operates on homogeneous networks, where the storage, computation load and communication load of each computing node is the same. In this paper, we address this limitation by proposing a novel combinatorial design which operates on heterogeneous networks where nodes have varying storage and computing capabilities. We provide an analytical characterization of the computation-communication trade-off and show that it is optimal within a constant factor and could outperform the state-of-the-art homogeneous schemes.
The goal of this study is to take advantage of artificial intelligence (AI) algorithms and distributed computing ecosystem to enhance the 3D-multibeam echo sounder data processing functionality. We consider first the ...
详细信息
ISBN:
(纸本)9781728114507
The goal of this study is to take advantage of artificial intelligence (AI) algorithms and distributed computing ecosystem to enhance the 3D-multibeam echo sounder data processing functionality. We consider first the post processing case, where a complete dataset has been recorded during sea trials. Hence, our suggested framework designed for massive real world data processing allows employing false alarm reduction and underwater object recognition techniques, which can be easily used as decision making for underwater autonomous vehicle safe navigation.
Coded computation is a framework which provides redundancy in distributed computing systems to speed up large-scale tasks. Although most existing works assume error-free scenarios, the link failures are common in curr...
详细信息
ISBN:
(纸本)9781538692912
Coded computation is a framework which provides redundancy in distributed computing systems to speed up large-scale tasks. Although most existing works assume error-free scenarios, the link failures are common in current wired/wireless networks. In this paper, we consider the straggler problem in distributed computing systems with link failures, by modeling the links between the master node and worker nodes as packet erasure channels. We first analyze the latency in this setting using an (n, k) maximum distance separable (MDS) code. Then, we consider a setup where the number of retransmissions is limited due to the bandwidth constraint. By formulating practical optimization problems related to latency, bandwidth and probability of successful computation, we obtain achievable performance curves as a function of packet erasure probability.
Traffic flows in a distributed computing network require both transmission and processing, and can be interdicted by removing either communication or computation resources. We study the robustness of a distributed com...
详细信息
ISBN:
(纸本)9781538684610
Traffic flows in a distributed computing network require both transmission and processing, and can be interdicted by removing either communication or computation resources. We study the robustness of a distributed computing network under the failures of communication links and computation nodes. We define cut metrics that measure the connectivity, and show a non-zero gap between the maximum flow and the minimum cut. Moreover, we study a network flow interdiction problem that minimizes the maximum flow by removing communication and computation resources within a given budget. We develop mathematical programs to compute the optimal interdiction, and polynomial-time approximation algorithms that achieve near-optimal interdiction in simulation.
In recent years, deep learning has made great progress in image classification and detection. Popular deep learning algorithms rely on deep networks and multiple rounds of back-propagations. In this paper, we propose ...
详细信息
ISBN:
(纸本)9783030374297;9783030374280
In recent years, deep learning has made great progress in image classification and detection. Popular deep learning algorithms rely on deep networks and multiple rounds of back-propagations. In this paper, we propose two approaches to accelerate deep networks. One is expanding the width of every layer. We reference to the Extreme Learning Machine, setting big number of convolution kernels to extract features in parallel. It can obtain multiscale features and improve network efficiency. The other is freezing part of layers. It can reduce back-propagations and speed up the training procedure. From the above, it is a random convolution architecture that network is proposed for image classification. In our architecture, every combination of random convolutions extracts distinct features. Apparently, we need a lot of experiments to choose the best combination. However, centralized computing may limit the number of combinations. Therefore, a decentralized architecture is used to enable the use of multiple combinations.
New generations of high-performance computing applications depend on an increasing number of components to satisfy their growing demand for computation. On such large systems, the execution of long-running jobs is mor...
详细信息
ISBN:
(纸本)9781728142227
New generations of high-performance computing applications depend on an increasing number of components to satisfy their growing demand for computation. On such large systems, the execution of long-running jobs is more likely affected by component failures. Failure classes vary from frequent transient memory faults to rather rare correlated node errors. Multilevel checkpoint/restart has been introduced to proactively cope with failures at different levels. Writing checkpoints on slower stable devices, which survive fatal failures, causes more overhead than writing them on fast devices (main memory or local SSD), which, however, only protect against light faults. Given a graph of the components of a particular storage hierarchy mapping their fault-domains and their expected mean time to failure (MTTF), we optimize the checkpoint frequencies for each level of the storage hierarchy (multilevel checkpointing) to minimize the overhead and runtime of a given job. We reduce the checkpoint/restart overhead of large dataintensive jobs compared to state-of-the-art solutions on multilevel checkpointing by up to 10 percent in the investigated cases. The improvement increases further with growing checkpoint sizes.
Most of the clustering algorithms are designed to work as a sequential algorithm that requires all data to be present, which limits the actual implementation to run on a single machine and does not support horizontal ...
详细信息
ISBN:
(纸本)9781728106854
Most of the clustering algorithms are designed to work as a sequential algorithm that requires all data to be present, which limits the actual implementation to run on a single machine and does not support horizontal scalability. This is problematic in today's context when volume of data gets larger each day and the need to process data quickly is essential. Hence, in this paper we propose a platform that allows running clustering algorithms in a distributed manner. This is achieved through splitting the data into smaller and equal partitions, and through redesigning the original clustering algorithms to allow working on a sub-set of the input data without having to interact with the processing of the rest of the input data. At the end the so-called reduce phase aggregates the partial results obtained from processing each partition and it produces the global result.
暂无评论