检索结果-内蒙古大学图书馆

Efficient Replication for Fast and Predictable Performance in distributed computing

IEEE-ACM TRANSACTIONS ON NETWORKING 2021年第4期29卷 1467-1476页

作者： Behrouzi-Far, Amir Soljanin, Emina Rutgers State Univ Dept Elect & Comp Engn New Brunswick NJ 08901 USA

Master-worker distributed computing systems use task replication to mitigate the effect of slow workers on job compute time. The master node groups tasks into batches and assigns each batch to one or more workers. We first assume that the batches do not overlap. Using majorization theory, we show that a balanced replication of batches minimizes the average job compute time for a general class of service time distributions. We then show that the balanced assignment of non-overlapping batches achieves a lower average job compute time than the overlapping schemes proposed in the literature. Next, we derive the optimum redundancy level as a function of the task service time distribution. We show that the redundancy level that minimizes the average job compute time may not coincide with the redundancy level that maximizes job compute time predictability. Therefore, there is a trade-off in optimizing the two metrics. By running experiments on Google cluster traces, we observe that redundancy can reduce the job compute time by one order of magnitude. The optimum level of redundancy depends on the distribution of task service time.

关键词： Task analysis Redundancy Computational modeling Machine learning Internet Computer architecture Training Redundancy replication distributed systems distributed computing latency coefficient of variations

来源：评论

学校读者我要写书评

暂无评论

Modular Reconfigurable Robot distributed computing System for Tracking Multiple Objects

引用

IEEE SYSTEMS JOURNAL 2021年第1期15卷 802-813页

作者： Romanov, Alexey M. Romanov, Mikhail P. Manko, Sergey V. Volkova, Maria A. Chiu, Wei-Yu Ma, Hsi-Pin Chiu, Kun-Yen MIREA Russian Technol Univ Inst Cybernet Moscow 119454 Russia Natl Tsing Hua Univ Dept Elect Engn Hsinchu 30013 Taiwan

A modular reconfigurable robot (MRR) is considered as a system of systems;that is, a robot control system consists of modular systems. For an MRR, onboard tracking of multiple objects is one of the most challenging tasks. This is because the task requires lots of computation resources, but an MRR system is constrained by several physical limitations. To enhance the computing performance of an MRR control system, a novel approach to organizing a distributed computing system is proposed. The key components of the proposed approach are FPGA-based specialized computational cores (SCCs). An SCC for tracking objects is designed to utilize spare resources of the MRR module's control devices. Analysis and experiments on a real testbench are performed to illustrate that the proposed approach can enhance overall computing performance of an MRR control system without adding additional components. As such, weight and dimensional parameters are saved;unnecessary power consumption and heat generation are reduced. One experimental example shows that the proposed SCC enables a Kalman filter to track objects up to 100 times faster than a comparable solution implemented with microcontrollers, while the whole distributed computing system can outperform a similar one built on high-performance CPUs.

关键词： Task analysis Robot sensing systems Field programmable gate arrays Cloud computing Performance evaluation distributed computing Ethernet POWERLINK FPGA Kalman filter matrix computation modular reconfigurable robot (MRR) object tracking

来源：评论

学校读者我要写书评

暂无评论

"Semi-Asynchronous": A New Scheduler in distributed computing

引用

IEEE ACCESS 2021年 9卷 41540-41557页

作者： Cicerone, Serafino di Stefano, Gabriele Navarra, Alfredo Univ Aquila Dept Informat Engn Comp Sci & Math I-67100 Laquila Italy Univ Perugia Dept Math & Comp Sci I-06123 Perugia Italy

The study of mobile entities that based on local information have to accomplish global tasks is of main interest for the scientific community. Classic models for the activation and synchronization of mobile entities are the fully-synchronous (FSync), semi-synchronous (SSync), and asynchronous (Async) models, where entities alternate between active and inactive states with different timing. According to the assumed synchronization model, very different results have been achieved in the field of distributed computing. One of the main outcomes is the big gap between the Async and the other models in terms of manageability and algorithm design. In fact, there are still many problems for which it is not known whether synchronicity is crucial for designing resolution algorithms or not. In order to better understand the Async case, here we propose a further model referred to as the semi-asynchronous (SAsync). This slightly deviates from SSync. In fact, like in SSync (and FSync), the duration of the activation of an entity is kept of fixed time whereas, like in Async, the starting instant of the activation is not fully synchronized with the possible activation of other entities. We show that for entities moving on graphs, the SSync model allows accomplishing more tasks than the SAsync that in turn allows accomplishing more tasks than the Async. Furthermore, our results show that, especially to tackle problems in the Euclidean plane, the SAsync model is already quite challenging, therefore there is no need to get involved with complications arising in the Async model.

关键词： Robots Computational modeling Synchronization Robot kinematics Task analysis distributed computing Standards distributed algorithms gathering mobile robots synchronization

来源：评论

学校读者我要写书评

暂无评论

Evaluating distributed computing Infrastructures: An Empirical Study Comparing Hadoop Deployments on Cloud and Local Systems

引用

IEEE TRANSACTIONS ON CLOUD computing 2021年第3期9卷 1075-1088页

作者： Bhattacharya, Devipsita Currim, Faiz Ram, Sudha Univ Albany SUNY Dept Informat Secur & Digital Forens Albany NY 12222 USA Univ Arizona Dept Management Informat Syst Tucson AZ 85721 USA

The popularity of distributed computing platforms (e.g., Hadoop) is largely due to their ability to address scalability issues that arise from data storage and processing limitations of standard computing systems. However, the decision to dedicate organizational resources and capital for such systems needs a careful consideration of several factors including evaluation of cloud-based distributed computing options. We propose a framework of metrics which we used to conduct an in-depth performance and cost benefit analysis of two standard Hadoop infrastructural choices, i.e., a Platform as a Service (PaaS) on-demand cloud setup and a local organizational setup. We evaluated the framework by means of an exploratory data analysis use-case for a large-scale graph processing research problem. Our analysis considered highly granular aspects of distributed computing performance and studied how utilization rates and infrastructure amortization times affect break-even times. We identified that virtual memory management adversely affects the performance of a cloud cluster during the reduce phase with the magnitude of degradation dependent on the type of MapReduce operation. Our study is intended not only as an evaluation of infrastructural choices but also a development of a metric framework that can serve as a baseline for researchers examining distributed infrastructures.

关键词： Cloud computing computers and information processing cost benefit analysis distributed computing data processing performance evaluation platform-as-a-service

来源：评论

学校读者我要写书评

暂无评论

Local-Global Modeling and distributed computing Framework for Nonlinear Plant-Wide Process Monitoring With Industrial Big Data

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2021年第8期32卷 3355-3365页

作者： Jiang, Qingchao Yan, Shifu Cheng, Hui Yan, Xuefeng East China Univ Sci & Technol Key Lab Adv Control & Optimizat Chem Proc Minist Educ Shanghai 200237 Peoples R China Tongji Univ Shanghai Inst Intelligent Sci & Technol Shanghai 200092 Peoples R China

Industrial big data and complex process nonlinearity have introduced new challenges in plant-wide process monitoring. This article proposes a local-global modeling and distributed computing framework to achieve efficient fault detection and isolation for nonlinear plant-wide processes. First, a stacked autoencoder is used to extract dominant representations of each local process unit and establish the local inner monitor. Second, mutual information (MI) is used to determine the neighborhood variables of a local unit. Afterward, a joint representation learning is then performed between the local unit and the neighborhood variables to extract the outer-related representations and establish the outer-related monitor for the local unit. Finally, the outer-related representations from all process units are used to establish global monitoring systems. Given that the modeling of each unit can be performed individually, the computation process can be efficiently completed with different CPUs. The proposed modeling and monitoring method is applied to the Tennessee Eastman (TE) and laboratory-scale glycerol distillation processes to demonstrate the feasibility of the method.

关键词： Monitoring Kernel Computational modeling Fault detection Correlation Principal component analysis Big Data Deep neural network (DNN) distributed computing industrial big data local-global modeling nonlinear plant-wide processes

来源：评论

学校读者我要写书评

暂无评论

On Batch-Processing Based Coded computing for Heterogeneous distributed computing Systems

引用

IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING 2021年第3期8卷 2438-2454页

作者： Wang, Baoqian Xie, Junfei Lu, Kejie Wan, Yan Fu, Shengli San Diego State Univ Dept Elect & Comp Engn San Diego CA 92182 USA Univ Calif San Diego San Diego CA 92182 USA Univ Puerto Rico Dept Comp Sci & Engn Mayaguez PR 00681 USA Univ Texas Arlington Dept Elect Engn Arlington TX 76019 USA Univ North Texas Dept Elect Engn Denton TX 76201 USA

In recent years, coded distributed computing (CDC) has attracted significant attention, because it can efficiently facilitate many delay-sensitive computation tasks against unexpected latencies in distributed computing systems. Despite such a salient feature, many design challenges and opportunities remain. In this paper, we focus on practical computing systems with heterogeneous computing resources, and design a novel CDC approach, called batch-processing based coded computing (BPCC), which exploits the fact that every computing node can obtain some coded results before it completes the whole task. To this end, we first describe the main idea of the BPCC framework, and then formulate an optimization problem for BPCC to minimize the task completion time by configuring the computation load. Through formal theoretical analyses, extensive simulation studies, and comprehensive real experiments on the Amazon EC2 computing clusters, we demonstrate promising performance of the proposed BPCC scheme, in terms of high computational efficiency and robustness to uncertain disturbances.

关键词： Task analysis distributed computing Encoding Optimization Computational modeling Heterogeneous networks Resource management Coded distributed computing heterogeneous computing cluster batch-processing asymptotic optimality latency

来源：评论

学校读者我要写书评

暂无评论

An Optimal Error Correction Scheme for the Shuffle Phase of a MapReduce distributed computing System

引用

IEEE COMMUNICATIONS LETTERS 2021年第12期25卷 3765-3769页

作者： Mahesh, Anjana A. Karat, Nujoom Sageer Rajan, B. Sundar Indian Inst Sci Dept Elect Commun Engn Bengaluru 560012 India Indian Inst Technol Dept Elect Engn Mumbai 400076 Maharashtra India

The MapReduce model of distributed computation accomplishes a task in three phases - two computation phases-Map and Reduce, with a communication phase - Shuffle, happening in between. This letter looks at the distributed communication problem in the shuffle phase with the assumption that the links through which the computing nodes exchange information is error-prone. Under this assumption, an optimal linear error-correcting transmission scheme is designed using index coding techniques.

关键词： Indexes Encoding distributed computing Computational modeling Task analysis Receivers Error correction codes distributed computing error correction index coding MapReduce

来源：评论

学校读者我要写书评

暂无评论

DistSNE: distributed computing and online visualization of DNA methylation-based central nervous system tumor classification

引用

BRAIN PATHOLOGY 2024年第3期34卷 e13228-e13228页

作者： Schmid, Kai Sehring, Jannik Nemeth, Attila Harter, Patrick N. Weber, Katharina J. Vengadeswaran, Abishaa Storf, Holger Seidemann, Christian Karki, Kapil Fischer, Patrick Dohmen, Hildegard Selignow, Carmen von Deimling, Andreas Grau, Stefan Schroeder, Uwe Plate, Karl H. Stein, Marco Uhl, Eberhard Acker, Till Amsel, Daniel Justus Liebig Univ Giessen Inst Neuropathol Giessen Germany Univ Hosp Frankfurt Neurol Inst Edinger Inst Frankfurt Germany German Canc Consortium DKTK Heidelberg Germany German Canc Res Ctr Heidelberg Germany Frankfurt Canc Inst FCI Frankfurt Germany Univ Canc Ctr UCT Frankfurt Frankfurt Germany Goethe Univ Frankfurt Univ Hosp Frankfurt Med Informat Grp MIG Frankfurt Germany Phillips Univ Marburg DIZ Marburg Marburg Germany Justus Liebig Univ Inst Med Informat Giessen Germany Univ Klinikum Heidelberg German Canc Res Ctr DKFZ Dept Neuropathol Heidelberg Germany CCU Neuropathol Heidelberg Germany Univ Appl Sci Fac Hlth Sci Giessen Germany Hosp Fulda Dept Neurosurg Fulda Germany MVZ Frankfurt Oder Dept Neurosurg Frankfurt Germany Univ Hosp Giessen & Marburg Locat Giessen Dept Neurosurg Giessen Germany Justus Liebig Univ Giessen Inst Neuropathol Arndtstr 16 Giessen Germany Univ Hosp Munich Ctr Neuropathol & Prion Res Munich Germany

The current state-of-the-art analysis of central nervous system (CNS) tumors through DNA methylation profiling relies on the tumor classifier developed by Capper and colleagues, which centrally harnesses DNA methylation data provided by users. Here, we present a distributed-computing-based approach for CNS tumor classification that achieves a comparable performance to centralized systems while safeguarding privacy. We utilize the t-distributed neighborhood embedding (t-SNE) model for dimensionality reduction and visualization of tumor classification results in two-dimensional graphs in a distributed approach across multiple sites (DistSNE). DistSNE provides an intuitive web interface () for user-friendly local data management and federated methylome-based tumor classification calculations for multiple collaborators in a DataSHIELD environment. The freely accessible web interface supports convenient data upload, result review, and summary report generation. Importantly, increasing sample size as achieved through distributed access to additional datasets allows DistSNE to improve cluster analysis and enhance predictive power. Collectively, DistSNE enables a simple and fast classification of CNS tumors using large-scale methylation data from distributed sources, while maintaining the privacy and allowing easy and flexible network expansion to other institutes. This approach holds great potential for advancing human brain tumor classification and fostering collaborative precision medicine in neuro-oncology. Overview of CNS tumor methylome data and the computational setup (a). Data contribution of the collected data. The most frequent subgroups are color-coded. (b) R Shiny web application for distributed t-SNE plotting. Upon user login, raw data can be uploaded to the analysis server via the R Shiny interface. A zoom-in of the area of interest provides a detailed view of the sample (marked with an asterisk) and its surrounding area. (c) Structure of the distributed t-SNE

关键词： brain tumor classification distributed computing DataSHIELD methylome tSNE webinterface

来源：评论

学校读者我要写书评

暂无评论

A Multi-Level distributed computing Approach to XDraw Viewshed Analysis Using Apache Spark

引用

REMOTE SENSING 2023年第3期15卷 761-761页

作者： Dong, Junduo Zhang, Jianbo China Univ Geosci Sch Geog Informat Engn Wuhan 430074 Peoples R China

Viewshed analysis is a terrain visibility computation method based on the digital elevation model (DEM). With the rapid growth of remote sensing and data collection technologies, the volume of large-scale raster DEM data has reached a great size (ZB). However, the data storage and GIS analysis based on such large-scale digital data volume become extra difficult. The usually distributed approaches based on Apache Hadoop and Spark can efficiently handle the viewshed analysis computation of large-scale DEM data, but there are still bottleneck and precision problems. In this article, we present a multi-level distributed XDraw (ML-XDraw) algorithm with Apache Spark to handle the viewshed analysis of large DEM data. The ML-XDraw algorithm mainly consists of 3 parts: (1) designing the XDraw algorithm into a multi-level distributed computing process, (2) introducing a multi-level data decomposition strategy to solve the calculating bottleneck problem of the cluster's executor, and (3) proposing a boundary approximate calculation strategy to solve the precision loss problem in calculation near the boundary. Experiments show that the ML-XDraw algorithm adequately addresses the above problems and achieves better speed-up and accuracy as the volume of raster DEM data increases drastically.

关键词： viewshed analysis XDraw Spark distributed computing

来源：评论

学校读者我要写书评

暂无评论

Harnessing the distributed computing Paradigm for Laser-Induced Breakdown Spectroscopy

引用

BIG DATA AND COGNITIVE computing 2024年第11期8卷 154页

作者： Silva, Nuno A. INESC TEC Inst Syst & Comp Engn Technol & Sci Ctr Appl Photon Rua do Campo Alegre 687 P-4150179 Porto Portugal

Laser-induced breakdown spectroscopy allows fast and versatile elemental analysis, standing as a promising technique for a wide range of applications both at the research and industry levels. Yet, its high operation speed comes with a high throughput of data, which introduces some challenges at the level of the data processing domain, mainly due to the large computational load and data volume. In this work, we analyze and discuss opportunities of distributed computing paradigms and resources to address some of these challenges, covering most of the procedures usually employed in typical applications. We infer the possible impact of such computing resources by presenting some metrics of simple processing prototypes running in state-of-the-art computing facilities. Our results allow us to conclude that, while underexplored so far, these computing resources may allow for the development of tools for timely research and analysis in demanding applications and introduce novel solutions toward a more agile working environment.

关键词： high-performance computing distributed computing big data laser-induced breakdown spectroscopy

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：