检索结果-内蒙古大学图书馆

Resolvable Designs for Speeding Up Distributed computing

IEEE-ACM TRANSACTIONS ON NETWORKING 2020年第4期28卷 1657-1670页

作者： Konstantinidis, Konstantinos Ramamoorthy, Aditya Iowa State Univ Dept Elect & Comp Engn Ames IA 50011 USA

Distributed computing frameworks such as MapReduce are often used to process large computational jobs. They operate by partitioning each job into smaller tasks executed on different servers. The servers also need to exchange intermediate values to complete the computation. Experimental evidence suggests that this so-called Shuffle phase can be a significant part of the overall execution time for several classes of jobs. Prior work has demonstrated a natural tradeoff between computation and communication whereby running redundant copies of jobs can reduce the Shuffle traffic load, thereby leading to reduced overall execution times. For a single job, the main drawback of this approach is that it requires the original job to be split into a number of files that grows exponentially in the system parameters. When extended to multiple jobs (with specific function types), these techniques suffer from a limitation of a similar flavor, i.e., they require an exponentially large number of jobs to be executed. In practical scenarios, these requirements can significantly reduce the promised gains of the method. In this work, we show that a class of combinatorial structures called resolvable designs can be used to develop efficient coded distributed computing schemes for both the single and multiple job scenarios considered in prior work. We present both theoretical analysis and exhaustive experimental results (on Amazon EC2 clusters) that demonstrate the performance advantages of our method. For the single and multiple job cases, we obtain speed-ups of 4.69x (and 2.6x over prior work) and 4.31x over the baseline approach, respectively.

关键词： Servers Task analysis IEEE transactions Protocols Machine learning Redundancy MapReduce data-intensive computing coded multicasting communication load TeraSort aggregate functions distributed learning

来源：评论

学校读者我要写书评

暂无评论

Toward a Microarchitecture for Efficient Execution of Irregular Applications

引用

ACM TRANSACTIONS ON PARALLEL computing 2020年第4期7卷 1–24页

作者： Leidel, John D. Wang, Xi Williams, Brody Chen, Yong Tact Comp Labs Box 43104 Lubbock TX 79409 USA Texas Tech Univ Box 43104 Lubbock TX 79409 USA

Given the increasing importance of efficient data-intensive computing, we find that modern processor designs are not well suited to the irregular memory access patterns often found in these algorithms. Applications and algorithms that do not exhibit spatial and temporal memory request locality induce high latency and low memory bandwidth due to the high cache miss rate. In response to the performance penalties inherently present in applications with irregular memory accesses, we introduce a GoblinCore-64 (GC64) architecture and a unique memory hierarchy that are explicitly designed to exploit memory performance from irregular memory access patterns. GC64 provides a pressure-driven hardware-managed concurrency control to minimize pipeline stalls and lower the latency of context switches. A novel memory coalescing model is also introduced to enhance the performance of memory systems via request aggregations. We have evaluated the performance benefits of our approach using a series of 24 benchmarks and the results show nearly 50% memory request reductions and a performance acceleration of up to 14.6x.

关键词： data-intensive computing irregular algorithms thread concurrency context switching

来源：评论

学校读者我要写书评

暂无评论

Canary: fault-tolerant FaaS for stateful time-sensitive applications 22

Canary: fault-tolerant FaaS for stateful time-sensitive appl...

引用

Proceedings of the International Conference on High Performance computing, Networking, Storage and Analysis

作者： Moiz Arif Kevin Assogba M. Mustafa Rafique Rochester Institute of Technology

Function-as-a-Service (FaaS) platforms have recently gained rapid popularity. Many stateful applications have been migrated to FaaS platforms due to their ease of deployment, scalability, and minimal management overhead. However, failures in FaaS have not been thoroughly investigated, thus making these desirable platforms unreliable for guaranteeing function execution and ensuring performance requirements. In this paper, we propose Canary, a highly resilient and fault-tolerant framework for FaaS that mitigates the impact of failures and reduces the overhead of function restart. Canary utilizes replicated container runtimes and application-level checkpoints to reduce application recovery time over FaaS platforms. Our evaluations using representative stateful FaaS applications show that Canary reduces the application recovery time and dollar cost by up to 83% and 12%, respectively over the default retry-based strategy. Moreover, it improves application availability with an additional average execution time and cost overhead of 14% and 8%, respectively, as compared to the ideal failure-free execution.

关键词： data-intensive computing deep learning serverless computing OpenWhisk data parallelism TensorFlow

来源：评论

学校读者我要写书评

暂无评论

DECO: Joint Computation, Caching and Forwarding in data-Centric computing Networks 19

DECO: Joint Computation, Caching and Forwarding in Data-Cent...

引用

20th ACM International Symposium on Mobile Ad Hoc Networking and computing (Mobihoc)

作者： Kamran, Khashayar Yeh, Edmund Ma, Qian Northeastern Univ Boston MA 02115 USA

ISBN: (纸本)9781450367646

The emergence of IoT devices and the predicted increase in the number of data-driven and delay-sensitive applications highlight the importance of dispersed computing platforms (e.g. edge computing and fog computing) that can intelligently manage in-network computation and data placement. In this paper, we propose the DECO (data-cEntric Computation) framework for joint computation, caching, and request forwarding in data-centric computing networks. DECO utilizes a virtual control plane which operates on the demand rates for computation and data, and an actual plane which handles computation requests, data requests, data objects and computation results in the physical network. We present a throughput optimal policy within the virtual plane, and use it as a basis for adaptive and distributed computation, caching, and request forwarding in the actual plane. We demonstrate the superior performance of the DECO policy in terms of request satisfaction delay as compared with several baseline policies, through extensive numerical simulations over multiple network topologies.

关键词： Distributed computing networks fog computing mobile edge computing data-intensive computing data-centric computing caching

来源：评论

学校读者我要写书评

暂无评论

PAC: Paged Adaptive Coalescer for 3D-Stacked Memory 20

PAC: Paged Adaptive Coalescer for 3D-Stacked Memory

引用

29th International Conference on High-Performance Parallel and Distributed computing (HPDC)

作者： Wang, Xi Leidel, John D. Williams, Brody Chen, Yong Texas Tech Univ Lubbock TX 79409 USA Tactical Comp Labs Muenster TX 76252 USA

ISBN: (纸本)9781450370523

Many contemporary data-intensive applications exhibit irregular and highly concurrent memory access patterns and thus challenge the performance of conventional memory systems. Driven by an expanding need for high-bandwidth memory featuring low access latency, 3D-stacked memory devices, such as the Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM), were designed to provide significantly higher throughput as compared to standard JEDEC DDR devices. However, existing memory interfaces and coalescing models, designed for conventional DDR devices, are unable to fully exploit the bandwidth potential inherent in these new 3D-stacked memory devices. In order to remedy this disparity, we introduce in this work a novel paged adaptive coalescer (PAC) infrastructure with a scalable coalescing network for 3D-stacked memory. We present the design and simulated implementation of this approach on RISC-V embedded cores with attached HMC devices. We have carried out extensive evaluations and the results show that the proposed PAC methodology yields an average coalescing efficiency of 56.01%. Further, our evaluation results also show that the PAC reduces bank conflicts and the power consumption by 85.16% and 59.21%, respectively. Overall, PAC achieves an average performance gain of 14.35% (and up to 26.06%) across 14 test suites. These results showcase the potential of the PAC methodology as applied to architecture design for increasingly critical data-intensive algorithms and applications.

关键词： 3D-stacked memory Memory coalescing data-intensive computing

来源：评论

学校读者我要写书评

暂无评论

pipsCloud: High performance cloud computing for remote sensing big data management and processing

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年第Part1期78卷 353-368页

作者： Wang, Lizhe Ma, Yan Yan, Jining Chang, Victor Zomaya, Albert Y. China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Chinese Acad Sci Inst Remote Sensing & Digital Earth Beijing Peoples R China Xian Jiaotong Liverpool Univ Suzhou Peoples R China Univ Sydney Sch Informat Technol Sydney NSW Australia

Massive, large-region coverage, multi-temporal, multi-spectral remote sensing (RS) datasets are employed widely due to the increasing requirements for accurate and up-to-date information about resources and the environment for regional and global monitoring. In general, RS data processing involves a complex multi-stage processing sequence, which comprises several independent processing steps according to the type of RS application. RS data processing for regional environmental and disaster monitoring is recognized as being computationally intensive and data intensive. We propose pipsCloud to address these issues in an efficient manner, which combines recent cloud computing and HPC techniques to obtain a large-scale RS data processing system that is suitable for on-demand real-time services. Due to the ubiquity, elasticity, and high-level transparency of the cloud computing model, massive RS data management and data processing for dynamic environmental monitoring can all be performed on the cloud via Web interfaces. A Hilbert-R+-based data indexing method is employed for the optimal querying and access of RS images, RS data products, and interim data. In the core platform beneath the cloud services, we provide a parallel file system for massive high dimensional RS data, as well as interfaces for accessing irregular RS data to improve data locality and optimize the I/O performance. Moreover, we use an adaptive RS data analysis workflow management system for on-demand workflow construction and the collaborative processing of a distributed complex chain of RS data, e.g., for forest fire detection, mineral resources detection, and coastline monitoring. Our experimental analysis demonstrated the efficiency of the pipsCloud platform. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Big data Cloud computing data-intensive computing High performance computing Remote sensing

来源：评论

学校读者我要写书评

暂无评论

An Improved Bayesian Inference Method for data-intensive computing

An Improved Bayesian Inference Method for Data-Intensive Com...

引用

6th International Symposium on Intelligence Computation and Applications (ISICA 2012)

作者： Ma, Feng Liu, Weiyi Yunnan Univ Sch Informat Sci & Engn Kunming Peoples R China

ISBN: (纸本)9783642342882

Recent years, data-intensive computing has become a research hotspot. It also proposed a new challenge to traditional Bayesian inference methods. It is known that, traditional Bayesian inference methods could do a good job when all data stay in a single station. However, when come to data-intensive computing, it would hard to apply to new situation directly because data often been distributed in multi stations under data intensive computing. Among different Bayesian inference methods, random algorithm often been regarded as a common and effective one. And the sampling method adopted in random algorithm would largely influence the efficiency of this random algorithm. Gibbs sampling method often been used in random algorithm for Bayesian inference. Taking all of this into consideration, an improved Bayesian inference method for data-intensive computing is developed in this paper, which first use improved Gibbs sampling method in each station to gain the suitable information, then union them together to infer the final result. The validity of this method is discussed in theory and illustrated by experiment.

关键词： data-intensive computing Bayesian Inference Gibbs sampling

来源：评论

学校读者我要写书评

暂无评论

DM-IMCA: A dual-mode in-memory computing architecture for general purpose processing

引用

IEICE ELECTRONICS EXPRESS 2020年第4期17卷 20200005-20200005页

作者： Zeng, Jianmin Zhang, Zhang Chen, Runhao Liang, Shiyue Cao, Tianlin Yu, Zhiyi Cheng, Xin Xie, Guangjun Hefei Univ Technol Sch Elect Sci & Appl Phys Hefei 230009 Peoples R China Sun Yat Sen Univ Sch Elect & Informat Technol Guangzhou 510006 Peoples R China

In-Memory computing (IMC) architectures based on Static Random Access Memory (SRAM) can improve system performance and energy-efficiency dramatically. However, most of the existing SRAM-based implementations are designed for specific purposes like accelerating neural networks, which limits the application scenarios of IMC. In this paper, we propose DM-LMCA, a novel LAC architecture with two work modes for general purpose processing. It utilizes our proposed 9T bitcell based computational SRAM as the location to perform LAIC operations. Besides, a new IMC Instruction Set Architecture (ISA) as well as an automated vector computing mechanism are also proposed to facilitate DM-IMCA's programming and accelerate in-memory computing, respectively. The simulation results show that DM-LMCA can bring a performance increase by up to 257x, and SRAM energy saving by up to 3x, compared to a baseline system.

关键词： Processing-In-Memory (PIM) In-Memory computing (IMC) Logic-in-Memory (LiM) computational SRAM data-intensive computing vector computing

来源：评论

学校读者我要写书评

暂无评论

Building A Scalable Forward Flux Sampling Framework using Big data and HPC 19

Building A Scalable Forward Flux Sampling Framework using Bi...

引用

Conference on Practice and Experience in Advanced Research computing on Rise of the Machines (learning) (PEARC)

作者： DeFever, Ryan S. Hanger, Walter Sarupria, Sapna Kilgannon, Jon Apon, Amy W. Ngo, Linh B. Clemson Univ Chem & Biomol Engn Clemson SC 29634 USA West Chester Univ Comp Sci W Chester PA USA Clemson Univ Sch Comp Clemson SC USA

ISBN: (纸本)9781450372275

Forward flux sampling (FFS) is an established scientific method for sampling rare events in molecular simulations. However, as the difficulty of the scientific problem increases, the amount of data and the number of tasks required for FFS is challenging to manage with traditional scripting tools and languages for high performance computing. The SAFFIRE software framework has been developed to address these challenges. SAFFIRE utilizes Hadoop to manage a large number of tasks and data for large scale FFS simulations. The framework is shown to be highly scalable and able to support large scale FFS simulations. This enables studies of rare events in complex molecular systems on commodity cluster computing systems.

关键词： Forward Flux Sampling Hadoop data-intensive computing molecular simulations rare events

来源：评论

学校读者我要写书评

暂无评论

An Innovative and Reliable Water Leak Detection Service Supported by data-intensive Remote Sensing Processing 1

引用

19th Annual International Conference on Computational Science (ICCS)

作者： Martins, Ricardo Azevedo, Alberto Fortunato, Andre B. Alves, Elsa Oliveira, Anabela Carvalho, Alexandra LNEC Natl Lab Civil Engn Hydraul & Environm Dept Lisbon Portugal EDIA Empresa Desenvolvimento & Infraestruturas Al Beja Portugal

ISBN: (数字)9783030227470

ISBN: (纸本)9783030227470;9783030227463

The WADI project (Water-tightness Airborne Detection Implementation), integrated within the H2020 initiative, is developing an airborne water leak detection surveillance service, based on manned and unmanned aerial vehicles. This service aims to provide water utilities with adequate information on leaks in large water distribution infrastructures outside urban areas. Given the high cost associated with water infrastructure networks repairs, a reliability layer is necessary to improve the trustworthiness of the WADI leak identification procedure, based on complementary technologies for leak detection. Herein, a methodology based on the combined use of Sentinel remote sensing data and a water leak pathways model is presented, based on data-intensive computing. The resulting water leak detection reliability service, provided to the users through a web interface, targets prompt and cost-effective infrastructure repairs with the required degree of confidence on the detected leaks. The web platform allows for both data analysis and visualization of Sentinel images and relevant leak indicators at the sites selected by the user. The user can also provide aerial imagery inputs, to be processed together with Sentinel remote sensing data at the satellite acquisition dates identified by the user. The platform provides information about the detected leaks location and time evolution, and will be linked in the future with the outputs from water pathway models.

关键词： Remote sensing Water leak service data-intensive computing HPC

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：