检索结果-内蒙古大学图书馆

A HYBRID SCHEDULER FOR many task computing IN BIG DATA SYSTEMS

INTERNATIONAL JOURNAL OF APPLIED MATHEMATICS AND COMPUTER SCIENCE 2017年第2期27卷 385-399页

作者： Vasiliu, Laura Pop, Florin Negru, Catalin Mocanu, Mariana Cristea, Valentin Kolodziej, Joanna Univ Politehn Bucuresti Comp Sci Dept Fac Automat Control & Comp 313 Splaiul Independentei Bucharest 060042 Romania Cracow Univ Technol Inst Comp Sci Ul Warszawska 24 PL-31155 Krakow Poland Natl Inst Res & Dev Informat ICI 8-10 Maresal Averescu Bucharest 011455 Romania

With the rapid evolution of the distributed computing world in the last few years, the amount of data created and processed has fast increased to petabytes or even exabytes scale. Such huge data sets need data-intensive computing applications and impose performance requirements to the infrastructures that support them, such as high scalability, storage, fault tolerance but also efficient scheduling algorithms. This paper focuses on providing a hybrid scheduling algorithm for many task computing that addresses big data environments with few penalties, taking into consideration the deadlines and satisfying a data dependent task model. The hybrid solution consists of several heuristics and algorithms (min-min, min-max and earliest deadline first) combined in order to provide a scheduling algorithm that matches our problem. The experimental results are conducted by simulation and prove that the proposed hybrid algorithm behaves very well in terms of meeting deadlines.

关键词： many task computing scheduling heuristics QoS big data systems simulation

来源：评论

学校读者我要写书评

暂无评论

Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance computing 7

Colmena: Scalable Machine-Learning-Based Steering of Ensembl...

引用

7th IEEE/ACM Workshop on Machine Learning in High Performance computing Environments (MLHPC)

作者： Ward, Logan Sivaraman, Ganesh Pauloski, J. Gregory Babuji, Yadu Chard, Ryan Dandu, Naveen Redfern, Paul C. Assary, Rajeev S. Chard, Kyle Curtiss, Larry A. Thakur, Rajeev Foster, Ian Argonne Natl Lab Data Sci & Learning Div Lemont IL 60439 USA Univ Chicago Dept Comp Sci Chicago IL 60637 USA Univ Chicago Joint Ctr Energy Storage Res Chicago IL 60637 USA

ISBN: (纸本)9781665411240

Scientific applications that involve simulation ensembles can be accelerated greatly by using experiment design methods to select the best simulations to perform. Methods that use machine learning (ML) to create proxy models of simulations show particular promise for guiding ensembles but are challenging to deploy because of the need to coordinate dynamic mixes of simulation and learning tasks. We present Colmena, an open-source Python framework that allows users to steer campaigns by providing just the implementations of individual tasks plus the logic used to choose which tasks to execute when. Colmena handles task dispatch, results collation, ML model invocation, and ML model (re)training, using Parsl to execute tasks on HPC systems. We describe the design of Colmena and illustrate its capabilities by applying it to electrolyte design, where it both scales to 65 536 CPUs and accelerates the discovery rate for high-performance molecules by a factor of 100 over unguided searches.

关键词： Machine learning Computational Steering many task computing

来源：评论

学校读者我要写书评

暂无评论

ComputErl - Erlang-Based Framework for many task computing

<i>ComputErl</i> - Erlang-Based Framework for Many Task Comp...

引用

11th Symposium on Trends in Functional Programming

作者： Ptaszek, Michal Malawski, Maciej Inst Comp Sci AGH PL-30059 Krakow Poland

ISBN: (纸本)9783642229404

This paper shows how Erlang programming language can be used for creating a framework for distributing and coordinating the execution of many task computing problems. The goals of the proposed solution are (1) to disperse the computation into many tasks, (2) to support multiple well-known computation models (such as master-worker, map-reduce, pipeline), (3) to exploit the advantages of Erlang for developing an efficient and scalable framework and (4) to build a system that can scale from small to large number of tasks with minimum effort. We present the results of work on designing, implementing and testing CornputErl framework. The preliminary experiments with benchmarks as well as real scientific applications show promising scalability on a computing cluster.

关键词： many task computing Erlang grid distributed computing parallelism

来源：评论

学校读者我要写书评

暂无评论

A computational pipeline for the IUCN risk assessment for Meso-American reef ecosystem 13

A computational pipeline for the IUCN risk assessment for Me...

引用

13th Annual IEEE International Conference on e-Science (e-Science)

作者： Hoang Anh Nguyen Bland, Lucie Roberts, Tristan Guru, Siddeswara Dinh, Minh Abramson, David Univ Queensland Res Comp Ctr Brisbane Qld Australia Deakin Univ Sch Life & Environm Sci Melbourne Vic Australia Univ Queensland Terr Ecosyst Res Network Brisbane Qld Australia

ISBN: (纸本)9781538626863

Coral reefs are of global economic and biological significance but are subject to increasing threats. As a result, it is essential to understand the risk of coral reef ecosystem collapse and to develop assessment process for those ecosystems. The International Union for Conservation of Nature (IUCN) Red List of Ecosystem (RLE) is a framework to assess the vulnerability of an ecosystem. Importantly, the assessment processes need to be repeatable as new monitoring data arises. The repeatability will also enhance transparency. In this paper, we discuss the evolution of a computational pipeline for risk assessment of the Meso-American reef ecosystem, a diverse reef ecosystem located in the Caribbean, with the focus on improving the execution time starting from sequential and parallel implementation and finally using Apache Spark. The final form of the pipeline is a scientific workflow to improve its repeatability and reproducibility.

关键词： Ecosystem risk assessment Scientific Workflow many task computing Apache Spark

来源：评论

学校读者我要写书评

暂无评论

Efficient and Parallel Data Processing and Resource Allocation in the Cloud by using Nephele's Data Processing Framework

引用

INTERNATIONAL JOURNAL OF GRID AND DISTRIBUTED computing 2016年第3期9卷 33-40页

作者： Saranya, V. Ramya, S. Kumar, R. G. Suresh Nalini, T. Rajiv Gandhi Coll Engn & Technol Dept Comp Sci & Engn Technol Pondicherry India Vels Univ Madras Tamil Nadu India Bharath Univ Dept Comp Sci & Engn Madras Tamil Nadu India

Cloud computing is a technology in which the Cloud Service Providers (CSP) provide many virtual servers to the users to store their information in the cloud. The faults occurring on the assignment and dismission of the virtual machines, the processing cost in the allocation of resources must also be considered. The parallel processing of the information on the virtual machines must be done effectively and in an efficient manner. A variety of systems were developed to facilitate many task computing (MTC). These systems aim to hide the issues of parallelism and fault tolerant and they are used in many applications. In this paper, we introduced Nephele, a data processing framework to exploit dynamic resource provisioning offered by IaaS clouds. The performance evaluation of the virtual machines has been evaluated and the allocation and deallocation of job tasks to the specific virtual machines has also been considered. A performance comparison with the well known data processing framework hadoop has been done. Thus this paper tells about the effective and efficient manner of processing the data by parallel processing and allocating the correct resources for the desired task. It also helps to reduce the cost of resource utilization by exploiting the dynamic resource utilization.

关键词： CSP Allocation De-allocation many task computing Nephele Dynamic resource utilization Hadoop Parallel processing

来源：评论

学校读者我要写书评

暂无评论

POSTER: Pagoda: A Runtime System to Maximize GPU Utilization in Data Parallel tasks with Limited Parallelism 16

POSTER: Pagoda: A Runtime System to Maximize GPU Utilization...

引用

International Conference on Parallel Architectures and Compilation (PACT)

作者： Yeh, Tsung Tai Sabne, Amit Sakdhnagool, Putt Eigenmann, Rudolf Rogers, Timothy G. Purdue Univ Dept Elect & Comp Engn W Lafayette IN 47907 USA

ISBN: (纸本)9781450341219

Massively multithreaded GPUs achieve high throughput by running thousands of threads in parallel. To fully utilize the hardware, contemporary workloads spawn work to the GPU in bulk by launching large tasks, where each task is a kernel that contains thousands of threads that occupy the entire GPU. GPUs face severe underutilization and their performance benefits vanish if the tasks are narrow, i.e., they contain less than 512 threads. Latency-sensitive applications in network, signal, and image processing that generate a large number of tasks with relatively small inputs are examples of such limited parallelism. Recognizing the issue, CUDA now allows 32 simultaneous tasks on GPUs;however, that still leaves significant room for underutilization. This paper presents Pagoda, a runtime system that virtualizes GPU resources, using an OS-like daemon kernel called MasterKernel. tasks are spawned from the CPU onto Pagoda as they become available, and are scheduled by the MasterKernel at the warp granularity. This level of control enables the GPU to keep scheduling and executing tasks as long as free warps are found, dramatically reducing under utilization. Experimental results on real hardware demonstrate that Pagoda achieves a geometric mean speedup of 2.44x over PThreads running on a 20-core CPU, 1.43x over CUDA-HyperQ, and 1.33x over GeMTC, the state-of-the-art runtime GPU task scheduling system.

关键词： GPU scheduling many task computing

来源：评论

学校读者我要写书评

暂无评论

MOMTH: multi-objective scheduling algorithm of many tasks in Hadoop

引用

CLUSTER computing-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2015年第3期18卷 1011-1024页

作者： Nita, Mihaela-Catalina Pop, Florin Voicu, Cristiana Dobre, Ciprian Xhafa, Fatos Univ Politehn Bucuresti Fac Automat Control & Comp Dept Comp Sci Bucharest Romania Univ Politecn Cataluna Barcelona Spain

A real challenge sits in front of the business solutions these days, in the context of the big amount of data generated by complex software applications: efficiently using the given limited resources to accomplish specific operations and tasks. Depending on the type of application dealing with, when trying to deliver a certain service in a specific time and with a limited budget, a sequential application may be redesigned in a convenient way so that it will become scalable and able to run on multiple resources. many task computing model brings together loosely coupled applications, composed of many dependent/independent tasks, which will work together for a common result. When asking for a certain service, the most frequently constraints addressed by the user are deadline and budget. This paper elaborates on a multi-objective scheduling algorithm of many tasks in Hadoop for big data processing, named MOMTH. We consider objective functions related to users and resources in the same time with constraints like deadline (scheduling in due time) and budget. The algorithm evaluation was realized in scheduling load simulator, a tool integrated in Hadoop. MobiWay, a collaboration platform that expose interoperability between a large number of sensing mobile devices and a wide-range of mobility applications, was chosen for performance analysis of MOMTH. We compared the proposed algorithm with first in first out and fair schedulers and we obtained similar performance for our approach.

关键词： task scheduling Hadoop MapReduce many task computing Big data Cloud computing

来源：评论

学校读者我要写书评

暂无评论

Disk Cache-Aware task Scheduling For Data-Intensive and many-task Workflow 16

Disk Cache-Aware Task Scheduling For Data-Intensive and Many...

引用

16th IEEE International Conference on Cluster computing (CLUSTER)

作者： Tanaka, Masahiro Tatebe, Osamu Univ Tsukuba Ctr Computat Sci Tsukuba Ibaraki Japan Univ Tsukuba Fac Engn Informat Syst Tsukuba Ibaraki Japan

ISBN: (纸本)9781479955480

Workflow scheduling to maximize I/O performance is one of the key issues in data-intensive, many-task computing. In our previous work, we proposed locality-aware workflow scheduling method using the Multi-Constraint Graph Partitioning. In this work, we focus on read performance of input files from the disk cache (buffer cache or page cache on main memory). In order to maximize the disk cache hit rate of input files, a LIFO-order scheduling is effective since created intermediate files may be read soon. However, LIFO policy has a disadvantage of so-called "trailing task problem." We propose a hybrid scheduling strategy of LIFO and HRF (Highest Rank First). In our strategy, one of two policies is applied depending on the number of highest-rank tasks in the queue to avoid the problem. In addition, scheduling for the overlap of computation and I/O is proposed. We implement our scheduling strategy for the Pwrake workflow system and the Gfarm distributed file system and evaluate it by executing data-intensive workflows using a computer cluster. Our scheduling strategy improves the performance of copyfile workflow by 30% due to increase in disk cache hit rate, and the performance of Montage workflow by 12% due to increase in core utilization.

关键词： workflow system many task computing task scheduling distributed file system

来源：评论

学校读者我要写书评

暂无评论

Toward loosely coupled programming on petascale systems 08

Toward loosely coupled programming on petascale systems

引用

Proceedings of the 2008 ACM/IEEE conference on Supercomputing

作者： Ioan Raicu Zhao Zhang Mike Wilde Ian Foster Pete Beckman Kamil Iskra Ben Clifford University of Chicago Chicago IL University of Chicago and Argonne National Laboratory Chicago IL Argonne National Laboratory Argonne IL and University of Chicago and Argonne National Laboratory Chicago IL Argonne National Laboratory Argonne IL and University of Chicago Chicago IL and University of Chicago and Argonne National Laboratory Chicago IL Argonne National Laboratory Argonne IL

ISBN: (纸本)9781424428359

We have extended the Falkon lightweight task execution framework to make loosely coupled programming on petascale systems a practical and useful programming model. This work studies and measures the performance factors involved in applying this approach to enable the use of petascale systems by a broader user community, and with greater ease. Our work enables the execution of highly parallel computations composed of loosely coupled serial jobs with no modifications to the respective applications. This approach allows a new---and potentially far larger---class of applications to leverage petascale systems, such as the IBM Blue Gene/P supercomputer. We present the challenges of I/O performance encountered in making this model practical, and show results using both microbenchmarks and real applications from two domains: economic energy modeling and molecular dynamics. Our benchmarks show that we can scale up to 160K processor-cores with high efficiency, and can achieve sustained execution rates of thousands of tasks per second.

关键词： Falkon loosely coupled applications many task computing Swift high throughput computing Blue Gene petascale

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：