The dynamic provisioning of virtual machines (VMs) supported by many cloud computing infrastructures eases the scalability of software applications. Unfortunately, VMs are relatively slow to boot and public cloud prov...
详细信息
ISBN:
(纸本)9783030105495;9783030105488
The dynamic provisioning of virtual machines (VMs) supported by many cloud computing infrastructures eases the scalability of software applications. Unfortunately, VMs are relatively slow to boot and public cloud providers do not allow users to vary their resources (vertical scalability) dynamically. To tackle both problems, a few years ago we presented a solution that combines the management of VMs with the use of containers specifically targeted to the efficient runtime management of the resources provisioned to Web applications. This paper borrows from this solution and addresses the problem of provisioning resources to big data, Spark applications at runtime. Spark does not allow for the runtime scalability of the resources associated with its executors, but resources must be provisioned statically. To tackle this problem, the paper describes a container-based version of Spark that supports the dynamic resizing of the memory and CPU cores associated with the different executors. The evaluation demonstrates the feasibility of the approach and identifies the trade-offs involved.
With the advent of Internet-of-Things (IoT) devices, including smart meters and sensors in the smart grid, there has been immense research interest in big data management, analytics, and parallelprocessing of data. H...
详细信息
ISBN:
(纸本)9781728143811
With the advent of Internet-of-Things (IoT) devices, including smart meters and sensors in the smart grid, there has been immense research interest in big data management, analytics, and parallelprocessing of data. However, complex hardware and software parameters configurations and in-depth understanding of the data processing design are essential for efficient utilization of big data analytics platforms. In this work, we analyze the parallelization of load prediction by utilizing spark regression python library to assess the performance with workloads of up to 8 nodes. The results of different configurations have been studied and analyzed against the performance of Apache Spark. It was found that a trade-off between the number of nodes and cores is necessary to perform efficient parallel computing. Multiple sets of combinations of nodes and cores are considered in this paper to evaluate the performance. The work also signifies the importance of high-performance computing capability for smart meters big data management. The obtained results indicate that the computational time is not only dependent on the data size but also on the number of compute nodes and the number of cores assigned to run the job.
The proceedings contain 59 papers. The special focus in this conference is on parallel Computing in Regular Structures. The topics include: Analytical modeling of parallel application in heterogeneous computing enviro...
ISBN:
(纸本)3540663630
The proceedings contain 59 papers. The special focus in this conference is on parallel Computing in Regular Structures. The topics include: Analytical modeling of parallel application in heterogeneous computing environments;skeletons and transformations in an integrated parallel programming environment;sequential unification and aggressive lookahead mechanisms for data memory accesses;a coordination model and facilities for efficient parallel computation;parallelizing of sequential programs on the basis of pipeline and speculative features of the operators;kinetic model of parallel data processing;PSA approach to population models for parallel genetic algorithms;highly accurate numerical methods for incompressible 3D fluid flows on parallel architectures;dynamic task scheduling with precedence constraints and communication delays;two-dimensional scheduling of algorithms with uniform dependencies;consistent lamport clocks for asynchronous groups with process crashes;comparative analysis of learning methods of cellular-neural associative memory;emergence and propagation of round autowave in cellular neural network;routing and embeddings in super cayley graphs;implementing cellular automata based models on parallel architectures;overview, design innovations, and preliminary results;implementing model checking and equivalence checking for time petri nets by the RT-MEC tool;learning concurrent programming;the speedup performance of an associative memory based logic simulator;a high-level programming environment for distributed memory architectures;virtual shared files;an object oriented environment to manage the parallelism of the FIIT applications;performance studies of shared-nothing parallel transaction processing systems;synergetic tool environments and logically instantaneous communication on top of distributed memory parallel machines.
In this article we present CORBA Lightweight Components, CORBA–LC, a new network-centered reflective component model which allows building distributedapplications assembling binary independent components spread on t...
详细信息
Software pipelining for nested loops remains a challenging problem for embedded system design. The existing software pipelining techniques for single loops can only explore the parallelism of the innermost loop, so th...
详细信息
Software pipelining for nested loops remains a challenging problem for embedded system design. The existing software pipelining techniques for single loops can only explore the parallelism of the innermost loop, so the final timing performance is inferior. While multi-dimensional (MD) retiming can explore the outer loop parallelism, it introduces large overheads in loop index generation and code size due to transformation. In this paper, we use MD retiming to model the software pipelining problem of nested loops. We show that the computation time and code size of a software-pipelined loop nest is affected by execution sequence and retiming function. The algorithm of Software Pipelining for NEsted loops technique (SPINE) is proposed to generate fully parallelized loops efficiently with the overheads as small as possible. The experimental results show that our technique outperforms both the standard software pipelining and MD retiming significantly.
We present AMP, a novel service architecture for countering distributed denial of service (dDos) attacks. AMP uses dynamically configured network components to perform traffic monitoring, filtering and detection of co...
详细信息
The Message Passing Interface (MPI) specifies a one-sided interface for Remote Memory Access (RMA), which allows one process to specify all communication parameters for both the sending and receiving side by providing...
详细信息
ISBN:
(纸本)9783319436593;9783319436586
The Message Passing Interface (MPI) specifies a one-sided interface for Remote Memory Access (RMA), which allows one process to specify all communication parameters for both the sending and receiving side by providing support for asynchronous reads and updates of distributed shared data. While MPI RMA communication can be highly efficient, proper synchronization of possibly conflicting accesses to shared data is a challenging task. This paper presents a novel debugging tool that supports developers in finding latent synchronization errors. It dynamically intercepts RMA calls and reschedules them into pessimistic executions which are valid in terms of the MPI-3 standard. Given an application with a latent synchronization error, we force a manifestation of this error which can easily be detected with the help of program invariants. An experimental evaluation shows that the tool can uncover synchronization errors which would otherwise likely go unnoticed for a wide range of scenarios.
Binary addition is a commonly used application in computational arithmetic. Adders are the basic building blocks of the various computational structures leading to wide applications in Digital Signal processing, arith...
详细信息
Operator-based programming languages provide an effective development model for large scale stream processingapplications. A stream processing application consists of many runtime deployable software processing eleme...
详细信息
Software testing is an important process to evaluate whether the developed software applications meet the required specifications. There is an emerging need for testing frameworks for big data software projects to ens...
详细信息
ISBN:
(纸本)9781665439022
Software testing is an important process to evaluate whether the developed software applications meet the required specifications. There is an emerging need for testing frameworks for big data software projects to ensure the quality of the big data applications and satisfy the user requirements. In this study, we propose a software testing framework that can be utilized in big data projects both in e-science and e-commerce. In particular, we design the proposed framework to test big data-based recommendation applications. To show the usability of the proposed framework, we provide a reference prototype implementation and use the prototype to test a big data recommendation application. We apply the prototype implementation to test both functional and non-functional methods of the recommendation application. The results indicate that the proposed testing framework is usable and efficient for testing the recommendation systems that use big data processingtechniques.
暂无评论