Today the gridcomputing, intended initially for the intensive computing, open towards the management of voluminous, heterogeneous, and distributed data on a large-scale environment. The grid data management raises ne...
详细信息
ISBN:
(纸本)9781424417513
Today the gridcomputing, intended initially for the intensive computing, open towards the management of voluminous, heterogeneous, and distributed data on a large-scale environment. The grid data management raises new problems and presents real challenges: resource discovery, efficiency of access, autonomic management, security, and benchmarking. This importance comes out of characteristics offered by grid systems: autonomy, heterogeneity and dynamicity of nodes Firstly, we recall the fundamental problems of the large scale data management in grid systems and characteristics of these systems. Then, we describe in a highlight way proposed approaches (Web services, P2P techniques, Agent-based approach) for resource discovery. The remainder of the paper is devoted to point out the contributions of mobile agents for some problems of large scale data management, in particularly: dynamic query optimization, task placement, and embedded cost model. We show how mobile agents can help for decentralized control, and scaling.
The Australian Government is making a $AUD 100 million investment in Compute and Storage for the academic community. The Compute facilities are provided in the form of 30,000 CPU cores located at 8 nodes around Austra...
详细信息
The Australian Government is making a $AUD 100 million investment in Compute and Storage for the academic community. The Compute facilities are provided in the form of 30,000 CPU cores located at 8 nodes around Australia in a distributed virtualized Infrastructure as a Service facility based on Open Stack. The storage will eventually consist of over 100 petabytes located at 6 nodes. All will be linked via a 100 Gb/s network. This proceeding describes the development of a fully connected WLCG Tier-2 grid site as well as a general purpose Tier-3 computing cluster based on this architecture. The facility employs an extension to Torque to enable dynamic allocations of virtual machine instances. A base Scientific Linux virtual machine (VM) image is deployed in the Open Stack cloud and automatically configured as required using Puppet. Custom scripts are used to launch multiple VMs, integrate them into the dynamic Torque cluster and to mount remote file systems. We report on our experience in developing this nation-wide ATLAS and Belle II Tier 2 and Tier 3 computing infrastructure using the national Research Cloud and storage facilities.
In the emerging field of multi-physics simulations, we often face the challenge to establish new connections between physical fields, to add additional aspects to existing models, or to exchange a solver for one of th...
详细信息
In the emerging field of multi-physics simulations, we often face the challenge to establish new connections between physical fields, to add additional aspects to existing models, or to exchange a solver for one of the involved physical fields. If in such cases a fast prototyping of a coupled simulation environment is required, a partitioned setup using existing codes for each physical field is the optimal choice. As accurate models require also accurate numerics, multi-physics simulations typically use very high grid resolutions and, accordingly, are run on massively parallel computers. Here, we face the challenge to combine flexibility with parallel scalability and hardware efficiency. In this paper, we present the coupling tool preCICE which offers the complete coupling functionality required for a fast development of a multi-physics environment using existing, possibly black-box solvers. We hereby restrict ourselves to bidirectional surface coupling which is too expensive to be done via file communication, but in contrast to volume coupling still a candidate for distributed memory parallelism between the involved solvers. The paper gives an overview of the numerical functionalities implemented in preCICE as well as the user interfaces, i.e., the application programming interface and configuration options. Our numerical examples and the list of different open-source and commercial codes that have already been used with preCICE in coupled simulations Show the high flexibility, the correctness, and the high performance and parallel scalability of coupled simulations with preCICE as the coupling unit. (C) 2016 Elsevier Ltd. All rights reserved.
Tensor completion is a powerful tool used to estimate or recover missing values in multi-way data. It has seen great success in domains such as product recommendation and healthcare. Tensor completion is most often ac...
详细信息
ISBN:
(纸本)9781467388153
Tensor completion is a powerful tool used to estimate or recover missing values in multi-way data. It has seen great success in domains such as product recommendation and healthcare. Tensor completion is most often accomplished via low-rank sparse tensor factorization, a computationally expensive non-convex optimization problem which has only recently been studied in the context of parallelcomputing. In this work, we study three optimization algorithms that have been successfully applied to tensor completion: alternating least squares (ALS), stochastic gradient descent (SGD), and coordinate descent (CCD++). We explore opportunities for parallelism on shared-and distributed-memory systems and address challenges such as memory-and operation-efficiency, load balance, cache locality, and communication. Among our advancements are an SGD algorithm which combines stratification with asynchronous communication, an ALS algorithm rich in level-3 BLAS routines, and a communication-efficient CCD++ algorithm. We evaluate our optimizations on a variety of real datasets using a modern supercomputer and demonstrate speedups through 1024 cores. These improvements effectively reduce time-to-solution from hours to seconds on real-world datasets. We show that after our optimizations, ALS is advantageous on parallel systems of small-to-moderate scale, while both ALS and CCD++ will provide the lowest time-to-solution on large-scale distributed systems.
The recent development in the field of power grid has observed that the affect of distributed energy resourcefulness's, electric vehicles, plug-in hybrid electric vehicles, and smart gadgets are favourable to the ...
详细信息
ISBN:
(纸本)9781728185293
The recent development in the field of power grid has observed that the affect of distributed energy resourcefulness's, electric vehicles, plug-in hybrid electric vehicles, and smart gadgets are favourable to the environment. It is economical. And also it is reliable. The needs can be enhanced by executing coordinated smart controls. Coordinated control's absence may have some negative effects, such as reduced lifelong service of power distribution components and in distribution transformers. This paper represents a smart houses with the application of energy management system & smart grid that permits organized control for a house resourcefulness's without consumer bother while minimizes overloading and overheating of the distributed substructure. Paper also aims an architectural structure and an operational model for house Energy Management System (EMS), a structure in a house that helps household inhabit in working house equipment to gain optimal energy utilization. The architectural system also looks a functional relation between parts and the whole system of the structure and brings into the notice with the smart grid which are the Distribution System Operators (DSOs) and Energy Service Providers (ESPs).
This paper reviews work, on distributed collaborative virtual design environments and the emerging technology of Metacomputing. The paper discusses the design of distributed collaborative design environments using Met...
详细信息
ISBN:
(纸本)0769511953
This paper reviews work, on distributed collaborative virtual design environments and the emerging technology of Metacomputing. The paper discusses the design of distributed collaborative design environments using Meta computing, for example Globus, to provide intensive data computing via air asynchronous collection of loosely synchronous components. The essential component, Globus, focuses on defining a toolkit of low-level services for security, communication, resource location, resource allocation, process management, and data access. Based on this approach, the requirements for concept, functionality and performance of the design environments are provided. The key functions in the distributed collaborative design environments such as analysis, visualisation, security and database management, particularly the 3D design storage and visualisation, are also discussed. At the end of the paper further work Oil implementing the Globus-based distributed collaborative design environments is highlighted.
The next generation resource monitoring system in distributedcomputing environments should not only accurately monitor the environment but also automatically reconfigure the environment. In this paper, we present INA...
详细信息
gridcomputing goes far beyond capabilities of traditional distributedcomputing, which combines scalable large databases, high network capacity, and pervasive elevated computational resources into a single pool of re...
详细信息
We develop a modeling and simulation framework capable of massively parallel simulation of multicellular systems with spatially resolved stochastic kinetics in individual cells. By the use of operator-splitting we dec...
详细信息
ISBN:
(纸本)9781538691564
We develop a modeling and simulation framework capable of massively parallel simulation of multicellular systems with spatially resolved stochastic kinetics in individual cells. By the use of operator-splitting we decouple the simulation of reaction-diffusion kinetics inside the cells from the simulation of molecular cell-cell interactions occurring on the boundaries between cells. This decoupling leverages the inherent scale separation in the underlying model to enable highly horizontally scalable parallel simulation, suitable for simulation on heterogeneous, distributedcomputing infrastructures such as public and private clouds. Thanks to its modular structure, our frameworks makes it possible to couple just any existing single-cell simulation software together with any cell signaling simulator. We exemplify the flexibility and scalability of the framework by using the popular single-cell simulation software eGFRD to construct and simulate a multicellular model of Notch-Delta signaling over OpenStack cloud infrastructure provided by the SNIC Science Cloud.
Vertex-centric graph processing systems such as Pregel, PowerGraph, or GraphX recently gained popularity due to their superior performance of data analytics on graph-structured data. These systems exploit the graph st...
详细信息
ISBN:
(纸本)9781509014828
Vertex-centric graph processing systems such as Pregel, PowerGraph, or GraphX recently gained popularity due to their superior performance of data analytics on graph-structured data. These systems exploit the graph structure to improve data access locality during computation, making use of specialized graph partitioning algorithms. Recent partitioning techniques assume a uniform and constant amount of data exchanged between graph vertices (i.e., uniform vertex traffic) and homogeneous underlying network costs. However, in real-world scenarios vertex traffic and network costs are heterogeneous. This leads to suboptimal partitioning decisions and inefficient graph processing. To this end, we designed GrapH, the first graph processing system using vertex-cut graph partitioning that considers both, diverse vertex traffic and heterogeneous network, to minimize overall communication costs. The main idea is to avoid frequent communication over expensive network links using an adaptive edge migration strategy. Our evaluations show an improvement of 60% in communication costs compared to state-of-the-art partitioning approaches.
暂无评论