Irregular applications comprise a significant and increasing portion of jobs running in parallel environments. Recent research has shown that, in parallel environments, both the system utilization and application turn...
详细信息
Irregular applications comprise a significant and increasing portion of jobs running in parallel environments. Recent research has shown that, in parallel environments, both the system utilization and application turn around time improve when resources allocated to applications can be dynamically adjusted at run-time, depending on the workload. To realize this, at least some of the parallel applications in the system need to be dynamically reconfigurable. We have implemented the Distributed Resource Management System (DRMS) that supports the development and execution of regular and irregular reconfigurable applications in time-variant resource environments. In this paper, we discuss DRMS support for developing reconfigurable irregular applications and describe the dynamicdata redistribution mechanisms in some detail. We also present performance levels achieved by the data redistribution primitives, using a sparse Cholesky factorization algorithms as a model irregular application. Our results show that the cost of dynamicdata redistribution among different processor configurations for irregular data are comparable to those for regular data. (C) 1998 Academic Press.
In this paper, we propose a new approach to manage data storage and distribution in a data warehouse (DWH) environment. This approach deals with the dynamic data distribution of the DWH on a set of servers. ne data di...
详细信息
ISBN:
(纸本)9783540728290
In this paper, we propose a new approach to manage data storage and distribution in a data warehouse (DWH) environment. This approach deals with the dynamic data distribution of the DWH on a set of servers. ne datadistribution that we consider is different from the "classical" one which depends on the data use. The distribution in our approach consists in distributing data when the server reaches his storage capacity limit. This distribution assures the scalability and exploits the storage and processing resources available in the organization using the DWH. It is worth noting that our approach is based on a multi-agent model mixed with the scalability distribution proposed by the Scalable Distributed data Structures. The proposed multi-agent model is composed of stationary agent classes: Client, Dispatcher, Domain and Server, and a mobile agent class called Messenger. These agents collaborate and interact to achieve automatically the storage, the splitting (distribution), the redirection and the access operations on the distributed DWH.
We propose Chunks and Tasks, a parallel programming model built on abstractions for both data and work. The application programmer specifies how data and work can be split into smaller pieces, chunks and tasks, respec...
详细信息
We propose Chunks and Tasks, a parallel programming model built on abstractions for both data and work. The application programmer specifies how data and work can be split into smaller pieces, chunks and tasks, respectively. The Chunks and Tasks library maps the chunks and tasks to physical resources. In this way we seek to combine user friendliness with high performance. An application programmer can express a parallel algorithm using a few simple building blocks, defining data and work objects and their relationships. No explicit communication calls are needed;the distribution of both work and data is handled by the Chunks and Tasks library. This makes efficient implementation of complex applications that require dynamicdistribution of work and data easier. At the same time, Chunks and Tasks imposes restrictions on data access and task dependencies that facilitate the development of high performance parallel back ends. We discuss the fundamental abstractions underlying the programming model, as well as performance, determinism, and fault resilience considerations. We also present a pilot C++ library implementation for clusters of multicore machines and demonstrate its performance for irregular block-sparse matrix-matrix multiplication. (C) 2013 Elsevier B.V. All rights reserved.
Pore networks can be simulated in silico by using the dual site-bond Model. In this approach, a set of cavities (sites) are interconnected to each other by means of a set of throats (bonds), while considering that eac...
详细信息
Pore networks can be simulated in silico by using the dual site-bond Model. In this approach, a set of cavities (sites) are interconnected to each other by means of a set of throats (bonds), while considering that each site should be always larger than any of its delimiting bonds. The NoMISS greedy algorithm has been implemented recently in order to address this task;nevertheless, even if this procedure is relatively fast, there arises problems related to large memory consumption and long computing time, as pore networks become somewhat large. Here, three parallel methods are proposed to allow a proficient construction of large pore networks. The first method is a parallel Monte Carlo procedure, which applies a number of exchanges among pore sizes in order to obtain a valid pore network. The other two methods are parallel versions of the pioneering NoMISS greedy algorithm. The first version uses a static data partitioning to speed up the running time, whilst the second applies a dynamic data distribution policy to improve the pore network quality. The obtained results show the behavior of each proposed version with respect to their performance and quality, by employing the resources of a 125-core Linux cluster.
Porous media simulation is an important contribution in the study of many physical phenomena. The NoMISS greedy algorithm outstands from the existing sequential algorithms for constructing a pore subnetwork, in a rela...
详细信息
ISBN:
(纸本)9780769548463
Porous media simulation is an important contribution in the study of many physical phenomena. The NoMISS greedy algorithm outstands from the existing sequential algorithms for constructing a pore subnetwork, in a relatively fast way. However, despite the NoMISS time reduction, there are still problems related to the required processing time when very large networks need to be studied. In this work, a non scalable parallel version of the NoMISS algorithm is presented, and a new approach is proposed to alleviate this issue;in both versions cluster cores work simultaneously on different porous subnetwork spaces. The first approach, named as Unbounded-NoMISS, allows the cores to go forward with the initialization of the porous subnetwork space, applying a balancing policy when a core needs more data. At the end, the cores require a sequential synchronization to finish the porous network construction. The second approach, named as Bounded-NoMISS, controls the porous subnetwork initialization by considering a site-size boundary, avoiding the final strong synchronization and improving considerably the scalability. The obtained results using a 125-core cluster are presented.
data streams, as an important pattern of big data, require online real-time processing because instances arrive one by one and are fleeting. Existing online learning methods make distinctive assumptions, such as a fix...
详细信息
data streams, as an important pattern of big data, require online real-time processing because instances arrive one by one and are fleeting. Existing online learning methods make distinctive assumptions, such as a fixed feature space, a varying feature space that follows specific patterns, and a fixed datadistribution. However, data streams generated from real-world scenarios typically have both randomly changing feature spaces and datadistributions, making existing methods inappropriate for practical applications. To fill this gap, this study proposes a novel O nline L earning for data Streams with B i-dynamic D istributions (OLBD) algorithm. OLBD has a two-fold main idea: 1) it overcomes random changes in the feature space by building a mapping matrix to space transform and projects the original instances onto the global feature space;2) it handles dynamic data distributions by constraining prior knowledge and transferring established mapping relationships to new distributions. To evaluate OLBD, we compared it with related state-of-theart algorithms. First, we use 13 datasets to simulate three scenarios of dynamic feature space, namely trapezoidal, feature evolvable, and capricious data streams. Second, we simulated the data streams with dynamic data distributions using eight real and four generated datasets. We then conducted ablation studies on the parameter alpha. Finally, we analyzed data streams with bidynamic data distributions under different feature missing ratios and verified the generalization. The results show that OLBD significantly outperforms its rivals. Additionally, a practical case study on movie review classification was conducted to illustrate the effectiveness of OLBD in real-world scenarios.
暂无评论