Many applications in scientific computing process very large sparse matrices on parallel architectures. The presented work in this paper is a part of a project where our general aim is to develop an auto-tuner system ...
详细信息
Many applications in scientific computing process very large sparse matrices on parallel architectures. The presented work in this paper is a part of a project where our general aim is to develop an auto-tuner system for the selection of the best matrix compression format in the context of high-performance computing. The target smart system can automatically select the best compression format for a given sparse matrix, a numerical method processing this matrix, a parallel programming model and a target architecture. Hence, this paper describes the design and implementation of the proposed concept. We consider a case study consisting of a numerical method reduced to the sparse matrix vector product (SpMV), some compression formats, the data parallel as a programming model and, a distributed multi-core platform as a target architecture. This study allows extracting a set of important novel metrics and parameters which are relative to the considered programming model. Our metrics are used as input to a machine-learning algorithm to predict the best matrix compression format. An experimental study targeting a distributed multi-core platform and processing random and real-world matrices shows that our system can improve in average up to 7% the accuracy of the machine learning.
This paper presents an open source toolkit allowing a rigorous distribution of stochastic simulations. It is designed according to the state of the art in pseudo-random numbers partitioning techniques. Based on a gene...
详细信息
This paper presents an open source toolkit allowing a rigorous distribution of stochastic simulations. It is designed according to the state of the art in pseudo-random numbers partitioning techniques. Based on a generic XML format for saving pseudo-random number generator states, each state contains adapted metadata. This toolkit named DistMe is usable by modelers who are non-specialists in parallelizing stochastic simulations, it helps in distributing the replications and in the generation of experimental plans. It automatically writes ready for runtime scripts for various parallel platforms, encapsulating the burden linked to the management of status files for different pseudo-random generators. The automation of this task avoids many human mistakes. The toolkit has been designed based on a model driven engineering approach: the user builds a model of its simulation and the toolkit helps in distributing independent stochastic experiments. In this paper, the toolkit architecture is exposed, and two examples in life science research domains are detailed. The preliminary design of the DistMe toolkit was achieved when dealing with the distribution of a nuclear medicine application using the largest European computinggrid: European grid Initiative (EGI). Thanks to our alpha version of the software toolbox, the equivalent of 3?years of computing was achieved in a few days. Next, we present the second application in another domain to show the potential and genericity of the DistMe toolkit. A small experimental plan with 1024 distributed stochastic experiments was run on a local computingcluster to explore scenarios of an environmental application. For both applications, the proposed toolkit was able to automatically generate distribution scripts with independent pseudo-random number streams, and it also automatically parameterized the simulation input files to follow an experimental design. The automatic generation of scripts and input files is achieved, thanks to mod
We present a Ruby-based batch language for Xgrid and its processor. Xgrid is an environment for distributed and parallel computing on the Mac OS X operating system, and Ruby is an object-oriented programming language ...
详细信息
ISBN:
(纸本)9780955301827
We present a Ruby-based batch language for Xgrid and its processor. Xgrid is an environment for distributed and parallel computing on the Mac OS X operating system, and Ruby is an object-oriented programming language for general purposes. In the standard Xgrid environment, jobs in batch les are statically de ned by an XML-based language, and submitted jobs are managed by their ID numbers. It is not easy for human to read and write XML-based batch les and to manage jobs by ID numbers. In our approach, jobs in batch les can be dynamically defined by a Ruby-based language, and submitted jobs can be managed by their logical names. Semantic checks and consistency managements are also done at submission in our approach. Our approach syntactically and semantically makes it easy to use Xgrid.
Broadcast is an important communication operation required by many real-world applications encountered in parallel, cluster, and gridcomputing environments. Broadcasting on regular networks has been widely investigat...
详细信息
Broadcast is an important communication operation required by many real-world applications encountered in parallel, cluster, and gridcomputing environments. Broadcasting on regular networks has been widely investigated in the past. However, most of the existing algorithms handle broadcast in a sequential manner and do not scale well;as a consequence, many applications cannot be efficiently supported using existing algorithms. In an effort to avoid this limitation, this article presents a new broadcast algorithm based on coded path routing. In addition to its simplicity, the proposed algorithm has shown to be capable of performing the broadcast operation in a fixed number of message-passing steps, irrespective of the network size. An extensive simulation study has been conducted to evaluate the performance of the proposed algorithm under different traffic working conditions. The analysis reveals that the new algorithm exhibits superior performance characteristics over those of the well-known recursive-doubling and extended-dominating node algorithms.
暂无评论