the proceedings contain 6 papers. the topics discussed include: python, performance and natural language processing;performance and productivity of parallel python programming - a study with a CFD test case;PeachPy me...
ISBN:
(纸本)9781450340106
the proceedings contain 6 papers. the topics discussed include: python, performance and natural language processing;performance and productivity of parallel python programming - a study with a CFD test case;PeachPy meets Opcodes: direct machine code generation from python;PyTACC: HPC python at the Texas advanced computing center;feature frequency profiles for automatic sample identification using PySpark;dispel4py: a python framework for data-intensive eScience;and solving large numerical optimization problems in HPC withpython.
python has been gaining some traction for years in the world of scientific applications. However, the high-level abstraction it provides may not allow the developer to use the machines to their peak performance. To ad...
详细信息
ISBN:
(纸本)9780738110868
python has been gaining some traction for years in the world of scientific applications. However, the high-level abstraction it provides may not allow the developer to use the machines to their peak performance. To address this, multiple strategies, sometimes complementary, have been developed to enrich the software ecosystem either by relying on additional libraries dedicated to efficient computation (e.g., NumPy) or by providing a framework to better use HPC scale infrastructures (e.g., PyCOMPSs). In this paper, we present a python extension based on SharedArray that enables the support of system-provided shared memory and its integration into the PyCOMPSs programming model as an example of integration to a complex python environment. We also evaluate the impact such a tool may have on performance in two types of distributed execution-flows, one for linear algebra with a blocked matrix multiplication application and the other in the context of data-clustering with a k-means application. We show that with very little modification of the original decorator (3 lines of code to be modified) of the task-based application the gain in performance can rise above 40% for tasks relying heavily on data reuse on a distributed environment, especially when loading the data is prominent in the execution time.
python has been adopted as programming language by a large number of scientific communities. Additionally to the easy programming interface, the large number of libraries and modules that have been made available by a...
详细信息
ISBN:
(纸本)9781450351249
python has been adopted as programming language by a large number of scientific communities. Additionally to the easy programming interface, the large number of libraries and modules that have been made available by a large number of contributors, have taken this language to the top of the list of the most popular programming languages in scientific applications. However, one main drawback of python is the lack of support for concurrency or parallelism. PyCOMPSs is a proved approach to support task-based parallelism in pythonthat enables applications to be executed in parallel in distributed computing platforms. this paper presents PyCOMPSs and how it has been tailored to execute tasks in heterogeneous and multi-threaded environments. We present an approach to combine the task-level parallelism provided by PyCOMPSs withthe thread-level parallelism provided by MKL. performance and behavioral results in distributed computing heterogeneous clusters show the benefits and capabilities of PyCOMPSs in both HPC and Big Data infrastructures.
high level abstractions in pythonthat can utilize computing hardware well seem to be an attractive option for writing data reduction and analysis tasks. In this paper, we explore the features available in python whic...
详细信息
ISBN:
(纸本)9781450351249
high level abstractions in pythonthat can utilize computing hardware well seem to be an attractive option for writing data reduction and analysis tasks. In this paper, we explore the features available in python which are useful and efficient for end user analysis in high Energy Physics (HEP). A typical vertical slice of an HEP data analysis is somewhat fragmented: the state of the reduction/analysis process must be saved at certain stages to allow for selective reprocessing of only parts of a generally time-consuming workflow. Also, algorithms tend to to be modular because of the heterogeneous nature of most detectors and the need to analyze different parts of the detector separately before combining the information. this fragmentation causes difficulties for interactive data analysis, and as data sets increase in size and complexity (O10 TiB for a "small" neutrino experiment to the O10 PiB currently held by the CMS experiment at the LHC), data analysis methods traditional to the field must evolve to make optimum use of emerging HPC technologies and platforms. Mainstream big data tools, while suggesting a direction in terms of what can be done if an entire data set can be available across a system and analysed withhigh-level programming abstractions, are not designed with either scientificcomputing generally, or modern HPC platform features in particular, such as data caching levels, in mind. Our example HPC use case is a search for a new elementary particle which might explain the phenomenon known as "Dark Matter". Using data from the CMS detector, we will use HDF5 as our input data format, and MPI withpython to implement our use case.
We propose a method of accelerating python code by just-in-time compilation leveraging type hints mechanism introduced in python 3.5. In our approach performance-critical kernels are expected to be written as if Pytho...
详细信息
Domain specific languages (DSL) have been used in a variety of fields to express complex scientific problems in a concise manner and provide automated performance optimization for a range of computational architecture...
详细信息
We propose a method of accelerating python code by just-in-time compilation leveraging type hints mechanism introduced in python 3.5. In our approach performance-critical kernels are expected to be written as if Pytho...
详细信息
ISBN:
(纸本)9781509052219
We propose a method of accelerating python code by just-in-time compilation leveraging type hints mechanism introduced in python 3.5. In our approach performance-critical kernels are expected to be written as if python was a strictly typed language, however without the need to extend python syntax. this approach can be applied to any python application, however we focus on a special case when legacy Fortran applications are automatically translated into python for easier maintenance. We developed a framework implementing two-way transpilation and achieved performance equivalent to that of python manually translated to Fortran, and better than using other currently available JIT alternatives (up to 5x times faster than Numba in some experiments).
python-based applications at the Texas Advanced computing Center (TACC) consume a significant and growing fraction of our computational resources. To meet this demand, TACC has developed an approach to provide its use...
详细信息
Numerical optimization is a complex problem in which many different algorithms can be used. Distributed metaheuristics have received attention but they normally focus on small problems. Many large scientific problems ...
详细信息
We present a case study of python-based workflow for a data-intensive natural language processing problem, namely word classification with vector space model methodology. Problems in the area of natural language proce...
详细信息
ISBN:
(纸本)9781450340106
We present a case study of python-based workflow for a data-intensive natural language processing problem, namely word classification with vector space model methodology. Problems in the area of natural language processing are typically solved in many steps which require transformation of the data to vastly different formats (in our case, raw text to sparse matrices to dense vectors). A python implementation for each of these steps would require a different solution. We survey existing approaches to using python for highperformance processing of large volumes of data, and we propose a sample solution for each step for our case study (aspectual classification of Russian verbs), attempting to preserve both efficiency and user-friendliness. For the most computationally intensive part of the workflow we develop a prototype distributed implementation of co-occurrence extraction module using *** cluster. Copyright is held by the owner/author(s).
暂无评论