Quantitative Finance is a field that replies on data analysis and big data enabling software to discover market signals. In this, a decisive factor is the speed that concerns execution speed and software development s...
详细信息
ISBN:
(纸本)9781538677575
Quantitative Finance is a field that replies on data analysis and big data enabling software to discover market signals. In this, a decisive factor is the speed that concerns execution speed and software development speed. So, an efficient software plays a key role in helping trading firms. Inspired by this, we present a novel software: QuantCloud to integrate a parallel python system with a C++-coded Big Data system. C++ is used to implement this big data system and python is used to code the user methods. The automated parallel execution of python codes is built upon a coprocess-based parallel strategy. We test our software using two popular algorithms: moving-window and autoregressive moving-average (ARMA). We conduct an extensive comparative study between Intel Xeon E5 and Xeon Phi processors. The results show that our method achieved a nearly linear speedup for executing python codes in parallel, prefect for today's multicore processors.
In this paper, the Numba, JAX, CuPy, PyTorch, and TensorFlow python GPU accelerated libraries were benchmarked using scientific numerical kernels on a NVIDIA V100 GPU. The benchmarks consisted of a simple Monte Carlo ...
详细信息
ISBN:
(纸本)9783031521850;9783031521867
In this paper, the Numba, JAX, CuPy, PyTorch, and TensorFlow python GPU accelerated libraries were benchmarked using scientific numerical kernels on a NVIDIA V100 GPU. The benchmarks consisted of a simple Monte Carlo estimation, a particle interaction kernel, a stencil evolution of an array, and tensor operations. The benchmarking procedure included general memory consumption measurements, a statistical analysis of scalability with problem size to determine the best libraries for the benchmarks, and a productivity measurement using source lines of code (SLOC) as a metric. It was statistically determined that the Numba library outperforms the rest on the Monte Carlo, particle interaction, and stencil benchmarks. The deep learning libraries show better performance on tensor operations. The SLOC count was similar for all the libraries except Numba which presented a higher SLOC count which implies more time is needed for code development.
Tools commonly leveraged to tackle large-scale data science workflows have traditionally shied away from existing high performance computing paradigms, largely due to their lack of fault tolerance and computation resi...
详细信息
Tools commonly leveraged to tackle large-scale data science workflows have traditionally shied away from existing high performance computing paradigms, largely due to their lack of fault tolerance and computation resiliency. However, these concerns are typically only of critical importance to problems tackled by technology companies at the highest level. For the average data scientist, the benefits of resiliency may not be as important as the overall execution performance. To this end, the work of this thesis aims to develop prototypes of tools favored by the data science community that function in a data-parallel environment, taking advantage of functionality commonly used in high performance computing. To achieve this goal, a prototype-distributed clone of the python NumPy library and a select module from the SciPy library were developed, which leverage MPI for inter-process communication and data transfers while abstracting away the complexity of MPI programming from its users. Through various benchmarks, the overhead introduced by logic necessary to resolve functioning in a data-parallel environment, as well as the scalability of using parallel compute resources for routines commonly used by the emulated libraries, are analyzed. For the distributed NumPy clone, it was found that for routines that could act solely on their local array contents, the impact of the introduced overhead was minimal; while for routines that required global scope of distributed elements, a considerable amount of overhead was introduced. In terms of scalability, both the distributed NumPy clone and select SciPy module, a distributed implementation of K-Means clustering, exhibited reasonably performant results; notably showing sensitivity to local process problem sizes and operations that required large amounts of collective communication/synchronization. As this work mainly focused on the initial exploration and prototyping of behavior, the results of the benchmarks can be used in future
Ambient intelligence (AmI) deals with a new world of ubiquitous computing devices, where physical environments interact intelligently and unobtrusively with people. AmI environments can be diverse, such as homes, offi...
详细信息
ISBN:
(纸本)9781509062317
Ambient intelligence (AmI) deals with a new world of ubiquitous computing devices, where physical environments interact intelligently and unobtrusively with people. AmI environments can be diverse, such as homes, offices, meeting rooms, schools, hospitals, control centers, vehicles, tourist attractions, stores, sports facilities, and music devices. In this paper, we present the design and implementation of a testbed for AmI using Raspberry Pi mounted on Raspbian OS. We analyze the performance of k-means clustering algorithm considering sensing data. For evaluation we considered respiratory rate and heart rate metrics. We speeded up the k-means clustering algorithm by using distributed concurrent processing.
Millions of tweets are generated each day on multifarious issues. Topical diversity in content demands domain-independent solutions for analysing twitter sentiments. Scalability is another issue when dealing with huge...
详细信息
Millions of tweets are generated each day on multifarious issues. Topical diversity in content demands domain-independent solutions for analysing twitter sentiments. Scalability is another issue when dealing with huge amount of tweets. This paper presents an unsupervised method for analysing tweet sentiments. Polarity of tweets is evaluated by using three sentiment lexicons-SenticNet, SentiWordNet and SentislangNet. SentislangNet is a sentiment lexicon built from SenticNet and SentiWordNet for slangs and acronyms. Experimental results show fairly good -score. The method is implemented and tested in parallel python framework and is shown to scale well with large volume of data on multiple cores.
High performance computing has been used in various fields of astrophysical research. But most of it is implemented on massively parallel systems (supercomputers) or graphical processing unit clusters. With the advent...
详细信息
High performance computing has been used in various fields of astrophysical research. But most of it is implemented on massively parallel systems (supercomputers) or graphical processing unit clusters. With the advent of multicore processors in the last decade, many serial software codes have been re-implemented in parallel mode to utilize the full potential of these processors. In this paper, we propose parallel processing recipes for multicore machines for astronomical data processing. The target audience is astronomers who use python as their preferred scripting language and who may be using PyRAF/IRAF for data processing. Three problems of varied complexity were benchmarked on three different types of multicore processors to demonstrate the benefits, in terms of execution time, of parallelizing data processing tasks. The native multiprocessing module available in python makes it a relatively trivial task to implement the parallel code. We have also compared the three multiprocessing approaches-Pool/Map, Process/Queue and parallel python. Our test codes are freely available and can be downloaded from our website. (c) 2013 Elsevier B.V. All rights reserved.
MPI for python provides bindings of the message passing interface (MPI) standard for the python programming language and allows any python program to exploit multiple processors. In its first release, NIPI for python ...
详细信息
MPI for python provides bindings of the message passing interface (MPI) standard for the python programming language and allows any python program to exploit multiple processors. In its first release, NIPI for python was constructed on top of the MPI-1 specification defining an object-oriented interface that closely followed the MPI-2 C + + bindings, and provided support for communications of general python objects. In the latest release, this package is improved to enable direct blocking/non-blocking communication of numeric arrays, and to support almost all MPI-2 features. Improvements in communication performance have been tested in a Beowulf class cluster. Results showed a negligible overhead in comparison to compiled C code. MPI for python is open source and available for download on the web (http://***/). (C) 2007 Elsevier Inc. All rights reserved.
MPI for python provides bindings of the Message Passing Interface (MPI) standard for the python progamming language and allows any python program to exploit multiple processors. This package is constructed on top of t...
详细信息
MPI for python provides bindings of the Message Passing Interface (MPI) standard for the python progamming language and allows any python program to exploit multiple processors. This package is constructed on top of the MPI-1 specification and defines an object-oriented interface which closely follows MPI-2 C++ bindings. It supports point-to-point (sends, receives) and collective (broadcasts, scatters, gathers) communications of general python objects. Efficiency has been tested in a Beowulf class cluster and satisfying results were obtained. MPI for python is open source and available for download on the web (http: / /www. cimec. org. ar/python). (c) 2005 Elsevier Inc. All rights reserved.
暂无评论