Nowadays, there are many real-time spatial applications like location-aware services and traffic monitoring and the need for real time spatial data processing becomes more and more important. As a result, there is a t...
详细信息
ISBN:
(纸本)9783030602390;9783030602383
Nowadays, there are many real-time spatial applications like location-aware services and traffic monitoring and the need for real time spatial data processing becomes more and more important. As a result, there is a tremendous amount of real-time spatial data in real-time spatial data warehouse. the continuous growth in the amount of data seems to outspeed the advance of the traditional centralized real-time spatial data warehouse. As a solution, many organizations use distributed real-time spatial data warehouse (DRTSDW) as a powerful technique to achieve OLAP (On Line Analytical processing) analysis and business intelligence (BI). Distributing data in real time data warehouse is divided into two steps: partitioning data and their allocation into sites. Several works have proposed many algorithms for partitioning and allocation data. But with a huge amount of real-time spatial data generated, the system performance degrades rapidly, especially in overload situations. In order to deal withthis volumetry and to increase query efficiency, we propose a novel approach for partitioning data in real-time spatial data warehouse to find the right number of clusters and to divides the RTSDW into partitions using the horizontal partitioning. Secondly, we suggest our allocation strategy to place the partitions on the sites where they are most used, to minimize data transfers between sites. We have evaluated those proposed approaches using the new TPC-DS (Transaction processing performance council, http://***, 2014) benchmark. the preliminary results show that the approach is quite interesting.
Most of cryptographic systems are based on modular exponentiation. It is performed using successive modular multiplications. One way of improving the throughput of a cryptographic system implementation is reducing the...
详细信息
Deep neural network training is a common issue that is receiving increasing attention in recent years and basically performed on Stochastic Gradient Descent or its variants. Distributed training increases training spe...
详细信息
ISBN:
(纸本)9783030389611;9783030389604
Deep neural network training is a common issue that is receiving increasing attention in recent years and basically performed on Stochastic Gradient Descent or its variants. Distributed training increases training speed significantly but causes precision loss at the mean time. Increasing batchsize can improve training parallelism in distributed training. However, if the batchsize is too large, it will bring difficulty to training process and introduce more training error. In this paper, we consider controlling the total batchsize and lowering batchsize on each GPU by increasing the number of GPUs in distributed training. We train Resnet50 [4] on CIFAR-10 dataset by different optimizers, such as SGD, Adam and NAG. the experimental results show that large batchsize speeds up convergence to some degree. However, if the batchsize of per GPU is too small, training process fails to converge. Large number of GPUs, which means a small batchsize on each GPU declines the training performance in distributed training. We tried several ways to reduce the training error on multiple GPUs. According to our results, increasing momentum is a well-behaved method in distributed training to improve training performance on condition of multiple GPUs of constant large batchsize.
In the field of signal process, Fast Fourier Transform (FFT) is a widely used algorithm to transform signal data from time to frequency. Unfortunately, withthe exponential growth of data, traditional methods cannot m...
详细信息
ISBN:
(纸本)9783030050573;9783030050566
In the field of signal process, Fast Fourier Transform (FFT) is a widely used algorithm to transform signal data from time to frequency. Unfortunately, withthe exponential growth of data, traditional methods cannot meet the demand of large-scale computation on these big data because of three main challenges of large-scale FFT, i.e., big data size, real-time data processing and high utilization of compute resources. To satisfy these requirements, an optimized FFT algorithm in Cloud is deadly needed. In this paper, we introduce a new method to conduct FFT in Cloud withthe following contributions: first, we design a parallel FFT algorithm for large-scaled signal data in Cloud;second, we propose a MapReduce-based mechanism to distribute data to compute nodes using big data processing framework;third, an optimal method of distributing compute resources is implemented to accelerate the algorithm by avoiding redundant data exchange between compute nodes. the algorithm is designed in MapReduce computation framework which contains three steps: data preprocessing, local data transform and parallel data transform to integrate processing results. the parallel FFT is implemented in a 16-node Cloud to process real signal data the experimental results reveal an obvious improvement in the algorithm speed. Our parallel FFT is approximately five times faster than FFT in Matlab in when the data size reaches 10 GB.
A novel MapReduce computation model in hybrid computing environment called HybridMR is proposed in the paper. Using this model, high performance cluster nodes and heterogeneous desktop PCs in Internet or Intranet can ...
详细信息
ISBN:
(数字)9783319111940
ISBN:
(纸本)9783319111940;9783319111933
A novel MapReduce computation model in hybrid computing environment called HybridMR is proposed in the paper. Using this model, high performance cluster nodes and heterogeneous desktop PCs in Internet or Intranet can be integrated to form a hybrid computing environment. In this way, the computation and storage capability of large-scale desktop PCs can be fully utilized to process large-scale datasets. HybridMR relies on a hybrid distributed file system called HybridDFS, and a time-out method has been used in HybridDFS to prevent volatility of desktop PCs, and file replication mechanism is used to realize reliable storage. A new node priority-based fair scheduling (NPBFS) algorithm has been developed in HybridMR to achieve both data storage balance and job assignment balance by assigning each node a priority through quantifying CPU speed, memory size and I/O bandwidth. Performance evaluation results show that the proposed hybrid computation model not only achieves reliable MapReduce computation, reduces task response time and improves the performance of MapReduce, but also reduces the computation cost and achieves a greener computing mode.
this two volume set LNCS 8285 and 8286 constitutes the proceedings of the 13thinternationalconference on algorithms and architectures for parallelprocessing , ica3pp 2013, held in Vietri sul Mare, Italy in December...
详细信息
ISBN:
(数字)9783319038896
ISBN:
(纸本)9783319038889
this two volume set LNCS 8285 and 8286 constitutes the proceedings of the 13thinternationalconference on algorithms and architectures for parallelprocessing , ica3pp 2013, held in Vietri sul Mare, Italy in December 2013. the first volume contains 10 distinguished and 31 regular papers selected from 90 submissions and covering topics such as big data, multi-core programming and software tools, distributed scheduling and load balancing, high-performance scientific computing, parallelalgorithms, parallelarchitectures, scalable and distributed databases, dependability in distributed and parallel systems, wireless and mobile computing. the second volume consists of four sections including 35 papers from one symposium and three workshops held in conjunction withica3pp 2013 main conference. these are 13 papers from the 2013 international Symposium on Advances of Distributed and parallel Computing (ADPC 2013), 5 papers of the international Workshop on Big Data Computing (BDC 2013), 10 papers of the international Workshop on Trusted Information in Big Data (TIBiDa 2013) as well as 7 papers belonging to Workshop on Cloud-assisted Smart Cyber-Physical Systems (C-Smart CPS 2013).
Many of the key features of file transfer mechanisms like reliable file transferring and parallel transferring are developed as part of the service. It makes very hard to re-use the same code for the different systems...
详细信息
the Intel Xeon Phi is a many-core accelerator which focuses on the high performance applications. To characterize the performance of the Intel Xeon Phi, a system of dual 8-core Intel Xeon E5-2670 processors is employe...
详细信息
暂无评论