检索结果-内蒙古大学图书馆

iPACS: Power-aware covering sets for energy proportionality and performance in data parallel computing clusters

JOURNAL OF parallel AND DISTRIBUTED computing 2014年第1期74卷 1762-1774页

作者： Kim, Jinoh Chou, Jerry Rotem, Doron Texas A&M Univ Dept Comp Sci Commerce TX 75429 USA Natl Tsing Hua Univ Dept Comp Sci Hsinchu 30013 Taiwan Univ Calif Berkeley Lawrence Berkeley Natl Lab Sci Data Management Grp Berkeley CA 94706 USA

Energy consumption in datacenters has recently become a major concern due to the rising operational costs and scalability issues. Recent solutions to this problem propose the principle of energy proportionality, i.e., the amount of energy consumed by the server nodes must be proportional to the amount of work performed. For data parallelism and fault tolerance purposes, most common file systems used in MapReduce-type clusters maintain a set of replicas for each data block. A covering subset is a group of nodes that together contain at least one replica of the data blocks needed for performing computing tasks. In this work, we develop and analyze algorithms to maintain energy proportionality by discovering a covering subset that minimizes energy consumption while placing the remaining nodes in low-power standby mode in a data parallel computing cluster. Our algorithms can also discover covering subset in heterogeneous computing environments. In order to allow more data parallelism, we generalize our algorithms so that it can discover k-covering subset, i.e., a set of nodes that contain at least k replicas of the data blocks. Our experimental results show that we can achieve substantial energy saving without significant performance loss in diverse cluster configurations and working environments. (C) 2013 Elsevier Inc. All rights reserved.

关键词： Energy proportionality data parallel computing Covering subset MapReduce

来源：评论

学校读者我要写书评

暂无评论

Straggler identification approach in large data processing frameworks using ensembled gradient boosting in smart-cities cloud services

引用

INTERNATIONAL JOURNAL OF SYSTEM ASSURANCE ENGINEERING AND MANAGEMENT 2022年第SUPPL 1期13卷 146-155页

作者： Deshmukh, Shyam Rao, Komati Thirupathi Koneru Lakshmaiah Educ Fdn Dept Comp Sci & Engn Guntur Andhra Pradesh India

A smart city's efficiency must be achieved by mining large amounts of data generated by cyber-physical systems and electronic platforms using the large-scale data processing framework in cloud environment. Many cloud services rely on data parallel computing frameworks in cloud environment, which runs on hundreds of interconnected nodes. These frameworks divide the computationally intensive and data-intensive tasks into smaller tasks and run them concurrently on different nodes to improve performance. But providing improved performance in the processing environment is a challenge due to runtime variability. Due to different internal and external factors, nodes running these tasks do not perform well, resulting in the delay in the execution of these jobs. As a result of the inherent complexity of runtime variability, preventive measures for stragglers proved inadequate, and the problem continued to affect compute workloads even after the measures were taken. Several researchers proposed dynamic straggler identification approaches based on historical log analysis. This paper analyzes the relationship between several parameters obtained during job execution that will aid us in formulating and detecting the stragglers. Using data analysis, we developed the straggler identification approach and labeled the generated dataset. To achieve high performance using statistical features of historical resource usage, the proposed approach trains distributed XGBoost classifier which showed highest accuracy of 88.57%. Furthermore, we have empirically shown that blacklisting predicted stragglers led to a significant reduction in CPU, I/O, and mixed application execution times.

关键词： data parallel computing Smart cities Spark Straggler identification Ubiquitous computing XGBoost Classifier

来源：评论

学校读者我要写书评

暂无评论

Application of Top-N Rule-based Optimal Recommendation System for Language Education Content based on parallel computing

引用

INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS 2023年第6期14卷 1027-1037页

作者： Hu, Nan Nanyang Med Coll Publ Teaching Dept Nanyang 473000 Henan Peoples R China Wuhan Univ Sch Publ Hlth Wuhan 430000 Hubei Peoples R China

In recent years personalized recommendation services have been applied to many areas of society, typically in the fields of e-commerce, short videos and so on. In response to the serious performance problems of the current online language education platform content recommendation, so in the face of the above opportunities and challenges, this paper designs a new online English education model to allow university students to get a full and more three-dimensional training of English language learning. Based on the MU platform, this paper obtains data from the platform and uses crawler technology to sample and standardize the learning resources for online education. Then user information, such as explicit and implicit ratings of courses, is selected as the main basis for training a user interest preference model. Immediately afterwards, a PRF algorithm combining data parallelism and task parallelism optimization was executed and implemented on Apeche Spark to provide some optimization of data accuracy and content recommendation methods. Finally, the top-N recommendation rule is used to propose a dynamic evolutionary process of identifying students' preferences or learning habits through the results of previous data analysis, so as to make more accurate course content recommendations and learning content guidance for students' English learning. The online three-dimensional teaching model proposed in this paper focuses more on time-series research than traditional algorithms, and can more accurately capture the dynamic changes in students' learning abilities.

关键词： data parallel computing cloud computing data crawlers top-N rules PRF algorithm

来源：评论

学校读者我要写书评

暂无评论

A data parallel approach to modelling and simulation of large crowd

引用

CLUSTER computing-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2015年第3期18卷 1307-1316页

作者： Yu, Tao Dou, Minggang Zhu, Mao Tsinghua Univ Inst Network Sci & Cyberspace Beijing 100084 Peoples R China China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China

The modeling and simulation (M&S) of large crowd has become increasingly important in the domain of public security, such as facility planning, disaster response, and anti-terrorism operations. The behavior of a large crowd is highly complex, and the M&S of a large crowd at the individual level therefore demands the support of a scalable and efficient computing technology. In this study, a method was proposed to formulate crowd behavior with the cell automata and multi-agent models, which were successfully mapped onto the MapReduce programming model. A simulation framework was developed upon Hadoop to simulate large crowd scenarios over a cluster. The simulation process was then transformed to a series of parallel operations on data streams. The simulation studies on a large-scale evacuation scenario had indicated that the simulation framework ensured the simulation process' logic correctness. Experimental results also showed that the Hadoop-based simulation framework could complete five times more tasks while consuming only 19 % CPU time in comparison with the conventional simulation technology.

关键词： data parallel computing Modeling and simulation of complex systems Cloud computing MapReduce Hadoop

来源：评论

学校读者我要写书评

暂无评论

parallel Processing of Massive EEG data with MapReduce

Parallel Processing of Massive EEG Data with MapReduce

引用

IEEE 18th International Conference on parallel and Distributed Systems (ICPADS)

作者： Wang, Lizhe Chen, Dan Ranjan, Rajiv Khan, Samee U. Kolodziej, Joanna Wang, Jun Chinese Acad Sci Ctr Earth Observat & Digital Earth Beijing 100864 Peoples R China China Univ Geosciences Sch Comp Beijing Peoples R China CSIRO ICT Ctr Informat Engn lab Marshfield 2122 WI USA N Dakota State Univ Dept Elect & Comp Engn Grand Forks ND 58201 USA Univ Technol Inst Comp Sci Krakow Poland Univ Cent Florida Dept Elect Engn & Comp Sci Orlando FL 32816 USA

ISBN: (纸本)9781467345651;9780769549033

Analysis of neural signals like electroencephalogram (EEG) is one of the key technologies in detecting and diagnosing various brain disorders. As neural signals are non-stationary and non-linear in nature, it is almost impossible to understand their true physical dynamics until the recent advent of the Ensemble Empirical Mode Decomposition (EEMD) algorithm. The neural signal processing with EEMD is highly compute-intensive due to the high complexity of the EEMD algorithm. It is also data-intensive because 1) EEG signals contain massive data sets 2) EEMD has to introduce a large number of trials in processing to ensure precision. The MapReduce programming mode is a promising parallel computing paradigm for data intensive computing. To increase the efficiency and performance of the neural signal analysis, this research develops parallel EEMD neural signal processing with MapReduce. In this paper, we implement the parallel EEMD with Hadoop in a modern cyberinfrastructure. Test results and performance evaluation show that parallel EEMD can significantly improve the performance of neural signal processing.

关键词： data-intensive computing MapReduce parallel processing parallel PROCESSING (COMPUTERS) signal processing Electroencephalography Brain Diseases Key technology MAPPING(MATHEMATICAL) Signal analysis data parallel computing data flow computing

来源：评论

学校读者我要写书评

暂无评论

Accelerating the Training of HTK on GPU with CUDA

Accelerating the Training of HTK on GPU with CUDA

引用

26th IEEE International parallel and Distributed Processing Symposium (IPDPS) / Workshop on High Performance data Intensive computing

作者： Du, Zhihui Li, Xiangyu Wu, Ji Tsinghua Univ Dept Comp Sci & Technol Tsinhua Natl Lab Informat Sci & Technol Beijing 100084 Peoples R China Beijing Univ Posts & Telecommun Sch Comp Beijing 100876 Peoples R China Tsinghua Univ Tsinghua Natl Lab Informat Sci & Technol Dept Elect Engn Beijing 100084 Peoples R China

ISBN: (纸本)9780769546766

The training procedure of Hidden Markov Model (HMM) based Speech Recognition is often very time consuming because of its high computational complexity. The new parallel hardware like GPU can provide multi-thread processing and very high floating-point capability. We take advantage of GPU to accelerate a popular HMM-based Speech Recognition package - HTK. Based on the sequential code of HTK, we design the "paraTraining", a parallel training model in HTK and develop different optimization methods to improve the performance of HTK on GPU which include unrolling the nested loops and using "reduction add" which can maximize the number of threads per block;using warp mechanism of GPU to reduce synchronizing latency;building different indices of threads to address data efficiently. Experimental results show that about 20+ speedup can be achieved without loss in accuracy. We also discuss the implementation of our method on multi-GPU and got around two times speedup compared with on single-GPU.

关键词： Speech Recognition data parallel computing GPU computing CUDA Stream Processor

来源：评论

学校读者我要写书评

暂无评论

Adaptive data parallel computing on workstation clusters

引用

JOURNAL OF parallel AND DISTRIBUTED computing 2004年第11期64卷 1241-1255页

作者： Mahanti, A Eager, DL Univ Calgary Dept Comp Sci Calgary AB T2N 1N4 Canada Univ Saskatchewan Dept Comp Sci Saskatoon SK S7N 5A9 Canada

Many important parallel applications are data parallel, and may be efficiently implemented on a workstation cluster by allocating each workstation a contiguous partition of the data domain. Implementation on non-dedicated clusters, however, is complicated by the possibility of changes in workstation availability. For example, a personal workstation may be reclaimed by its primary user for interactive use. In such situations, a node must be removed from the collection of workstations forming the "virtual parallel machine" allocated to the application. and data redistributed accordingly. Conversely, workstations may become available to join the virtual parallel machine. This paper identifies fundamental characteristics of efficient policies for data redistribution following addition/removal of workstations front the cluster. The following conclusions are obtained based on mathematical analysis and simulations: (a) allocating data to a new node from the center of the data domain substantially reduces data migration costs compared to allocation from the edge;(b) addition in groups is beneficial compared to repeated single additions;and (c) even a large number of incremental adjustments of the data domain partitions, owing to successive additions/removals of nodes, do not appear to substantially degrade partition quality compared to that obtained by partitioning from scratch. We believe that these observations can be fruitfully incorporated in the design of workstation cluster support systems for data parallel computing. (C) 2004 Elsevier Inc. All rights reserved.

关键词： data parallel computing workstation clusters data redistribution data domain decomposition

来源：评论

学校读者我要写书评

暂无评论

Hybrid image classification and parameter selection using a shared memory parallel algorithm

引用

COMPUTERS & GEOSCIENCES 2007年第7期33卷 875-897页

作者： Phillips, Rhonda D. Watson, Layne T. Wynne, Randolph H. Virginia Polytech Inst & State Univ Dept Comp Sci Blacksburg VA 24061 USA Virginia Polytech Inst & State Univ Dept Math Blacksburg VA 24061 USA Virginia Polytech Inst & State Univ Dept Forestry Blacksburg VA 24061 USA

This work presents a shared memory parallel version of the hybrid classification algorithm IGSCR (iterative guided spectral class rejection) to facilitate the transition from serial to parallel processing. This transition is motivated by a demonstrated need for more computing power driven by the increasing size of remote sensing data sets due to higher resolution sensors, larger study regions, and the like. parallel IGSCR was developed to produce fast and portable code using Fortran 95, OpenMP, and the Hierarchical data Format version 5 (HDF5) and accompanying data access library. The intention of this work is to provide an efficient implementation of the established IGSCR classification algorithm. The applicability of the faster parallel IGSCR algorithm is demonstrated by classifying Landsat data covering most of Virginia, USA into forest and non-forest classes with approximately 90% accuracy. parallel results are given using the SGI Altix 3300 shared memory computer and the SGI Altix 3700 with as many as 64 processors reaching speedups of almost 77. parallel IGSCR allows an analyst to perform and assess multiple classifications to refine parameters. As an example, parallel IGSCR was used for a factorial analysis consisting of 42 classifications of a 1.2 GB image to select the number of initial classes (70) and class purity (70%) used for the remaining two images. (C) 2007 Elsevier Ltd. All rights reserved.

关键词： parallel processing remote sensing landsat forest area high performance computing data parallel computing

来源：评论

学校读者我要写书评

暂无评论

Interactive k-D Tree GPU Raytracing 07

Interactive k-D Tree GPU Raytracing

引用

Symposium on Interactive 3D Graphics and Games

作者： Horn, Daniel Reiter Sugerman, Jeremy Houston, Mike Hartrahan, Pat Stanford Univ Stanford CA 94305 USA

ISBN: (纸本)9781595936288

Over the past few years, the powerful computation rates and high memory bandwidth of GPUs have attracted efforts to run raytracing oil GPUs. Our work extends Foley et al.'s GPU k-d tree research. We port their kd-restart algorithm from multi-pass, using CPU load balancing, to single pass, using current GPUs' branching and looping abilities. We introduce three optimizations: a packetized formulation, a technique for restarting partially down the tree instead of at the root, and a small, fixed-size stack that is checked before resorting to restart. Our optimized implementation achieves 15 - 18 million primary rays per second and 16 - 27 million shadow rays per second oil our test scenes. Our system also takes advantage of GPUs' strengths at rasterization and shading to offer a mode where rasterization replaces eye ray scene intersection, and primary hits and local shading are produced with standard Direct3D code. For 1024x1024 renderings of our scenes with shadows and Phong shading, we achieve 12-18 frames per second. Finally, we investigate the efficiency of our implementation relative to the computational resources of our GPUs and also compare it against conventional CPUs and the Cell processor, which both have been shown to raytrace well.

关键词： Programmable Graphics Hardware data parallel computing Stream computing GPU computing Brook

来源：评论

学校读者我要写书评

暂无评论

ClawHMMER: A Streaming HMMer-Search Implementatio 05

ClawHMMER: A Streaming HMMer-Search Implementatio

引用

Proceedings of the 2005 ACM/IEEE conference on Supercomputing

作者： Daniel Reiter Horn Mike Houston Pat Hanrahan Stanford University

ISBN: (纸本)9781595930613

The proliferation of biological sequence data has motivated the need for an extremely fast probabilistic sequence search. One method for performing this search involves evaluating the Viterbi probability of a hidden Markov model (HMM) of a desired sequence family for each sequence in a protein database. However, one of the difficulties with current implementations is the time required to search large databases. Many current and upcoming architectures offering large amounts of compute power are designed with data-parallel execution and streaming in mind. We present a streaming algorithm for evaluating an HMM's Viterbi probability and refine it for the specific HMM used in biological sequence search. We implement our streaming algorithm in the Brook language, allowing us to execute the algorithm on graphics processors. We demonstrate that this streaming algorithm on graphics processors can outperform available CPU implementations. We also demonstrate this implementation running on a 16 node graphics cluster.

关键词： data parallel computing Brook Bio Science Stream computing Programmable Graphics Hardware GPU computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：