检索结果-内蒙古大学图书馆

On using MapReduce to scale algorithms for big data analytics: a case study

JOURNAL OF big data 2019年第1期6卷 1页

作者： Kijsanayothin, Phongphun Chalumporn, Gantaphon Hewett, Rattikorn Naresuan Univ Dept Elect & Comp Engn NU Phitsanulok Thailand Texas Tech Univ Dept Comp Sci TTU Lubbock TX 79409 USA

IntroductionMany data analytics algorithms are originally designed for in-memory data. Parallel and distributed computing is a natural first remedy to scale these algorithms to "big algorithms" for large-scale data. Advances in many big data analytics algorithms are contributed by MapReduce, a programming paradigm that enables parallel and distributed execution of massive data processing on large clusters of machines. Much research has focused on building efficient naive MapReduce-based algorithms or extending MapReduce mechanisms to enhance performance. However, we argue that these should not be the only research directions to pursue. We conjecture that when naive MapReduce-based solutions do not perform well, it could be because certain classes of algorithms are not amendable to MapReduce model and one should find a fundamentally different approach to a new MapReduce-based *** descriptionThis paper investigates a case study of a scaling problem of "big algorithms" for a popular association rule-mining algorithm, particularly the development of Apriori algorithm in MapReduce *** and evaluationFormal and empirical illustrations are explored to compare our proposed MapReduce-based Apriori algorithm with previous solutions. The findings support our conjecture and our study shows promising results compared to the state-of-the-art performer with 7% increase in performance on the average of transactions ranging from 10,000 to 120,*** results confirm that effective MapReduce implementation should avoid dependent iterations, such as that of the original sequential Apriori algorithm. These findings could lead to many more alternative non-naive MapReduce-based "big algorithms".

关键词： big data analytics algorithms Association rules mining MapReduce Parallel computing

来源：评论

学校读者我要写书评

暂无评论

A Sub-Linear Scalable MapReduce-based Apriori Algorithm 19

A Sub-Linear Scalable MapReduce-based Apriori Algorithm

引用

Proceedings of the 3rd International Conference on big data Research

作者： Gantaphon Chalumporn Phongphun Kijsanayothin Rattikorn Hewett Department of Computer Science Texas Tech University TTU Lubbock USA Department of Electrical and Computer Engineering Naresuan University NU Phitsanulok Thailand

ISBN: (纸本)9781450372015

Association Rule Mining is one of the most popular data analytic algorithms where the well-known Apriori algorithm is its core. Like most machine learning and data mining algorithms, Apriori algorithm is designed for in-memory data. One natural way to cope with this limitation and emerging bigdata challenges is to adapt the algorithm to parallel and distributed computing infrastructures, particularly a widely used Map-Reduce model. Much research has developed a variety of MapReduce-based Apriori algorithms. However, most have focused on either improving performance over that of the original Apriori algorithm or mechanisms MapReduce infrastructure. This paper presents yet another MapReduce-based Apriori algorithm. Unlike most traditional MapReduce-based Apriori that mimics incremental level-wise computation of the original Apriori, our MapReduce-based algorithm opportunistically exploits the map's keys for non-level-wise parallelism to fully benefit of parallel processing to gain efficiency. The paper describes our proposed approach and shows an empirical evaluation of its performance compared with that of the representative traditional MapReduce-based algorithm. The results show significant improvement with an average reduction of the execution time of about 70%, over 50,000-200,000 transactions on one to three machines, with 10% of support threshold. In fact, the execution time of the proposed MapReduce-based Apriori algorithm is empirically shown to scale sub-linearly (better than linear) in the number of transactions.

关键词： MapReduce-based Apriori Parallel computing Association rules mining big data analytics algorithms

来源：评论

学校读者我要写书评

暂无评论

A Sub-Linear Scalable MapReduce-based Apriori Algorithm

A Sub-Linear Scalable MapReduce-based Apriori Algorithm

引用

作者： Gantaphon Chalumporn Phongphun Kijsanayothin Rattikorn Hewett Department of Computer Science Texas Tech UniversityTTU Department of Electrical and Computer Engineering Naresuan UniversityNU

Association Rule Mining is one of the most popular data analytic algorithms where the well-known Apriori algorithm is its core. Like most machine learning and data mining algorithms, Apriori algorithm is designed for in-memory data. One natural way to cope with this limitation and emerging bigdata challenges is to adapt the algorithm to parallel and distributed computing infrastructures, particularly a widely used Map-Reduce model. Much research has developed a variety of MapReduce-based Apriori algorithms. However, most have focused on either improving performance over that of the original Apriori algorithm or mechanisms MapReduce infrastructure. This paper presents yet another MapReduce-based Apriori algorithm. Unlike most traditional MapReduce-based Apriori that mimics incremental level-wise computation of the original Apriori, our MapReducebased algorithm opportunistically exploits the map's keys for nonlevel-wise parallelism to fully benefit of parallel processing to gain efficiency. The paper describes our proposed approach and shows an empirical evaluation of its performance compared with that of the representative traditional MapReduce-based algorithm. The results show significant improvement with an average reduction of the execution time of about 70%, over 50,000-200,000 transactions on one to three machines, with 10% of support threshold. In fact, the execution time of the proposed MapReduce-based Apriori algorithm is empirically shown to scale sub-linearly(better than linear) in the number of transactions.

关键词： big data analytics algorithms Association rules mining Parallel computing MapReduce-based Apriori

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：