检索结果-内蒙古大学图书馆

Load Feedback-Based resource scheduling and Dynamic Migration-Based data locality for Virtual hadoop Clusters in openstack-Based Clouds

引用

Tsinghua Science and Technology 2017年第2期22卷 149-159页

作者： Dan Tao Zhaowen Lin Bingxu Wang School of Electronic and Information Engineering Beijing Jiaotong UniversityBeijing 100044China Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks Nanjing 210003China Network and Information Center Institute of Network TechnologyScience and Technology on Information Transmission and Dissemination in Communication Networks LaboratoryNational Engineering Laboratory for Mobile Network SecurityBeijing University of Posts and TelecommunicationsBeijing 100876China

With cloud computing technology becoming more mature, it is essential to combine the big data processing tool hadoop with the infrastructure as a service（Iaa S） cloud platform. In this study, we first propose a new Dynamic hadoop Cluster on Iaa S（DHCI） architecture, which includes four key modules： monitoring,scheduling, Virtual Machine（VM） management, and VM migration modules. The load of both physical hosts and VMs is collected by the monitoring module and can be used to design resource scheduling and data locality solutions. Second, we present a simple load feedback-based resource scheduling scheme. The resource allocation can be avoided on overburdened physical hosts or the strong scalability of virtual cluster can be achieved by fluctuating the number of VMs. To improve the flexibility, we adopt the separated deployment of the computation and storage VMs in the DHCI architecture, which negatively impacts the data locality. Third, we reuse the method of VM migration and propose a dynamic migration-based data locality scheme using parallel computing entropy. We migrate the computation nodes to different host（s） or rack（s） where the corresponding storage nodes are deployed to satisfy the requirement of data locality. We evaluate our solutions in a realistic scenario based on Open *** experimental results demonstrate the effectiveness of our solutions that contribute to balance the workload and performance improvement, even under heavy-loaded cloud system conditions.

关键词： hadoop resource scheduling data locality infrastructure as a service（iaas） openstack

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：