Resource Bricolage for Parallel DBMSs on Heterogeneous Clusters

作     者：Li, Jiexing Naughton, Jeffrey Nehme, Rimma V. 

作者机构：Google Inc Mountain View CA USA Univ Wisconsin Madison WI USA Microsoft Jim Gray Syst Lab Madison WI USA 

出 版 物：《SIGMOD RECORD》 (SIGMOD Rec.)

年 卷 期：2016年第45卷第1期

页      面：42-49页

核心收录：

学科分类：08[工学] 0835[工学-软件工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：Microsoft Jim Gray Systems Lab  Madison  WI 

主　　题：Database systems machinery machine processing time Parallel Lines parallel databases Heterogeneous 

摘      要：Running parallel database systems in an environment with heterogeneous resources has become increasingly common, due to cluster evolution and increasing interest in moving applications into public clouds or shared infrastructures. For database systems running in a heterogeneous cluster, the default uniform data partitioning strategy may overload some of the slow machines while at the same time it may under-utilize the more powerful machines. Since the processing time of a parallel query is determined by the slowest machine, such an allocation strategy may result in a significant query performance degradation. We take a first step to address this problem by introducing a technique we call resource bricolage that improves database performance in heterogeneous environments. Our approach quantifies the performance differences among machines with various resources as they process workloads with diverse resource requirements. We formalize the problem of minimizing workload execution time and view it as an optimization problem, and then we employ linear programming to obtain a recommended data partitioning scheme. We verify the effectiveness of our technique with an extensive experimental study on a commercial database system.