检索结果-内蒙古大学图书馆

A parallel query processing system based on graph-based database partitioning

INFORMATION SCIENCES 2019年 480卷 237-260页

作者： Nam, Yoon-Min Han, Donghyoung Kim, Min-Soo DGIST Daegu South Korea

As parallel database systems have large amounts of data to process, it is important to utilize a scalable and efficient horizontal database partitioning method. The existing partitioning methods have major drawbacks that not only cause large amounts of data redundancy but also still require expensive shuffle operations for join queries in many cases-despite their high data redundancy. We elucidate upon the drawbacks originating from the tree-based partitioning schemes and propose a novel graph-based database partitioning method called GPT that both improves the query performance and reduces data redundancy. We integrate the proposed GPT method into a parallel query processing system, Spark SQL, across all the relevant layers and modules, including the query plan generator and the scan operator. Through extensive experiments using three benchmarks, TPC-DS, IMDB and BioWarehouse, we show that GPT significantly outperforms the state-of-the-art method in terms of both storage overhead and query performance. (C) 2018 Elsevier Inc. All rights reserved.

关键词： Horizontal database partitioning graph-based partitioning Parallel query processing

来源：评论

学校读者我要写书评

暂无评论

Algebraic multiscale grid coarsening using unsupervised machine learning for subsurface flow simulation

引用

JOURNAL OF COMPUTATIONAL PHYSICS 2024年 496卷

作者： Kumar, Kishan Ramesh Tene, Matei Delft Univ Technol Fac Civil Engn & Geosci Dept Geosci & Engn Stevinweg 1 NL-2628CV Delft Netherlands SLB Norway Technol Ctr Fornebuveien 3 N-1366 Lysaker Norway

Subsurface flow simulation is vital for many geoscience applications, including geoenergy extraction and gas (energy) storage. Reservoirs are often highly heterogeneous and naturally fractured. Therefore, scalable simulation strategies are crucial to enable efficient and reliable operational strategies. One of these scalable methods, which has also been recently deployed in commercial reservoir simulators, is algebraic multiscale (AMS) solvers. AMS, like all multilevel schemes, is found to be highly sensitive to the types (geometries and size) of coarse grids and local basis functions. Commercial simulators benefit from a graph-based partitioner;e.g., METIS to generate the multiscale coarse grids. METIS minimizes the amount of interfaces between coarse partitions, while keeping them of similar size which may not be the requirement to create a coarse grid. In this work, we employ a novel approach to generate the multiscale coarse grids, using unsupervised learning methods which is based on optimizing different parameter. We specifically use the Louvain algorithm and Multi-level Markov clustering. The Louvain algorithm optimizes modularity, a measure of the strength of network division while Markov clustering simulates random walks between the cells to find clusters. It is found that the AMS performance is improved when compared with the existing METIS-based partitioner on several field-scale test cases. This development has the potential to enable reservoir engineers to run ensembles of thousands of detailed models at a much faster rate.

关键词： Reservoir simulation Unsupervised learning graph-based partitioning Computational performance Algebraic multiscale methods

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：