检索结果-内蒙古大学图书馆

33rd International Conference on Scientific and Statistical database Management (SSDBM)

作者： Schuele, Maximilian Lang, Harald Springer, Maximilian Kemper, Alfons Neumann, Thomas Guennemann, Stephan Tech Univ Munich Munich Germany

ISBN: (纸本)9781450384131

In machine learning, continuously retraining a model guarantees accurate predictions based on the latest data as training input. But to retrieve the latest data from a database, time-consuming extraction is necessary as database systems have rarely been used for operations such as matrix algebra and gradient descent. In this work, we demonstrate that SQL with recursive tables makes it possible to express a complete machine learning pipeline out of data preprocessing, model training and its validation. To facilitate the specification of loss functions, we extend the code-generating database system Umbra by an operator for automatic differentiation for use within recursive tables: With the loss function expressed in SQL as a lambda function, Umbra generates machine code for each partial derivative. We further use automatic differentiation for a dedicated gradient descent operator, which generates LLVM code to train a user-specified model on GPUs. We fine-tune GPU kernels at hardware level to allow a higher throughput and propose non-blocking synchronisation of multiple units. In our evaluation, automatic differentiation accelerated the run-time by the number of cached subexpressions compared to compiling each derivative separately. Our GPU kernels with independent models allowed maximal throughput even for small batch sizes, making machine learning pipelines within SQL more competitive.

关键词： in-database machine learning Automatic Differentiation GPU

来源：评论

学校读者我要写书评

暂无评论

in-database machine learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle 22

In-Database Machine Learning with CorgiPile: Stochastic Grad...

引用

International Conference on Management of Data (SIGMOD)

作者： Xu, Lijie Qiu, Shuang Yuan, Binhang Jiang, Jiawei Renggli, Cedric Gan, Shaoduo Kara, Kaan Li, Guoliang Liu, Ji Wu, Wentao Ye, Jieping Zhang, Ce Swiss Fed Inst Technol Zurich Switzerland Chinese Acad Sci State Key Lab Comp Sci Inst Software Beijing Peoples R China Univ Chicago Chicago IL 60637 USA Tsinghua Univ Beijing Peoples R China Kwai Inc Beijing Peoples R China Microsoft Res Redmond WA USA Univ Michigan Ann Arbor MI 48109 USA

ISBN: (纸本)9781450392495

Stochastic gradient descent (SGD) is the cornerstone of modern ML systems. Despite its computational efficiency, SGD requires random data access that is inherently inefficient when implemented in systems that rely on block-addressable secondary storage such as HDD and SSD, e.g., in-DB ML systems and TensorFlow/PyTorch over large files. To address this impedance mismatch, various data shuffling strategies have been proposed to balance the convergence rate of SGD (which favors randomness) and its I/O performance (which favors sequential access). In this paper, we first conduct a systematic empirical study on existing data shuffling strategies, which reveals that all existing strategies have room for improvement-they suffer in terms of I/O performance or convergence rate. With this in mind, we propose a simple but novel hierarchical data shuffling strategy, CorgiPile. Compared with existing strategies, CorgiPile avoids a full data shuffle while maintaining comparable convergence rate of SGD as if a full shuffle were performed. We provide a non-trivial theoretical analysis of CorgiPile on its convergence behavior. We further integrate CorgiPile into PostgreSQL by introducing three new physical operators with optimizations. Our experimental results show that CorgiPile can achieve comparable convergence rate to the full shuffle based SGD, and 1.6x-12.8x faster than two state-ofthe-art in-DB ML systems, Apache MADlib and Bismarck, on both HDD and SSD.

关键词： in-database machine learning Stochastic Gradient Descent Shuffle

来源：评论

学校读者我要写书评

暂无评论

Recursive SQL and GPU-support for in-database machine learning

引用

DISTRIBUTED AND PARALLEL databaseS 2022年第2-3期40卷 205-259页

作者： Schuele, Maximilian E. Lang, Harald Springer, Maximilian Kemper, Alfons Neumann, Thomas Guennemann, Stephan TUM Boltzmannstr 3 D-85748 Garching Bavaria Germany

In machine learning, continuously retraining a model guarantees accurate predictions based on the latest data as training input. But to retrieve the latest data from a database, time-consuming extraction is necessary as database systems have rarely been used for operations such as matrix algebra and gradient descent. In this work, we demonstrate that SQL with recursive tables makes it possible to express a complete machine learning pipeline out of data preprocessing, model training and its validation. To facilitate the specification of loss functions, we extend the code-generating database system Umbra by an operator for automatic differentiation for use within recursive tables: With the loss function expressed in SQL as a lambda function, Umbra generates machine code for each partial derivative. We further use automatic differentiation for a dedicated gradient descent operator, which generates LLVM code to train a user-specified model on GPUs. We fine-tune GPU kernels at hardware level to allow a higher throughput and propose non-blocking synchronisation of multiple units. In our evaluation, automatic differentiation accelerated the runtime by the number of cached subexpressions compared to compiling each derivative separately. Our GPU kernels with independent models allowed maximal throughput even for small batch sizes, making machine learning pipelines within SQL more competitive.

关键词： SQL GPU in-database machine learning Code-generation

来源：评论

学校读者我要写书评

暂无评论

Stochastic gradient descent without full data shuffle: with applications to in-database machine learning and deep learning systems

引用

VLDB JOURNAL 2024年第5期33卷 1231-1255页

作者： Xu, Lijie Qiu, Shuang Yuan, Binhang Jiang, Jiawei Renggli, Cedric Gan, Shaoduo Kara, Kaan Li, Guoliang Liu, Ji Wu, Wentao Ye, Jieping Zhang, Ce Swiss Fed Inst Technol Zurich Switzerland Chinese Acad Sci Inst Software Beijing Peoples R China Hong Kong Univ Sci & Technol Clear Water Bay Hong Kong Peoples R China Wuhan Univ Wuhan Peoples R China Tsinghua Univ Beijing Peoples R China Meta Menlo Pk CA USA Microsoft Res Washington DC USA Univ Michigan Ann Arbor MI USA Univ Chicago Chicago IL USA

Modern machine learning (ML) systems commonly use stochastic gradient descent (SGD) to train ML models. However, SGD relies on random data order to converge, which usually requires a full data shuffle. For in-DB ML systems and deep learning systems with large datasets stored on block-addressable secondary storage such as HDD and SSD, this full data shuffle leads to low I/O performance-the data shuffling time can be even longer than the training itself, due to massive random data accesses. To balance the convergence rate of SGD (which favors data randomness) and its I/O performance (which favors sequential access), previous work has proposed several data shuffling strategies. In this paper, we first perform an empirical study on existing data shuffling strategies, showing that these strategies suffer from either low performance or low convergence rate. To solve this problem, we propose a simple but novel two-level data shuffling strategy named CorgiPile, which can avoid a full data shuffle while maintaining comparable convergence rate of SGD as if a full shuffle were performed. We further theoretically analyze the convergence behavior of CorgiPile and empirically evaluate its efficacy in both in-DB ML and deep learning systems. For in-DB ML systems, we integrate CorgiPile into PostgreSQL by introducing three new physical operators with optimizations. For deep learning systems, we extend single-process CorgiPile to multi-process CorgiPile for the parallel/distributed environment and integrate it into PyTorch. Our evaluation shows that CorgiPile can achieve comparable convergence rate with the full-shuffle-based SGD for both linear models and deep learning models. For in-DB ML with linear models, CorgiPile is 1.6 x \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} - \d

关键词： in-database machine learning Stochastic gradient descent Data shuffle Deep learning

来源：评论

学校读者我要写书评

暂无评论

LLVM Code Optimisation for Automatic Differentiation When Forward and Reverse Mode Lead in the Same Direction 6

LLVM Code Optimisation for Automatic Differentiation When Fo...

引用

6th Workshop on Data Management for End-To-End machine learning (DEEM)

作者： Schuele, Maximilian E. Springer, Maximilian Kemper, Alfons Neumann, Thomas Tech Univ Munich Munich Germany

ISBN: (纸本)9781450393751

Both forward and reverse mode automatic differentiation derive a model function as used for gradient descent automatically. Reverse mode calculates all derivatives in one run, whereas forward mode requires rerunning the algorithm with respect to every variable for which the derivative is needed. To allow for in-database machine learning, we have integrated automatic differentiation as an SQL operator inside the Umbra database system. To benchmark code-generation to GPU, we implement forward as well as reverse mode automatic differentiation. The inspection of the optimised LLVM code shows that nearly the same machine code is executed after the generated LLVM code has been optimised. Thus, both modes yield similar runtimes but different compilation times.

关键词： in-database machine learning Automatic Differentiation GPU

来源：评论

学校读者我要写书评

暂无评论

On Functional Aggregate Queries with Additive Inequalities 19

On Functional Aggregate Queries with Additive Inequalities

引用

38th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of database Systems (PODS)

作者： Khamis, Mahmoud Abo Curtin, Ryan R. Moseley, Benjamin Ngo, Hung Q. Nguyen, XuanLong Olteanu, Dan Schleich, Maximilian RelationalAl Berkeley CA 94704 USA Carnegie Mellon Univ Pittsburgh PA 15213 USA Univ Michigan Ann Arbor MI 48109 USA Univ Oxford Oxford England

ISBN: (纸本)9781450362276

Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-AI in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle's geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k-means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.

关键词： functional aggregate queries inequality joins in-database machine learning non-polynomial loss

来源：评论

学校读者我要写书评

暂无评论

Multi-layer Optimizations for End-to-End Data Analytics 2020

Multi-layer Optimizations for End-to-End Data Analytics

引用

18th ACM/IEEE International Symposium on Code Generation and Optimization (CGO)

作者： Shaikhha, Amir Schleich, Maximilian Ghita, Alexandru Olteanu, Dan Univ Oxford Oxford England

ISBN: (纸本)9781450370479

We consider the problem of training machine learning models over multi-relational data. The mainstream approach is to first construct the training dataset using a feature extraction query over input database and then use a statistical software package of choice to train the model. In this paper we introduce Iterative Functional Aggregate Queries (IFAQ), a framework that realizes an alternative approach. IFAQ treats the feature extraction query and the learning task as one program given in the IFAQ's domain-specific language, which captures a subset of Python commonly used in Jupyter notebooks for rapid prototyping of machine learning applications. The program is subject to several layers of IFAQ optimizations, such as algebraic transformations, loop transformations, schema specialization, data layout optimizations, and finally compilation into efficient low-level C++ code specialized for the given workload and data. We show that a Scala implementation of IFAQ can outperform mlpack, Scikit, and TensorFlow by several orders of magnitude for linear regression and regression tree models over several relational datasets.

关键词： in-database machine learning Multi-Query Optimization Query Compilation

来源：评论

学校读者我要写书评

暂无评论

IDEL: In-database Neural Entity Linking

IDEL: In-Database Neural Entity Linking

引用

IEEE International Conference on Big Data and Smart Computing (BigComp)

作者： Kilias, Torsten Loeser, Alexander Gers, Felix Zhang, Ying Koopmanschap, Richard Kersten, Martin Beuth Univ Appl Sci Luxemburger Str 10 D-13353 Berlin Germany MonetDB Solut Sci Pk 123 NL-1098XG Amsterdam Netherlands

ISBN: (纸本)9781538677896

We present a novel architecture In-database Entity Linking (IDEL), in which we integrate the analytical RDBMS MonetDB with neural text mining abilities. To the best of our knowledge, this is the first defacto implemented system integrating entity-linking in a database. IDEL represents text and relational data in a joint vector space with neural embeddings and can compensate errors with ambiguous entity representations. To detect matching entities, we propose a novel similarity function based on joint neural embeddings which are learned via minimizing pairwise contrastive ranking loss. This function utilizes high dimensional index structures for fast retrieval of matching entities. The system achieves zero cost for data shipping and transformation by utilizing MonetDB's ability to embed Python processes in its kernel and exchange data in NumPy arrays. We report from experiments on the WebNLG corpus on ten entity types high F-measures and low execution times.

关键词： Entity Linking Joint Embedding Space Pairwise Contrastive Ranking Loss in-database machine learning

来源：评论

学校读者我要写书评

暂无评论

Functional Aggregate Queries with Additive Inequalities

引用

ACM TRANSACTIONS ON database SYSTEMS 2020年第4期45卷 17-17页

作者： Khamis, Mahmoud Abo Curtin, Ryan R. Moseley, Benjamin Ngo, Hung Q. Nguyen, Xuanlong Olteanu, Dan Schleich, Maximilian Relat AI Inc 2120 Univ Ave Berkeley CA 94704 USA Carnegie Mellon Univ Tepper Sch Business 5000 Forbes Ave Pittsburgh PA 15213 USA Univ Michigan 61 West Hall1085 South Univ Ann Arbor MI 48109 USA Univ Zurich Dept Informat Andreasstr 15 CH-8050 Zurich Switzerland Univ Washington Paul G Allen Sch Comp Sci & Engn 3800 East Stevens Way NE Seattle WA 98105 USA

Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-Al in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle's geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k-means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.

关键词： Functional aggregate queries inequality joins in-database machine learning non-polynomial loss

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：