检索结果-内蒙古大学图书馆

31st Symposium of Advanced Database Systems, SEBD 2023

作者： Ioannidis, Yannis University of Athens Greece

On 3rd July 2023, Yannis Ioannidis delivered a keynote talk at the 31st Symposium on Advanced Database Systems in Galzignano Terme (Padua, Italy). The following is the abstract of his talk and a short biography. © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

关键词： SEBD 2023 Keynote SQL user-defined functions YeSQL

来源：评论

学校读者我要写书评

暂无评论

Declarative Parameterizations of user-defined functions for Large-Scale Machine Learning and Optimization

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2019年第11期31卷 2079-2092页

作者： Gao, Zekai J. Pansare, Niketan Jermaine, Christopher Rice Univ Houston TX 77251 USA IBM Res Almaden San Jose CA 95120 USA

Large-scale optimization has become an important application for data management systems, particularly in the context of statistical machine learning. In this paper, we consider how one might implement the join-and-co-group pattern in the context of a fully declarative data processing system. The join-and-co-group pattern is ubiquitous in iterative, large-scale optimization. In the join-and-co-group pattern, a user-defined function g is parameterized with a data object x as well as the subset of the statistical model Theta(x) that applies to that object, so that g(x vertical bar Theta(x)) can be used to compute a partial update of the model. This is repeated for every x in the full data set X. All partial updates are then aggregated and used to perform a complete update of the model. The join-and-co-group pattern has several implementation challenges, including the potential for a massive blow-up in the size of a fully parameterized model. Thus, unless the correct physical execution plan be chosen for implementing the join-and-co-group pattern, it is easily possible to have an execution that takes a very long time or even fails to complete. In this paper, we carefully consider the alternatives for implementing the join-and-co-group pattern on top of a declarative system, as well as how the best alternative can be selected automatically. Our focus is on the SimSQL database system, which is an SQL-based system with special facilities for large-scale, iterative optimization. Since it is an SQL-based system with a query optimizer, those choices can be made automatically.

关键词： Large-scale machine learning user-defined functions declarative systems join-and-co-group

来源：评论

学校读者我要写书评

暂无评论

Scalable architecture for Big Data financial analytics: user-defined functions vs. SQL

引用

JOURNAL OF BIG DATA 2019年第1期6卷 1页

作者： Stockinger, Kurt Bundi, Nils Heitz, Jonas Breymann, Wolfgang Zurich Univ Appl Sci Technikumstr 9 CH-8400 Winterthur Switzerland Ariadne Business Analyt AG Alpenstr 16 CH-6300 Zug Switzerland Stevens Inst Technol 1 Castle Point Terrace Hoboken NJ 07030 USA

Large financial organizations have hundreds of millions of financial contracts on their balance sheets. Moreover, highly volatile financial markets and heterogeneous data sets within and across banks world-wide make near real-time financial analytics very challenging and their handling thus requires cutting edge financial algorithms. However, due to a lack of data modeling standards, current financial risk algorithms are typically inconsistent and non-scalable. In this paper, we present a novel implementation of a real-world use case for performing large-scale financial analytics leveraging Big Data technology. We first provide detailed background information on the financial underpinnings of our framework along with the major financial calculations. Afterwards we analyze the performance of different parallel implementations in Apache Spark based on existing computation kernels that apply the ACTUS data and algorithmic standard for financial contract modeling. The major contribution is a detailed discussion of the design trade-offs between applying user-defined functions on existing computation kernels vs. partially re-writing the kernel in SQL and thus taking advantage of the underlying SQL query optimizer. Our performance evaluation demonstrates almost linear scalability for the best design choice.

关键词： Financial analytics Query processing user-defined functions Performance evaluation

来源：评论

学校读者我要写书评

暂无评论

Data Leakage Mitigation of user-defined functions on Secure Personal Data Management Systems 22

Data Leakage Mitigation of User-Defined Functions on Secure ...

引用

34th International Conference on Scientific and Statistical Database Management (SSDBM)

作者： Carpentier, Robin Popa, Iulian Sandu Anciaux, Nicolas Univ Versailles St Q En Yvelines Versailles France Inria Saclay Ile de France Palaiseau France

ISBN: (纸本)9781450396677

Personal Data Management Systems (PDMSs) arrive at a rapid pace providing individuals with appropriate tools to collect, manage and share their personal data. At the same time, the emergence of Trusted Execution Environments (TEEs) opens new perspectives in solving the critical and conflicting challenge of securing users' data while enabling a rich ecosystem of data-driven applications. In this paper, we propose a PDMS architecture leveraging TEEs as a basis for security. Unlike existing solutions, our architecture allows for data processing extensiveness through the integration of any user-defined functions, albeit untrusted by the data owner. In this context, we focus on aggregate computations of large sets of database objects and provide a first study to mitigate the very large potential data leakage. We introduce the necessary security building blocks and show that an upper bound on data leakage can be guaranteed to the PDMS user. We then propose practical evaluation strategies ensuring that the potential data leakage remains minimal with a reasonable performance overhead. Finally, we validate our proposal with an Intel SGX-based PDMS implementation on real data sets.

关键词： Personal Data Management Systems user-defined functions Untrusted Code Information leakage Trusted Execution Environment

来源：评论

学校读者我要写书评

暂无评论

Demonstration of Udon: Line-by-line Debugging of user-defined functions in Data Workflows 24

Demonstration of Udon: Line-by-line Debugging of User-Define...

引用

ACM International Conference on Management of Data (SIGMOD)

作者： Huang, Yicong Wang, Zuozhi Li, Chen Univ Calif Irvine Dept Comp Sci Irvine CA 92717 USA

ISBN: (纸本)9798400704222

Many big data systems are written in languages such as C, C++, Java, and Scala for high efficiency, whereas data analysts often use Python to conduct data wrangling, statistical analysis, and machine learning. user-defined functions (UDFs) are commonly used in these systems to bridge the gap between the two ecosystems. Debugging complex UDFs in data-processing systems is challenging due to the required coordination between language debuggers and the data-processing engine, as well as the debugging overhead on large volumes of data. In this paper, we showcase Udon, a novel debugger to support line-by-line debugging of UDFs in data-processing systems. Udon encapsulates modern line-by-line debugging primitives, such as those to set breakpoints, perform code inspections, and make code modifications while executing a UDF on a single tuple. In this demonstration, we use real-world scenarios to showcase the experience of using Udon for line-by-line debugging of a UDF.

关键词： data workflows user-defined functions debugging python udf

来源：评论

学校读者我要写书评

暂无评论

Consolidation of Queries with user-defined functions 14

Consolidation of Queries with User-Defined Functions

引用

35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)

作者： Sousa, Marcelo Dillig, Isil Vytiniotis, Dimitrios Dillig, Thomas Gkantsidis, Christos Univ Oxford Oxford OX1 2JD England UT Austin Austin TX USA Microsoft Res Oxford England

ISBN: (纸本)9781450327848

Motivated by streaming and data analytics scenarios where many queries operate on the same data and perform similar computations, we propose program consolidation for merging multiple user-defined functions (UDFs) that operate on the same input. Program consolidation exploits common computations between UDFs to generate an equivalent optimized function whose execution cost is often much smaller (and never greater) than the sum of the costs of executing each function individually. We present a sound consolidation calculus and an effective algorithm for consolidating multiple UDFs. Our approach is purely static and uses symbolic SMT-based techniques to identify shared or redundant computations. We have implemented the proposed technique on top of the Naiad data processing system. Our experiments show that our algorithm dramatically improves overall job completion time when executing user-defined filters that operate on the same data and perform similar computations.

关键词： Consolidation user-defined functions query optimization

来源：评论

学校读者我要写书评

暂无评论

Towards a Cost Model to Optimize user-defined functions in an ETL Workflow Based on user-defined Performance Metrics 23rd

Towards a Cost Model to Optimize User-Defined Functions in a...

引用

23rd European Conference on Advances in Databases and Information Systems (ADBIS)

作者： Ali, Syed Muhammad Fawad Wrembel, Robert Poznan Univ Tech Poznan Poland

ISBN: (纸本)9783030287306;9783030287290

Today's ETL tools provide capabilities for developing custom code as user-defined functions (UDFs) to extend the expressiveness of standard ETL operators. However, a custom code of an UDF may execute inefficiently due to its poor implementation (e.g., due to the lack of using parallel processing or adequate data structures). In this paper we address the problem of the optimization of UDFs in data-intensive workflows and presented our approach to construct a cost model to determine the degree of parallelism for parallelizable UDFs.

关键词： ETL workflow ETL execution optimization user-defined functions Cost model Parallelization

来源：评论

学校读者我要写书评

暂无评论

Soft querying powered by user-defined functions in J-CO-QL

引用

NEUROCOMPUTING 2023年第1期546卷

作者： Fosci, Paolo Psaila, Giuseppe Univ Bergamo Dept Management Informat & Prod Engn Viale Marconi 5 I-24044 Dalmine BG Italy

Soft querying on databases (i.e., selecting data items that partially match selection conditions) was inves-tigated on top of classical relational databases in past research works;however, constraints and limita-tions posed by relational DBMSs significantly limited the practical effects of this research. The advent of JSON as the format for representing and sharing data over the Internet, together with the birth of JSON document stores (a specific category of NoSQL databases), is now changing the panorama. In fact, the need to integrate and query large JSON data sets is now calling for novel and powerful tools for managing and integrating JSON data sets in a flexible way. At the University of Bergamo (Italy), we are devising the J-CO Framework, which is a platform-independent tool that relies on a high-level and general-purpose language named J-CO-QL': among all its features, it provides capabilities towards "soft querying" of JSON documents. However, a general-purpose language, although extremely powerful, cannot provide support for domain-specific computations that often relies on procedural algorithms. In this paper, we show how supporting user-defined functions actually empowers J-CO-QL' users towards applying soft querying on JSON data sets. user-defined functions written both in JavaScript and in Java are accepted by the J-CO-QL' Engine: in this paper, we present how to define them and the different execution performance. (c) 2023 Elsevier B.V. All rights reserved.

关键词： Soft querying on JSON data sets user-defined functions functions Fuzzy operators powered by user-defined

来源：评论

学校读者我要写书评

暂无评论

Optimization of Complex Dataflows with user-defined functions

引用

ACM COMPUTING SURVEYS 2017年第3期50卷 38-38页

作者： Rheinlaender, Astrid Leser, Ulf Graefe, Goetz Humboldt Univ Dept Comp Sci D-10099 Berlin Germany Google 10 N Livingston St Madison WI 53703 USA

In many fields, recent years have brought a sharp rise in the size of the data to be analyzed and the complexity of the analysis to be performed. Such analyses are often described as dataflows specified in declarative dataflow languages. A key technique to achieve scalability for such analyses is the optimization of the declarative programs;however, many real-life dataflows are dominated by user-defined functions (UDFs) to perform, for instance, text analysis, graph traversal, classification, or clustering. This calls for specific optimization techniques as the semantics of such UDFs are unknown to the optimizer. In this article, we survey techniques for optimizing dataflows with UDFs. We consider methods developed over decades of research in relational database systems as well as more recent approaches spurred by the popularity of Map/Reduce-style data processing frameworks. We present techniques for syntactical dataflow modification, approaches for inferring semantics and rewrite options for UDFs, and methods for dataflow transformations both on the logical and the physical levels. Furthermore, we give a comprehensive overview on declarative dataflow languages for Big Data processing systems from the perspective of their build-in optimization techniques. Finally, we highlight open research challenges with the intention to foster more research into optimizing dataflows that contain UDFs.

关键词： Dataflow optimization user-defined functions big data processing systems

来源：评论

学校读者我要写书评

暂无评论

PARALLELIZING user-defined functions IN THE ETL WORKFLOW USING ORCHESTRATION STYLE SHEETS

引用

INTERNATIONAL JOURNAL OF APPLIED MATHEMATICS AND COMPUTER SCIENCE 2019年第1期29卷 69-79页

作者： Ali, Syed Muhammad Fawad Mey, Johannes Thiele, Maik Poznan Univ Tech Fac Comp Piotrowo 2 PL-60965 Poznan Poland Trivago NV Leipzig Data Engn Bosestr 4 D-04109 Leipzig Germany Tech Univ Dresden Fac Comp Sci Helmholtzstr 10 D-01069 Dresden Germany

Today's ETL tools provide capabilities to develop custom code as user-defined functions (UDFs) to extend the expressiveness of the standard ETL operators. However, while this allows us to easily add new functionalities, it also comes with the risk that the custom code is not intended to be optimized, e.g., by parallelism, and for this reason, it performs poorly for data-intensive ETL workflows. In this paper we present a novel framework, which allows the ETL developer to choose a design pattern in order to write parallelizable code and generates a configuration for the UDFs to be executed in a distributed environment. This enables ETL developers with minimum expertise in distributed and parallel computing to develop UDFs without taking care of parallelization configurations and complexities. We perform experiments on large-scale datasets based on TPC-DS and BigBench. The results show that our approach significantly reduces the effort of ETL developers and at the same time generates efficient parallel configurations to support complex and data-intensive ETL tasks.

关键词： ETL workflow parallel ETL operators parallel algorithmic skeletons user-defined functions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：