检索结果-内蒙古大学图书馆

distributed rrays: an algebra for generic distributed query processing

distributed AND PARALLEL DATABASES 2021年第4期39卷 1009-1064页

作者： Gueting, Ralf Hartmut Behr, Thomas Nidzwetzki, Jan Kristof Fernuniv Fac Math & Comp Sci D-58084 Hagen Germany

We propose a simple model for distributed query processing based on the concept of a distributed array. Such an array has fields of some data type whose values can be stored on different machines. It offers operations to manipulate all fields in parallel within the distributed algebra. The arrays considered are one-dimensional and just serve to model a partitioned and distributed data set. distributed arrays rest on a given set of data types and operations called the basic algebra implemented by some piece of software called the basic engine. It provides a complete environment for query processing on a single machine. We assume this environment is extensible by types and operations. Operations on distributed arrays are implemented by one basic engine called the master which controls a set of basic engines called the workers. It maps operations on distributed arrays to the respective operations on their fields executed by workers. The distributed algebra is completely generic: any type or operation added in the extensible basic engine will be immediately available for distributed query processing. To demonstrate the use of the distributed algebra as a language for distributed query processing, we describe a fairly complex algorithm for distributed density-based similarity clustering. The algorithm is a novel contribution by itself. Its complete implementation is shown in terms of the distributed algebra and the basic algebra. As a basic engine the Secondo system is used, a rich environment for extensible query processing, providing useful tools such as main memory M-trees, graphs, or a DBScan implementation.

关键词： distributed database distributed query processing Density-based similarity clustering

来源：评论

学校读者我要写书评

暂无评论

Detection of Data Leaks through Large Scale distributed query processing using Machine Learning

引用

INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS 2021年第12期12卷 284-290页

作者： Kiranmai, M. V. S. V. Haritha, D. JNTUK UCEK Dept CSE Kakinada India

With the growth in the distributed data processing and data being the fuel for each of the processes, the query processes of the data are expected to be significantly lower. Hence, the distribution of the data is highly expected and during the distributing of the data, the chances for data leakage increases to a significant extend. The data leakage problems are not generally caused by intentional errors, rather this is caused by the higher visibility of the data over multiple clusters. Henceforth, the detection process is also very critical. Many of the parallel research attempts have demonstrated various methods for the detection and as well as the prevention methods. The works in the direction of the detection of the data leaks are highly dependent either on the historical information of the leaks or depends on the contextual importance of the data. In both the cases, the outcomes of the detection process accuracy cannot be ensured. In the other hand, the preventive measures can also turn into a reactive process for detection by reversing the principles proposed in these research outcomes, but the computational complexities are significantly higher. Thus, this work proposes a novel strategy for detection of the data leakages after the data distribution during the query processing events. This work proposes an initial Occurrence Based Rule Set Extraction method using Adaptive Threshold for generating the rulesets, further for reducing the time complexity and reducing the loss of dataset attribute information, this work introduces yet another algorithm for Dynamic Inference-based Rule Set Reduction. After the inferences are generated, finally this work deploys the Attribute Subset Equivalence-based Leak Detection mechanism for final detection of the clusters with data leaks. This work demonstrates nearly 89% accuracy for the detection process.

关键词： distributed query processing distributed data leak data leak detection attribute subset equivalence dynamic inference adaptive threshold model introduction

来源：评论

学校读者我要写书评

暂无评论

distributed query processing in the presence of blank nodes

引用

SEMANTIC WEB 2017年第6期8卷 1001-1021页

作者： Stolpe, Audun Halvorsen, Jonas Norwegian Def Res Estab FFI Postboks 25 N-2027 Kjeller Norway Univ Oslo Oslo Norway

This paper demonstrates that the presence of blank nodes in RDF data represents a problem for distributed processing of SPARQL queries. It is shown that the usual decomposition strategies from the literature will leak information - even when information derives from a single source. It is argued that this leakage, and the proper reparational measures, need to be accounted for in a formal semantics. To this end a set semantics for SPARQL is generalized with a parameter representing execution contexts. This makes it possible to keep tabs on the naming of blank nodes across execution contexts, which in turn makes it possible to articulate a decomposition strategy that is provably sound and complete wrt. any selection of RDF sources even when blank nodes are allowed. Alas, this strategy is not computationally tractable. However, there are ways of utilizing knowledge about the sources, if one has it, that will help considerably.

关键词： SPARQL federation distributed query processing logical foundation completeness decomposition scheme

来源：评论

学校读者我要写书评

暂无评论

SPARQL distributed query processing over Linked Data

SPARQL Distributed Query Processing over Linked Data

引用

作者： Abdoul Macina Universite Cote d'Azur

学位级别：博士

Driven by the Semantic Web standards, an increasing number of RDF data sources are published and connected over the Web by data providers, leading to a large distributed linked data network. However, exploiting the wealth of these data sources is very challenging for data consumers considering the data distribution, their volume growth and data sources autonomy. In the Linked Data context, federation engines allow querying these distributed data sources by relying on distributed query processing (DQP) techniques. Nevertheless, a naive implementation of the DQP approach may generate a tremendous number of remote requests towards data sources and numerous intermediate results, thus leading to costly network communications. Furthermore, the distributed query semantics is often overlooked. query expressiveness, data partitioning, and data replication are other challenges to be taken into account. To address these challenges, we first proposed a distributed query processing semantics which preserves the SPARQL language expressiveness. Afterwards, we presented several strategies for a federated query engine that transparently addresses distributed data sources. Firstly, we proposed a federated query semantics on top of RDF and SPARQL standards, and following this, we specified the semantics of federated SPARQL queries on top of the standard SPARQL semantics through a set of rewrite rules relying on service clause. Secondly, we proposed both static and dynamic strategies for the main steps of DQP approach in order to allow transparent and efficient distributed and autonomous data sources querying, and therefore enhance the federated query processing. The static optimizations rely on the results of the source selection step in which we proposed a Sampling query-Based approach. On the one hand, based on these results we proposed a Hybrid BGP-Triple query rewriting approach which addresses both horizontal and vertical data partitions and reduces the query engines workload. On

关键词： Semantic Web Web of Data Linked Data Linked Open Data Data Integration distributed query processing Federated query evaluation SPARQL query Optimization

来源：评论

学校读者我要写书评

暂无评论

Optimizing Communication Cost in distributed query processing

Optimizing Communication Cost in Distributed Query Processin...

引用

作者： Abdeslem Belghoul Universite Clermont Auvergne

学位级别：博士

In this thesis, we take a complementary look to the problem of optimizing the time for commu- nicating query results in distributed query processing, by investigating the relationship between the communication time and the middleware configuration. Indeed, the middleware determines, among others, how data is divided into batches and messages before being communicated over the network. Concretely, we focus on the research question: given a query Q and a network environment, what is the best middleware configuration that minimizes the time for transferring the query result over the network? To the best of our knowledge, the database research community does not have well-established strategies for middleware tuning. We present first an intensive experimental study that emphasizes the crucial impact of mid- dleware configuration on the time for communicating query results. We focus on two middleware parameters that we empirically identified as having an important influence on the communication time: (i) the fetch size F (i. e., the number of tuples in a batch that is communicated at once to an application consuming the data) and (ii) the message size M (i. e., the size in bytes of the middleware buffer, which corresponds to the amount of data that can be communicated at once from the middleware to the network layer; a batch of F tuples can be communicated via one or several messages of M bytes). Then, we describe a cost model for estimating the communication time, which is based on how data is communicated between computation nodes. Precisely, our cost model is based on two crucial observations: (i) batches and messages are communicated differently over the network: batches are communicated synchronously, whereas messages in a batch are communicated in pipeline (asynchronously), and (ii) due to network latency, it is more expensive to communicate the first message in a batch compared to any other message that is not the first in its batch. We propose an effective strate

关键词： middleware distributed query processing communication cost model fetch size message size optimizing communication cost

来源：评论

学校读者我要写书评

暂无评论

Efficient distributed query processing

引用

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2016年第3期13卷 1230-1246页

作者： Kolcun, Roman Boyle, David E. McCann, Julie A. Imperial Coll London Dept Elect & Elect Engn London SW7 2AZ England Imperial Coll London Dept Comp London SW7 2AZ England

A variety of wireless networks, including applications of Wireless Sensor Networks, Internet of Things, and Cyber-physical Systems, increasingly pervade our homes, retail, transportation systems, and manufacturing processes. Traditional approaches communicate data from all sensors to a central system, and users (humans or machines) query this central point for results, typically via the web. As the number of deployed sensors, and thus generated data streams, is increasing exponentially, this traditional approach may no longer be sustainable or desirable in some application contexts. Therefore, new approaches are required to allow users to directly interact with the network, for example, requesting data directly from sensor nodes. This is difficult, as it requires every node to be capable of point-to-point routing, in addition to identifying a subset of nodes that can fulfil a user's query. This paper presents DRAGON, a platform that allows any node in the network to identify all nodes that satisfy user queries, i.e., request data from nodes, and relay the result to the user. The DRAGON platform achieves this in a fully distributed way. No central orchestration is required, network overheads are low, and latency is improved over existing comparable methods. DRAGON is evaluated on networks of various topologies and different network densities. It is compared with the state-of-the-art algorithms based on summary trees, like Innet and SENS-Join. DRAGON is shown to outperform these approaches up to 88% in terms of network traffic required, also a proxy for energy efficiency, and 84% in terms of processing delay. Note to Practitioners-This work is motivated by the continuing deluge of constrained, wirelessly connected sensing and control devices. Networks of communicable sensors and actuators are finding increased applicability across a range of industries and application scenarios. They are often thought of as a subset of the '"nternet of Things." However, due to the inh

关键词： Autonomous and automated sensing and control systems cyber-physical systems distributed query processing distributed storage routing algorithm wireless sensor networks

来源：评论

学校读者我要写书评

暂无评论

Generating Optimal query Plans for distributed query processing using Teacher-Learner Based Optimization 11

Generating Optimal Query Plans for Distributed Query Process...

引用

11th International Conference on Data Mining and Warehousing (ICDMW)

作者： Mishra, Vikash Singh, Vikram Natl Inst Technol Kurukshetra 136119 Haryana India

Modern day's queries are posed on database spread across the globe, this may impose a challenge on processing queries efficiently, and a strategy is required to generate optimal query plans. In distributed relational database systems, due to partitioning or replication on relations at multiple sites, the relations required by a query to answer, may be stored at multiple sites. This leads to an exponential increase in the number of possible equivalent alternatives or query plans for a user query. Though it is not computationally reasonable to explore exhaustively all possible query plans in a large search space, the query plan with most cost-effective option for query processing is measured necessary and must be generated for a given query. In this paper, an attempt has been made to generate such optimal query plans using parameter less optimization technique Teaching-Learner based Optimization (TLBO). The TLBO algorithm was observed to go one better than the other optimization algorithms for the multi-objective unconstrained and constrained benchmark problems. Experimental comparisons of this algorithm with the multi-objective GA based distributed query plan generation algorithm shows that for higher number of relations, the TLBO based algorithm is able to generate comparatively better quality Top-K query plans. (C) 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).

关键词： Aggregation based genetic algorithm distributed query processing Top-K Teacher learner based optimization VEGA

来源：评论

学校读者我要写书评

暂无评论

Generating Optimal query Plans for distributed query processing using Teacher-Learner Based Optimization

引用

Procedia Computer Science 2015年 54卷 281-290页

作者： Vikash Mishra Vikram Singh National Institute of Technology Kurukshetra Haryana 136 119 India

关键词： Aggregation based genetic algorithm distributed query processing Top-K Teacher learner based optimization VEGA.

来源：评论

学校读者我要写书评

暂无评论

Improved Centralized XML query processing Using distributed query Workload

引用

IEEE ACCESS 2021年 9卷 29127-29142页

作者： Subramaniam, Samini Haw, Su-Cheng Soon, Lay-Ki Multimedia Univ Fac Comp & Informat Cyberjaya 63000 Malaysia Monash Univ Malaysia Sch Informat Technol Bandar Sunway 47500 Malaysia

eXtensible Markup Language (XML) is used widely to transfer data among a wide variety of systems. Due to an increase in query workloads and management of larger datasets, centralized processing is no longer feasible for XML query processing. To address this issue, we propose a technique that improves XML query processing through query workload distribution. An effective distributed XML query processing can be affected by several criteria such as indexing, fragmentation, distribution strategy, and well as the query handling in the distributed servers. However, we believe that an efficient labeling mechanism and an inexpensive centralized query processors or a pruning method at dedicated servers contribute greatly to the overall performance of a distributed query processor. In this paper, we present an effective centralized pruning technique that is adopted into our proposed distributed XML query processing technique to process XML queries robustly. Experimental evaluations showed that the proposed distributed query processor superseded the performance of centralized query processor.

关键词： XML query processing Servers Indexing Vegetation Program processors Merging XML labeling XML indexing XML query processing XML database centralized query processing distributed query processing

来源：评论

学校读者我要写书评

暂无评论

Connectivity-Oriented Property Graph Partitioning for distributed Graph Pattern query processing

引用

Proceedings of the ACM on Management of Data 2024年第6期2卷 1-26页

作者： Min Shi Peng Peng Xu Zhou Jiayu Liu Guoqing Xiao Kenli Li Hunan University Changsha Hunan China

Graph pattern query is a powerful tool for extracting crucial information from property graphs. With the exponential growth of sizes, property graphs are typically divided into multiple subgraphs (referred to as partitions) and stored across various sites in distributed environments. Existing graph partitioning methods have not been efficiently optimized for pattern queries, resulting in numerous query matches across multiple partitions, called crossing matches. Identifying these matches requires much inter-partition communication, which is the primary performance bottleneck in distributed query processing. To address this issue, this paper introduces a novel connectivity-oriented relationship-disjoint partitioning method, namely RCP (Relationship Connectivity Partitioning), aimed at enhancing the efficiency of graph pattern query processing by reducing crossing matches. By employing each weakly connected component of the subgraph, which is induced by different relationship labels, as a basic unit of partition, RCP ensures that matches for both variable-length path and labeled graph pattern queries are not crossing matches. Here, variable-length path and labeled graph pattern are two common components in graph pattern queries to identify paths meeting specific label constraints and retrieve subgraphs with consistent relationship types, respectively. Moreover, in the query processing phase, we further demonstrate that all graph pattern queries, belonging to these two basic queries or their extensions, can be executed independently under RCP, thereby avoiding crossing matches. In experiments, we implemented two prototype distributed property graph systems based on Neo4j and JanusGraph, which use declarative and functional query language, respectively. Experimental results on billion-scale datasets demonstrate that our approach brings a performance improvement of nearly two orders of magnitude over state-of-the-art partitioning methods.

关键词： distributed query processing graph pattern query property graph partitioning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：