检索结果-内蒙古大学图书馆

A new vertical fragmentation algorithm based on ant collective behavior in distributed database systems

KNOWLEDGE AND INFORMATION SYSTEMS 2012年第2期30卷 435-455页

作者： Goli, Mehdi Rankoohi, Seyed Mohammad Taghi Rouhani Shahid Beheshti Univ Elect & Comp Engn Dept Tehran Iran

Considering the existing massive volumes of data processed nowadays and the distributed nature of many organizations, there is no doubt how vital the need is for distributed database systems. In such systems, the response time to a transaction or a query is highly affected by the distribution design of the database system, particularly its methods for fragmentation, replication, and allocation data. According to the relevant literature, from the two approaches to fragmentation, namely horizontal and vertical fragmentation, the latter requires the use of heuristic methods due to it being NP-Hard. Currently, there are a number of different methods of providing vertical fragmentation, which normally introduce a relatively high computational complexity or do not yield optimal results, particularly for large-scale problems. In this paper, because of their distributed and scalable nature, we apply swarm intelligence algorithms to present an algorithm for finding a solution to vertical fragmentation problem, which is optimal in most cases. In our proposed algorithm, the relations are tried to be fragmented in such a way so as not only to make transaction processing at each site as much localized as possible, but also to reduce the costs of operations. Moreover, we report on the experimental results of comparing our algorithm with several other similar algorithms to show that ours outperforms the other algorithms and is able to generate a better solution in terms of the optimality of results and computational complexity.

关键词： Distributed database design data fragmentation Vertical fragmentation Optimal fragment numbers Minimum transaction access cost Ant clustering algorithm

来源：评论

学校读者我要写书评

暂无评论

A survey of methods for distributed machine learning

引用

PROGRESS IN ARTIFICIAL INTELLIGENCE 2013年第1期2卷 1-11页

作者： Peteiro-Barral, Diego Guijarro-Berdinas, Bertha Univ A Coruna Dept Comp Sci Campus Elvina S-N La Coruna 15071 Spain

Traditionally, a bottleneck preventing the development of more intelligent systems was the limited amount of data available. Nowadays, the total amount of information is almost incalculable and automatic data analyzers are even more needed. However, the limiting factor is the inability of learning algorithms to use all the data to learn within a reasonable time. In order to handle this problem, a new field in machine learning has emerged: large-scale learning. In this context, distributed learning seems to be a promising line of research since allocating the learning process among several workstations is a natural way of scaling up learning algorithms. Moreover, it allows to deal with data sets that are naturally distributed, a frequent situation in many real applications. This study provides some background regarding the advantages of distributed environments as well as an overview of distributed learning for dealing with "very large" data sets.

关键词： Machine learning Large-scale learning data fragmentation Distributed learning Scalability

来源：评论

学校读者我要写书评

暂无评论

A survey on deduplication systems

引用

INTERNATIONAL JOURNAL OF GRID AND UTILITY COMPUTING 2024年第2期15卷 143-159页

作者： Godavari, Amdewar Sudhakar, Chapram Natl Inst Technol Warangal Comp Sci & Engn Warangal Telangana India

With the arrival of new technological trends such as Big data and Internet of Things, tremendous amount of duplicate data is being generated. Duplicate data causes the wastage of storage capacity and degradation of performance of the storage systems. data deduplication is a storage optimisation technique that is used to eliminate duplicate data. Deploying deduplication system for primary storage or secondary storage is challenging due to extra latency incurred in deduplication processing. Apart from this, as duplicates are eliminated, deduplication affects contiguous placement of data on the disk, which is known as disk fragmentation problem. This paper gives overview of issues and solutions proposed for deploying deduplication component for primary and/or secondary storage systems with centralised or distributed approaches. Experiments are conducted using Destor tool on different data sets. The results are used to study the effect of different chunking algorithms on deduplication phases.

关键词： data fragmentation deduplication disk bottleneck

来源：评论

学校读者我要写书评

暂无评论

Publishing public transport data on the Web with the Linked Connections framework

引用

SEMANTIC WEB 2023年第4期14卷 659-693页

作者： Rojas, Julian Andres Delva, Harm Colpaert, Pieter Verborgh, Ruben Ghent Univ Imec Dept Elect & Informat Syst IDLab Ghent Belgium

Publishing transport data on the Web for consumption by others poses several challenges for data publishers. In addition to planned schedules, access to live schedule updates (e.g. delays or cancellations) and historical data is fundamental to enable reliable applications and to support machine learning use cases. However publishing such dynamic data further increases the computational burden for data publishers, resulting in often unavailable historical data and live schedule updates for most public transport networks. In this paper we apply and extend the current Linked Connections approach for static data to also support cost-efficient live and historical public transport data publishing on the Web. Our contributions include (i) a reference specification and system architecture to support cost-efficient publishing of dynamic public transport schedules and historical data;(ii) empirical evaluations on route planning query performance based on data fragmentation size, publishing costs and a comparison with a traditional route planning engine such as OpenTripPlanner;(iii) an analysis of potential correlations of query performance with particular public transport network characteristics such as size, average degree, density, clustering coefficient and average connection duration. Results confirm that fragmentation size influences route planning query performance and converges on an optimal fragment size per network. Size (stops), density and connection duration also show correlation with route planning query performance. Our approach proves to be more cost-efficient and in some cases outperforms OpenTripPlanner when supporting the earliest arrival time route planning use case. Moreover, the cost of publishing live and historical schedules remains in the same order of magnitude for server-side resources compared to publishing planned schedules only. Yet, further optimizations are needed for larger networks (>1000 stops) to be useful in practice. Additional dataset fra

关键词： Linked data Semantic Web Linked data Fragments Linked Connections public transport route planning data fragmentation

来源：评论

学校读者我要写书评

暂无评论

QUERY-PROCESSING IN DISTRIBUTED dataBASES WITH NONDISJOINT data

引用

INFORMATION SYSTEMS 1993年第7期18卷 419-427页

作者： GOYAL, P NARAYANAN, TS SADRI, F BELL NO RES LTD OTTAWAONCANADA CONCORDIA UNIV MONTREALPQCANADA

In a distributed relational database, relations are divided into disjoint fragments. These fragments are allocated to different sites in the database using some allocation scheme to improve the data retrieval time. Allocation schemes that are not constrained by other features of the DBMS will make the allocation easier to implement and provide desired performance (retrieval time). However, at present, the allocation schemes are constrained by the assumptions made by the existing query processing schemes. Most of the existing query processing schemes assume a restricted form of fragment allocation Some of these assume that a fragment is allocated to only one site, while others assume that the sets of fragments allocated to two different sites are either disjoint or the same. This paper emphasizes the importance of nondisjoint data among sites in a distributed database environment and presents query processing framework for such an allocation. Using the framework, a number of query processing schemes can be implemented. This paper also presents a heuristic query processing scheme using this framework. The heuristics presented here attempts to use the redundant data to eliminate the expensive join, I/O, and communication cost.

关键词： QUERY PROCESSING DISTRIBUTED dataBASE RELATIONAL dataBASE NONDISJOINT data data fragmentation data ALLOCATION

来源：评论

学校读者我要写书评

暂无评论

Hybrid Deduplication System-A Block-Level Similarity-Based Approach

引用

IEEE SYSTEMS JOURNAL 2021年第3期15卷 3860-3870页

作者： Godavari, Amdewar Sudhakar, Chapram Ramesh, T. Natl Inst Technol Warangal Warangal 506004 Andhra Pradesh India

Deploying deduplication for primary storage is a challenging task in view of random access patterns of I/O requests and the requirement of quick response time. Existing deduplication approaches designed for primary workloads are locality based with the assumption that I/O requests follow the locality principle. However, primary workloads in cloud systems need not necessarily follow the locality principle, and hence, the existing methods for deduplication are likely to exhibit poor response time. To provide a solution to this challenging problem, we propose and implement a hybrid deduplication system (HDS), a block-based partial deduplication system with similarity-based indexing. The proposed system applies deduplication in the background to decrease the latency and also aims at decreasing the data fragmentation. It applies similarity-based indexing to reduce high number of metadata lookups arising out of random access patterns of the requests. HDS for primary workloads is simulated in the Linux environment using three different types of FIU traces, and the effectiveness of the system is compared with full deduplication based on the parameters-metadata access overhead, average segment length, and response time. The experimental results show that the system has performed consistently better in reducing the metadata overhead and increasing the average segment length for all three sets of I/O trace data.

关键词： Indexing Metadata Time factors Cloud computing Redundancy Optimization data fragmentation deduplication disk bottleneck similarity-based indexing

来源：评论

学校读者我要写书评

暂无评论

Improving the Performance of Deduplication-Based Backup Systems via Container Utilization Based Hot Fingerprint Entry Distilling

引用

ACM TRANSACTIONS ON STORAGE 2021年第4期17卷 1-23页

作者： Zhang, Datong Deng, Yuhui Zhou, Yi Zhu, Yifeng Qin, Xiao Jinan Univ Dept Comp Sci Guangzhou 510632 Guangdong Peoples R China Columbus State Univ TSYS Sch Comp Sci 4225 Univ Ave Columbus GA 31907 USA Univ Maine Sch Elect & Comp Engn Orono ME USA Auburn Univ Dept Comp Sci & Software Engn Auburn AL 36849 USA

data deduplication techniques construct an index consisting of fingerprint entries to identify and eliminate duplicated copies of repeating data. The bottleneck of disk-based index lookup and data fragmentation caused by eliminating duplicated chunks are two challenging issues in data deduplication. Deduplication-based backup systems generally employ containers storing contiguous chunks together with their fingerprints to preserve data locality for alleviating the two issues, which is still inadequate. To address these two issues, we propose a container utilization based hot fingerprint entry distilling strategy to improve the performance of deduplication-based backup systems. We divide the index into three parts: hot fingerprint entries, fragmented fingerprint entries, and useless fingerprint entries. A container with utilization smaller than a given threshold is called a sparse container. Fingerprint entries that point to non-sparse containers are hot fingerprint entries. For the remaining fingerprint entries, if a fingerprint entry matches any fingerprint of forthcoming backup chunks, it is classified as a fragmented fingerprint entry. Otherwise, it is classified as a useless fingerprint entry. We observe that hot fingerprint entries account for a small part of the index, whereas the remaining fingerprint entries account for the majority of the index. This intriguing observation inspires us to develop a hot fingerprint entry distilling approach named HID. HID segregates useless fingerprint entries from the index to improve memory utilization and bypass disk accesses. In addition, HID separates fragmented fingerprint entries to make a deduplication-based backup system directly rewrite fragmented chunks, thereby alleviating adverse fragmentation. Moreover, HID introduces a feature to treat fragmented chunks as unique chunks. This feature compensates for the shortcoming that a Bloom filter cannot directly identify certain duplicated chunks (i.e., the fragmented chun

关键词： data deduplication data backup storage system disk bottleneck data fragmentation

来源：评论

学校读者我要写书评

暂无评论

Heterogeneous Information Sharing of Sensor Information in Contested Environments 8

Heterogeneous Information Sharing of Sensor Information in C...

引用

8th Conference on Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR

作者： Wampler, Jason A. Hsieh, Chien Toth, Andrew Sheatsley, Ryan INCA Engn 127 Accident Bittinger Rd Accident MD 21520 USA ICF 9300 Lee Highway Fairfax VA 22031 USA US Army Res Lab 2800 Powder Mill Rd Adelphi MD 20783 USA

ISBN: (数字)9781510608825

ISBN: (纸本)9781510608818;9781510608825

The inherent nature of unattended sensors makes these devices most vulnerable to detection, exploitation, and denial in contested environments. Physical access is often cited as the easiest way to compromise any device or network. A new mechanism for mitigating these types of attacks developed under the Assistant Secretary of Defense for Research and Engineering, ASD(R& E) project, "Smoke Screen in Cyberspace", was demonstrated in a live, over- the- air experiment. Smoke Screen encrypts, slices up, and disburses redundant fragments of files throughout the network. Recovery is only possible after recovering all fragments and attacking/denying one or more nodes does not limit the availability of other fragment copies in the network. This experiment proved the feasibility of redundant file fragmentation, and is the foundation for developing sophisticated methods to blacklist compromised nodes, move data fragments from risks of compromise, and forward stored data fragments closer to the anticipated retrieval point. This paper outlines initial results in scalability of node members, fragment size, file size, and performance in a heterogeneous network consisting of the Wireless Network after Next (WNaN) radio and Common Sensor Radio (CSR).

关键词： heterogeneous sensor contested wireless distributed data redundancy data fragmentation

来源：评论

学校读者我要写书评

暂无评论

A Day's Work on Facebook and Other Collaborative Trends in Animal Welfare 19

A Day's Work on Facebook and Other Collaborative Trends in A...

引用

22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW)

作者： Munch, Mikako Georgia Inst Technol Atlanta GA 30332 USA

ISBN: (纸本)9781450366922

Animal shelters generate an enormous amount of data and require extensive information and communication management, however very little focus has been given to this field through the CSCW lens. The aim of this autoethnographic study is to portray the different events and resulting artifacts from daily shelter tasks using a combination of workflow modeling and customer journey mapping techniques. The Shelter Touchpoint Workflow sheds light on unique trends in the collaborative environment of animal welfare. Thus, demonstrating room for the CSCW community to help develop solutions to alleviate shortcomings of legacy technology and other tools that have been adapted to fit into their daily tasks. The two specific insights we are examining are the data fragmentation and siloing of information about an animal, as well as the contribution of social media to compassion fatigue.

关键词： Cooperative work Workflow Nonprofit Animal Welfare Social Computing data fragmentation

来源：评论

学校读者我要写书评

暂无评论

A Modal Logic Formalism for Distributed and Parallel Knowledge Bases

引用

Parallel Algorithms and Applications 1993年第1期1卷 11-27页

作者： Levene, Mark Loizou, George Department of Computer Science University College London London WCIE 6BT Gower Street United Kingdom Department of Computer Science Birkbeck College London WCIE 7HX Malet Street United Kingdom

Knowledge bases are currently being investigated by database researchers in order to extend the expressiveness of the relational model to include deductive capabilities in the form of rules. The motivation behind this paper is to present a formalism which extends the relational approach to homogeneous distributed databases to the knowledge base paradigm. The basis of our formalism of distributed knowledge bases is the S5 modal logic system. We define an extensional distributed database as a modal structure and an intentional distributed database as a datalog^k logic program. We interpret a modal structure, (W, V), as a network of user sites, W, together with a valuation function, V, which assigns a database to each site w e W. datalog^k extends datalog + negation by incorporating into it the modal knowledge operator, K. Knowing in our context can be viewed as data replication, and possibility can be viewed as data fragmentation. We define the model-theoretic semantics of datalog^k via implication in a proper subset of first-order modal logic and its operational semantics in terms of two inflationary fixpoint operators, the local fixpoint and the global fixpoint operators. It is shown that the operational semantics of datalog^k coincides with its model-theoretic semantics and that datalog^k queries are computable in polynomial time. We also demonstrate that datalog^k can be implemented in a parallel knowledge base environment over a shared-nothing architecture for a multiprocessor database. © 1993, Taylor & Francis Group, LLC. All rights reserved.

关键词： data fragmentation data replication datalog Distributed knowledge base Global inflationary fixpoint Local inflationary fixpoint Modal logic Parallel knowledge base Shared-nothing architecture

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：