检索结果-内蒙古大学图书馆

Garbage collecting the internet: A survey of distributed garbage collection

ACM COMPUTING SURVEYS 1998年第3期30卷 330-373页

作者： Abdullahi, SE Ringwood, GA Queen Mary Univ London London WC1E 7HU England

Internet programming languages such as Java present new challenges to garbage-collection design. The spectrum of garbage-collection sehema for linked structures distributed over a network are reviewed here. distributed garbage collectors are classified first because they evolved from single-address-space collectors, This taxonomy is used as a framework to explore distribution issues: locality of action, communication overhead and indeterministic communication latency.

关键词： automatic storage reclamation distributed object-oriented management distributed file systems distributed memories memory management network communication distributed object-oriented databases reference counting

来源：评论

学校读者我要写书评

暂无评论

Replicated server placement with QoS constraints

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed systems 2006年第10期17卷 1151-1162页

作者： Rodolakis, Georgios Siachalou, Stavroula Georgiadis, Leonidas INRIA Rocquencourt Hipercom Project F-78153 Le Chesnay France Aristotle Univ Thessaloniki Dept Telecommun Thessaloniki 54124 Greece

The network planning problem of placing replicated servers with QoS constraints is considered. Each server site may consist of multiple server types with varying capacities and each site can be placed in any location among those belonging to a given set. Each client can be served by more than one location as long as the round-trip delay of data requests satisfies predetermined upper bounds. Our main focus is to minimize the cost of using the servers and utilizing the link bandwidth, while serving requests according to their delay constraint. This is an NP-hard problem. A pseudopolynomial and a polynomial algorithm that provide guaranteed approximation factors with respect to the optimal for the problem at hand are presented.

关键词： algorithm design and analysis constrained optimization applications distributed file systems client/server

来源：评论

学校读者我要写书评

暂无评论

Specializing Data Access in a distributed file System (Generative Pearl) 2024

Specializing Data Access in a Distributed File System (Gener...

引用

Proceedings of the 23rd ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences

作者： Pratyush Das Anxhelo Xhebraj Tiark Rompf Purdue University West Lafayette USA

ISBN: (纸本)9798400712111

We propose DDLoader, a system that embeds information about data partitioning and data distribution in distributed file systems using a metaprogramming framework. We demonstrate that this technique has practical benefits for building applications that interact with distributed file system applications. By using metaprogramming, we bring the traditional benefits of staging, such as partial evaluation to the domain of distributed file system access. Furthermore, this approach also fuses the data access code with the computation code, allowing end-to-end optimization. We test our framework on a real-world use case and show that this approach allows users to generate code that performs faster than traditional data loading by up to 12.56x on a single thread and even more when parallelized.

关键词： distributed file systems

来源：评论

学校读者我要写书评

暂无评论

Cost-based Data Prefetching and Scheduling in Big Data Platforms over Tiered Storage systems

引用

ACM TRANSACTIONS ON DATABASE systems 2023年第4期48卷 1-40页

作者： Herodotou, Herodotos Kakoulli, Elena Cyprus Univ Technol 30 Arch Kyprianos Str CY-3036 Limassol Cyprus Neapolis Univ Pafos 2 Danais Ave CY-8042 Pafos Cyprus

The use of storage tiering is becoming popular in data-intensive compute clusters due to the recent advancements in storage technologies. The Hadoop distributed file System, for example, now supports storing data in memory, SSDs, and HDDs, while OctopusFS and hatS offer fine-grained storage tiering solutions. However, current big data platforms (such as Hadoop and Spark) are not exploiting the presence of storage tiers and the opportunities they present for performance optimizations. Specifically, schedulers and prefetchers will make decisions only based on data locality information and completely ignore the fact that local data are now stored on a variety of storage media with different performance characteristics. This article presents Trident, a scheduling and prefetching framework that is designed to make task assignment, resource scheduling, and prefetching decisions based on both locality and storage tier information. Trident formulates task scheduling as aminimum cost maximummatching problem in a bipartite graph and utilizes two novel pruning algorithms for bounding the size of the graph, while still guaranteeing optimality. In addition, Trident extends YARN's resource request model and proposes a new storage-tier-aware resource scheduling algorithm. Finally, Trident includes a cost-based data prefetching approach that coordinates with the schedulers for optimizing prefetching operations. Trident is implemented in both Spark and Hadoop and evaluated extensively using a realistic workload derived from Facebook traces as well as an industry-validated benchmark, demonstrating significant benefits in terms of application performance and cluster efficiency.

关键词： distributed file systems tiered storage data prefetching task scheduling

来源：评论

学校读者我要写书评

暂无评论

HBA: distributed metadata management for large cluster-based storage systems

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed systems 2008年第6期19卷 750-763页

作者： Zhu, Yifeng Jiang, Hong Wang, Jun Xian, Feng Univ Maine Dept Elect & Comp Engn Orono ME 04473 USA Univ Nebraska Dept Comp Sci & Engn Lincoln NE 68588 USA Univ Cent Florida Sch Elect Engn & Comp Sci Orlando FL 32816 USA

An efficient and distributed scheme for file mapping or file lookup is critical in decentralizing metadata management within a group of metadata servers. This paper presents a novel technique called Hierarchical Bloom Filter Arrays (HBA) to map filenames to the metadata servers holding their metadata. Two levels of probabilistic arrays, namely, the Bloom filter arrays with different levels of accuracies, are used on each metadata server. One array, with lower accuracy and representing the distribution of the entire metadata, trades accuracy for significantly reduced memory overhead, whereas the other array, with higher accuracy, caches partial distribution information and exploits the temporal locality of file access patterns. Both arrays are replicated to all metadata servers to support fast local lookups. We evaluate HBA through extensive trace-driven simulations and implementation in Linux. Simulation results show our HBA design to be highly effective and efficient in improving the performance and scalability of file systems in clusters with 1,000 to 10,000 nodes (or superclusters) and with the amount of data in the petabyte scale or higher. Our implementation indicates that HBA can reduce the metadata operation time of a single-metadata-server architecture by a factor of up to 43.9 when the system is configured with 16 metadata servers.

关键词： distributed file systems file system management metadata management Bloom filter

来源：评论

学校读者我要写书评

暂无评论

Building large, complex, distributed safety-critical operating systems

引用

REAL-TIME systems 1997年第3期13卷 277-302页

作者： Wedde, HF Lind, JA Informatik III University of Dortmund Dortmund Germany

Safety-critical systems typically operate in unpredictable environments. Requirements for safety and reliability are in conflict with those for real-time responsiveness. Due to unpredictable environmental needs there is no static trade-off between measures to accommodate the conflicting objectives. Instead every feature or operating system service has to be adaptive. Finally, for any design problem, there cannot be any closed-form (formal) approach taking care at the same time of(external) time constraints or deadlines, and synchronization requirements in distributed design. The reason is that these two aspects are causally independent. -In this situation we worked out a heuristic experimental, performance-driven and performance-based methodology that allows in an educated way to start with a coarse system model, with accurate logical expectations regarding its behavior. Through experiments these expectations are validated. If they are found to successfully stand the tests extended expectations and model features are generated for refining the previous design as well as its performance criteria. The refinement is done in such a way that the previous experimental configurations are extreme model cases or data profiles which both logically and experimentally are to reproduce the behavior of the previous modeling step. Thus the novel performance aspects or tendencies could then unambiguously be attributed to the influences of the refined model features. We termed this methodology Incremental Experimentation. As a general methodology it relies on a principle of comparative performance studies rather than on realistic data for narrow application ranges. The paper describes how we applied a 5-step design and refinement procedure for developing, analyzing, and evaluating our distributed operating system MELODY that exhibits novel services for supporting real-time and safety-critical applications in unpredictable environments. Experimental set-ups and theme-related findings

关键词： distributed real-lime systems distributed operating systems distributed file systems safety-critical systems performance-driven design comparative performance distributed task and resource scheduling

来源：评论

学校读者我要写书评

暂无评论

A congestion-aware and robust multicast protocol in SDN-based data center networks

引用

JOURNAL OF NETWORK AND COMPUTER APPLICATIONS 2017年第Oct.期95卷 105-117页

作者： Zhu, Tingwei Feng, Dan Wang, Fang Hua, Yu Shi, Qingyu Xie, Yanwen Wan, Yong Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect Key Lab Informat Storage Syst Sch Comp Sci & TechnolMinist Educ China Wuhan Hubei Peoples R China Jingchu Univ Technol Comp Engn Coll Jingmen Peoples R China

Continuously enriched distributed systems in data centers generate much network traffic in push-style one-to many group mode, raising new requirements for multicast transport in terms of efficiency and robustness. Existing reliable multicast solutions, which suffer from low robustness and inefficiency in either host-side protocols or multicast routing, are not suitable for data centers. In order to address the problems of inefficiency and low robustness, we present a sender-initiated, efficient, congestion-aware and robust reliable multicast solution mainly for small groups in SDN-based data centers, called MCTCP. The main idea behind MCTCP is to manage the multicast groups in a centralized manner, and reactively schedule multicast flows to active and low utilized links, by extending TCP as the host-side protocol and managing multicast groups in the SDN-controller. The multicast spanning trees are calculated and adjusted according to the network status to perform a better allocation of resources. Our experiments show that, MCTCP can dynamically bypass the congested and failing links, achieving high efficiency and robustness. As a result, MCTCP outperforms the state-of-the-art reliable multicast schemes. Moreover, MCTCP improves the performance of data replication in HDFS (Hadoop distributed file System) compared with the original and TCP-SMO (an alternative reliable multicast scheme) based ones, e.g., achieves up to 1.5x and 1.1x improvements in terms of throughput, respectively.

关键词： Reliable multicast protocol Software-defined networking Data center networks distributed file systems Data replication

来源：评论

学校读者我要写书评

暂无评论

A COMBINED METHOD FOR MAINTAINING LARGE INDEXES IN MULTIPROCESSOR MULTIDISK ENVIRONMENTS

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 1994年第3期6卷 479-496页

作者： MATSLIACH, G SHMUELI, O Department of Computer Science Haifa Israel

Consider the problem of maintaining large indices (or secondary memory indices) in a multiprocessor multidisk environment in which each processor has a dedicated secondary memory (one disk or more). The processors either reside in the same site and communicate via shared memory, or reside in different sites and communicate via a local broadcast network. The straightforward method (SFM) for maintaining such an index, which is commonly called declustering, is to partition the index records equally among the processors, each of which maintains its part of the index in a local B+-tree. In prior work, we have presented another method (called TDB) in which all processors together implement a ''wide'' B+-tree. We showed by an approximate analysis in a different study that there are settings in which the second method is better than the first method, and vice versa. In this paper, we present a new method that combines the ideas underlying SFM and TDB. In tightly coupled environments, the combined method outperforms both SFM and TDB in almost all practical settings (in many settings by more than 30%. This is shown by an approximate analysis and verified by simulations. Note that CDM's approach can improve performance in database systems that use RAID.

关键词： DATA STRUCTURES file systems distributed file systems distributed INDEXES B+-TREES MULTIPROCESSOR ENVIRONMENTS

来源：评论

学校读者我要写书评

暂无评论

Dynamic Virtual Chunks: On Supporting Efficient Accesses to Compressed Scientific Data

引用

IEEE TRANSACTIONS ON SERVICES COMPUTING 2016年第1期9卷 96-109页

作者： Zhao, Dongfang Qiao, Kan Yin, Jian Raicu, Ioan IIT Dept Comp Sci Chicago IL 60616 USA Pacific NW Natl Lab Div Math & Comp Sci Richland WA 99354 USA Argonne Natl Lab Div Comp Sci 9700 S Cass Ave Argonne IL 60439 USA

Data compression could ameliorate the I/O pressure of data-intensive scientific applications. Unfortunately, the conventional wisdom of naively applying data compression to the file or block brings the dilemma between efficient random accesses and high compression ratios. file-level compression barely supports efficient random accesses to the compressed data: any retrieval request need trigger the decompression from the beginning of the compressed file. Block-level compression provides flexible random accesses to the compressed blocks, but introduces extra overhead when applying the compressor to each and every block that results in a degraded overall compression ratio. This paper extends our prior work that introduces virtual chunks offering efficient random accesses to the compressed scientific data without sacrificing the compression ratio. Virtual chunks are logical blocks pointed at by appended references without breaking the physical continuity of the file content. These references allow the decompression to start from an arbitrary position (efficient random accesses), while no per-block overhead is introduced because the file's physical entirety is retained (high compression ratio). One limitation of virtual chunk is it only supports static references. This paper presents the algorithms, analysis, and evaluations of dynamic virtual chunks to deal with the cases where the references are updated dynamically.

关键词： file compression distributed file systems parallel file systems big data data-intensive computing scientific computing

来源：评论

学校读者我要写书评

暂无评论

PERFORMANCE ANALYSIS OF CLIENT-SERVER STORAGE-systems

引用

IEEE TRANSACTIONS ON COMPUTERS 1992年第11期41卷 1442-1452页

作者： DRAKOPOULOS, E MERGES, MJ AT&T Bell Lab. Naperville IL USA

This paper discusses the performance analysis of a system in which storage is provided by a file server node on a network of client processors. The server controls a hierarchy of storage devices, and provides all the necessary file management functions. The clients are large mainframe systems that have their own local storage devices. The movement of files among the various storage levels is managed by a hierarchical storage management system. We develop and solve a closed queueing network model of the client-server storage system, which is used to analyze the performance of the system for various file movement criteria, and to study the tradeoffs between recalling a file and accessing it by a distributed file system. Several network topologies (Ethernet, Token ring, Channel-to-Channel) and their effect on performance are considered.

关键词： CLOSED QUEUING NETWORKS distributed file systems distributed STORAGE systems file MIGRATION LOCAL AREA NETWORKS

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：