检索结果-内蒙古大学图书馆

zCeph: Design and implementation of a ZNS-friendly distributed file system

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 169卷

作者： Ha, Jin Yong Son, Yongseok Samsung Elect 1 Samsungjeonja Ro Hwaseong Si 18448 Gyeonggi Do South Korea Chung Ang Univ 84 Heukseok Ro Seoul 06974 South Korea

This article presents zCeph, a ZNS-friendly distributed file system designed to efficiently utilize zoned namespace (ZNS) SSDs. Specifically, we first propose MZAllocator which enables multiple zones to be utilized simultaneously to maximize the performance of ZNS SSDs. Second, we adopt an append command to eliminate the need for synchronization in write ordering within distributed storage systems to improve scalability. Third, we present zBlueFS, a ZNS-aware user-level file system based on BlueFS to update the metadata on the ZNS SSD without a conventional SSD. Finally, we propose a delta write technique, DeltaWriter, which writes only a modified part of the metadata (i.e., onode) to reduce read-modify-write overhead whenever the metadata are updated. We implement zCeph with four techniques based on Ceph, an open-source distributed file system. Further, we evaluate zCeph on a pair of 48-core machines with ZNS SSDs using micro and macro benchmarks, and the results reveal that zCeph improves performance by up to 4.2x and 8.8x compared with Ceph, respectively.

关键词： distributed file system Object storage Zoned namespace solid state drive (ZNS SSD) I/O Performance Parallelism

来源：评论

学校读者我要写书评

暂无评论

distributed file system virtualization techniques supporting on-demand Virtual Machine environments for grid computing

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2006年第1期9卷 45-56页

作者： Zhao, Ming Zhang, Jian Figueiredo, Renato J. Univ Florida Dept Elect & Comp Engn Adv Comp & Informat Syst Lab Gainesville FL 32611 USA

This paper presents a data management solution which allows fast Virtual Machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in Grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: (1) it provides on-demand cross-domain access to VM state for unmodified VM monitors;(2) it enables private file system channels for VM instantiation by secure tunneling and session-key based authentication;(3) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models;and (4) it leverages application-specific meta-data associated with files to expedite data transfers. The paper reports on its performance in wide-area setups using VMware-based VMs. Results show that the solution delivers performance over 30% better than native NFS and with warm caches it can bring the application-perceived overheads below 10% compared to a local-disk setup. The solution also allows a VM with 1.6 GB virtual disk and 320 MB virtual memory to be cloned within 160 seconds for the first clone and within 25 seconds for subsequent clones.

关键词： distributed file system virtual machine grid computing

来源：评论

学校读者我要写书评

暂无评论

distributed file system: Efficiency Experiments for Data Access and Communication

Distributed File System: Efficiency Experiments for Data Acc...

引用

4th International Conference on Networked Computing and Advanced Information Management

作者： Upadhyaya, Bipin Azimov, Fahriddin Doan, Thanh Tran Choi, Eunmi Kim, SangBum Kim, Pilsung Kookmin Univ Sch Business IT Seoul South Korea SK Telecom Convergence & nternet R&D Ctr Seoul South Korea

ISBN: (纸本)9780769533223

In a distributed file system, operations such as read, write, send and receive should be efficient and must increase the overall performance of the system. Especially every remote read and write operation we encounter network and secondary storage latencies. In order to reduce this latency time and improve the performance of distributed file system, we analyze the problem of selecting application-level buffer size for the Network communication and Disk Input/Output operations. A number of experiments were carried out to identify the optimal size for our application-level Buffer.

关键词： distributed file system Buffer management Disk Input/Output Performance I/O tuning

来源：评论

学校读者我要写书评

暂无评论

distributed file system for NDN: an IoT Application

Distributed File System for NDN: an IoT Application

引用

International Conference on Selected Topics in Mobile and Wireless Networking (MoWNeT)

作者： Dongo, Junior Atik, Youssef Mahmoudi, Charif Mourlin, Fabrice Univ Paris Est Creteil France Quattro IT Paris France

ISBN: (纸本)9781538642481

Named Data Networking (NDN) has been introduced as a Future Internet Architecture. Adopting this new networking architecture implies a change in the way today applications are developed. NDN brings a transition from a host centric communication to an information centric communication, where data are the masterpiece. As Internet of Things (IoT) is a source of tremendous quantity of data shared and processed on the network, a new approach is needed to enable a smooth and optimized data sharing among the IoT devices. This paper introduces an approach of a network optimized, content oriented, and fully distributed file system that enables IoT devices and processing hosts to share both the raw data and the generated results.

关键词： Internet of Things Big data Named Data Networking distributed file system Replication Strategy Failover

来源：评论

学校读者我要写书评

暂无评论

distributed file system with Online Multi-Master Replicas

Distributed File System with Online Multi-Master Replicas

引用

2nd Eastern European Regional Conference on the Engineering of Computer Based systems (ECBS-EERC)

作者： Matejka, Lubos Safarik, Jiri Pesicka, Ladislav Univ W Bohemia Dept Comp Sci & Engn Plzen Czech Republic

ISBN: (纸本)9780769544182

Mobile devices manage limited resources, above all data storage capacity and data transfer rate. Data amount used by nowadays applications is beyond the capability of mobile devices. A distributed file system presents an appropriate solution, but current distributed file systems are not suitable for mobile devices. We have explored the properties of current distributed file systems in relation to mobile devices. We propose a new distributed file system. The design of the system was verified by its implementation for GNU/Linux operating system at the server side and by the implementation of clients for Microsoft Windows, Windows Mobile, Linux and Android systems.

关键词： distributed file system Mobile Device

来源：评论

学校读者我要写书评

暂无评论

distributed file system for Spacecraft Integrated Test system 4

Distributed File System for Spacecraft Integrated Test Syste...

引用

4th IEEE International Conference on Computer and Communication systems (ICCCS)

作者： Du, Jianhai Lv, Jianghua Gao, Shiwei Zhang, Ling Li, Qianqian Li, Qinyong Ma, Shilong Beijing Univ Aeronaut & Astronaut Sch Comp Sci & Engn Beijing 100083 Peoples R China Beijing Inst Aerosp Control Devices Beijing 100039 Peoples R China Natl Earthquake Response Support Serv Beijing 100049 Peoples R China

ISBN: (纸本)9781728113227

Spacecraft is generally a complex system, which could produce a large amount of test data during the integrated testing process. The existing data management system for spacecraft integrated testing has the following disadvantages: the management of test data is not efficient;the existing traditional data management mode is not conducive to the query and management of historical model data;the security and reliability of data management is not high. In order to manage and maintain these test data more, according to the actual situation of spacecraft test and the particularity of data, a distributed file system of data management system for spacecraft integrated test is designed and implemented in the paper. It mainly includes: the data organization management form of the distributed file system, the allocation of storage space, the mechanism of file backup, the error correction scheme after data error, and the management scheme of hot and cold data.

关键词： distributed file system data management data classification spacecraft integrated testing

来源：评论

学校读者我要写书评

暂无评论

Performance Analysis of distributed file system Based on RAID Storage for Tapeless Storage

引用

IEEE ACCESS 2023年 11卷 116153-116168页

作者： Kim, Jungbin Yu, Hyeon-Jin Kang, Hyeongbin Shin, Jae-Hyuck Jeong, Heeseok Noh, Seo-Young Chungbuk Natl Univ Dept Comp Sci Cheongju 28644 South Korea Korea Inst Sci & Technol Informat Global Sci Expt Data Hub Ctr Daejeon 34141 South Korea

Tape storage is used as a long-term data storage solution in institutions and research centers around the world, contributing to the advancement of business and scientific research. Despite the existence of other storage media, tape storage has gained popularity owing to its low cost per storage capacity, making it an affordable option for initial implementation. However, recently, the strengths of tape storage have become ambiguous, and their drawbacks have become more evident. In the current storage system, tape storage requires complex management compared to other storage media, which incurs additional human resource costs, making it less cost effective. On the other side, other storage media continue to become cheaper and have larger storage capacities. Therefore, companies and research centers are no longer required to continuously purchase tape storage. With this trend, distributed file systems can be an alternative to tape storage. distributed file systems combine storage nodes into a network through Reliable Array of Independent Nodes (RAIN). Erasure coding is used to store data in this file system, which makes the system cost-effective and highly reliable. This allows the construction of a safe and efficient storage system. In this study, we constructed and evaluated a distributed file system based on Redundant Array of Independent Disks (RAID) storage. We configured and evaluated distributed file systems that use Reed-Solomon in the same environment, providing the performance characteristics of each distributed file system.

关键词： Tapeless storage distributed file system RAID RAIN performance EOS GlusterFS

来源：评论

学校读者我要写书评

暂无评论

Quantification and analysis of performance fluctuation in distributed file system

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2024年第3期27卷 3149-3162页

作者： Zhang, Yuchen Zhang, Xiao Yu, Jinyang Shi, Yulong Su, Linxuan Northwestern Polytech Univ IT Acad Dong Xiang Rd Xian 710129 Shaanxi Peoples R China

Performance is a key requirement in quality of service for cloud storage system users. Cloud storage system is complex, so the performance results may be different even under exact same experimental conditions. Many benchmarks are developed to measure performance, but there are only several research focus on performance fluctuations. In this paper, we present a generic method and two metrics to measure the performance fluctuations of a distributed file system. It analyzes the IO path and finds related factors such as storage media, network, configurations and software components. We tested the performance fluctuations of Ceph under different storage media HDD, SSD, and different workloads Webserver, fileserver, and Mailserver. We adjust the software configuration and software components to reduce the performance fluctuations. By using filestore, it reduces 85.4% performance fluctuations than filestore. By using EXT4 instead of XFS, it reduces performance fluctuations by 33.2%. But reducing performance fluctuations of lower components does not always reduce the performance fluctuation of distributed file system. We reduced the performance fluctuation of the local file system by 80.7%, but the performance fluctuation of DFS by 38.0%.

关键词： distributed file system Performance fluctuations Quantitative analysis Ceph

来源：评论

学校读者我要写书评

暂无评论

A Study of Effective Replica Reconstruction Schemes for the Hadoop distributed file system

引用

IEICE TRANSACTIONS ON INFORMATION AND systemS 2015年第4期E98D卷 872-882页

作者： Higai, Asami Takefusa, Atsuko Nakada, Hidemoto Oguchi, Masato Ochanomizu Univ Dept Comp Sci Tokyo 1128610 Japan Natl Inst Adv Ind Sci & Technol Tsukuba Ibaraki 3058568 Japan

distributed file systems, which manage large amounts of data over multiple commercially available machines, have attracted attention as management and processing systems for Big Data applications. A distributed file system consists of multiple data nodes and provides reliability and availability by holding multiple replicas of data. Due to system failure or maintenance, a data node may be removed from the system, and the data blocks held by the removed data node are lost. If data blocks are missing, the access load of the other data nodes that hold the lost data blocks increases, and as a result, the performance of data processing over the distributed file system decreases. Therefore, replica reconstruction is an important issue to reallocate the missing data blocks to prevent such performance degradation. The Hadoop distributed file system (HDFS) is a widely used distributed file system. In the HDFS replica reconstruction process, source and destination data nodes for replication are selected randomly. We find that this replica reconstruction scheme is inefficient because data transfer is biased. Therefore, we propose two more effective replica reconstruction schemes that aim to balance the workloads of replication processes. Our proposed replication scheduling strategy assumes that nodes are arranged in a ring, and data blocks are transferred based on this one-directional ring structure to minimize the difference in the amount of transfer data for each node. Based on this strategy, we propose two replica reconstruction schemes: an optimization scheme and a heuristic scheme. We have implemented the proposed schemes in HDFS and evaluate them on an actual HDFS cluster. We also conduct experiments on a large-scale environment by simulation. From the experiments in the actual environment, we confirm that the replica reconstruction throughputs of the proposed schemes show a 45% improvement compared to the HDFS default scheme. We also verify that the heuristic scheme is

关键词： HDFS distributed file system replica reconstruction heuristic optimization

来源：评论

学校读者我要写书评

暂无评论

PARADISE: Big data analytics using the DBMS tightly integrated with the distributed file system

引用

WORLD WIDE WEB-INTERNET AND WEB INFORMATION systemS 2016年第3期19卷 299-322页

作者： Kim, Jun-Sung Whang, Kyu-Young Kwon, Hyuk-Yoon Song, Il-Yeol Korea Adv Inst Sci & Technol Dept Comp Sci Daejeon South Korea Drexel Univ Coll Comp & Informat Philadelphia PA 19104 USA

There has been a lot of research on MapReduce for big data analytics. This new class of systems sacrifices DBMS functionality such as query languages, schemas, or indexes in order to maximize scalability and parallelism. However, as high functionality of the DBMS is considered important for big data analytics as well, there have been a lot of efforts to support DBMS functionality in MapReduce. HadoopDB is the only work that directly utilizes the DBMS for big data analytics in the MapReduce framework, taking advantage of both the DBMS and MapReduce. However, HadoopDB does not support sharability for the entire data since it stores the data into multiple nodes in a shared-nothing manner-i.e., it partitions a job into multiple tasks where each task is assigned to a fragment of data. Due to this limitation, HadoopDB cannot effectively process queries that require internode communication. That is, HadoopDB needs to re-load the entire data to process some queries (e.g., 2-way joins) or cannot support some complex queries (e.g., 3-way joins). In this paper, we propose a new notion of the DFS-integrated DBMS where a DBMS is tightly integrated with the distributed file system (DFS). By using the DFS-integrated DBMS, we can obtain sharability of the entire data. That is, a DBMS process in the system can access any data since multiple DBMSs are run on an integrated storage system in the DFS. To process big data analytics in parallel, our approach use the MapReduce framework on top of a DFS-integrated DBMS. We call this framework PARADISE. In PARADISE, we employ a job splitting method that logically splits a job based on the predicate in the integrated storage system. This contrasts with physical splitting in HadoopDB. We also propose the notion of locality mapping for further optimization of logical splitting. We show that PARADISE effectively overcomes the drawbacks of HadoopDB by identifying the following strengths. (1) It has a significantly faster (by up to 6.41 times) amo

关键词： Big data analytics MapReduce DBMS distributed file system Integration HadoopDB

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：