检索结果-内蒙古大学图书馆

A Simulation Analysis of Redundancy and Reliability in Primary Storage Deduplication

IEEE TRANSACTIONS ON COMPUTERS 2018年第9期67卷 1259-1272页

作者： Fu, Min Han, Shujie Lee, Patrick P. C. Feng, Dan Chen, Zuoning Xiao, Yu Huazhong Univ Sci & Technol Sch Comp Wuhan Natl Lab Optoelect Wuhan 430074 Hubei Peoples R China Chinese Univ Hong Kong Shatin Hong Kong Peoples R China Natl Engn Res Ctr Parallel Comp Beijing 100080 Peoples R China

Deduplication has been widely used to improve storage efficiency in modern primary and secondary storage systems, yet how deduplication fundamentally affects storage system reliability remains debatable. This paper aims to analyze and compare storage system reliability with and without deduplication in primary workloads using public file system snapshots from two research groups. We first study the redundancy characteristics of the file system snapshots. We then propose a trace-driven, deduplication-aware simulation framework to analyze data loss in both chunk and file levels due to sector errors and whole-disk failures. Compared to without deduplication, our analysis shows that deduplication consistently reduces the damage of sector errors due to intra-file redundancy elimination, but potentially increases the damages of whole-disk failures if the highly referenced chunks are not carefully placed on disk. To improve reliability, we examine a deliberate copy technique that stores and repairs first the most referenced chunks in a small dedicated physical area (e.g., 1 percent of the physical capacity), and demonstrate its effectiveness through our simulation framework.

关键词： Deduplication reliability primary storage systems experiments and implementation

来源：评论

学校读者我要写书评

暂无评论

Enabling Concurrent Failure Recovery for Regenerating-Coding-Based Storage Systems: From Theory to Practice

引用

IEEE TRANSACTIONS ON COMPUTERS 2015年第7期64卷 1898-1911页

作者： Li, Runhui Lin, Jian Lee, Patrick P. C. Chinese Univ Hong Kong Dept Comp Sci & Engn Shatin Hong Kong Peoples R China

Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which have been designed to provide fault-tolerant storage and minimum bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes for the recovery of a general number of failures including single and concurrent failures. We show theoretically that CORE achieves the minimum possible bandwidth for most cases. We implement a CORE prototype and evaluate it atop an HDFS cluster testbed with up to 20 storage nodes. We demonstrate that our CORE prototype conforms to our theoretical findings and achieves bandwidth savings when compared to the conventional recovery approach based on erasure codes.

关键词： Regenerating codes failure recovery distributed storage systems coding theory experiments and implementation

来源：评论

学校读者我要写书评

暂无评论

Enabling Efficient and Reliable Transition from Replication to Erasure Coding for Clustered File Systems

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2017年第9期28卷 2500-2513页

作者： Li, Runhui Hu, Yuchong Lee, Patrick P. C. Chinese Univ Hong Kong Shatin Hong Kong Peoples R China Huazhong Univ Sci & Technol Wuhan 430074 Hubei Peoples R China

To balance performance and storage efficiency, modern clustered file systems often first store data with replication, followed by encoding the replicated data with erasure coding. We argue that the commonly used random replication does not take into account erasure coding in its design, thereby raising both performance and availability issues in the subsequent encoding operation. We propose encoding-aware replication, which carefully places the replicas so as to (i) eliminate cross-rack downloads of data blocks during the encoding operation, (ii) preserve availability without data relocation after the encoding operation, and (iii) maintain load balancing across replicas as in random replication before the encoding operation. We conduct extensive HDFS-based testbed experiments and discrete-event simulations, and demonstrate the performance gains of encoding-aware replication over random replication.

关键词： Replication erasure codes distributed storage systems experiments and implementation

来源：评论

学校读者我要写书评

暂无评论

CORE: Augmenting Regenerating-Coding-Based Recovery for Single and Concurrent Failures in Distributed Storage Systems

CORE: Augmenting Regenerating-Coding-Based Recovery for Sing...

引用

29th IEEE Symposium on Massive Storage Systems and Technologies (MSST) - Research Track

作者： Li, Runhui Lin, Jian Lee, Patrick P. C. Chinese Univ Hong Kong Dept Comp Sci & Engn Hong Kong Hong Kong Peoples R China

ISBN: (纸本)9781479902170;9781479902187

Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which are shown to provide fault-tolerant storage and minimum recovery bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes to support a general number of failures including single and concurrent failures. We theoretically show that CORE achieves the minimum possible recovery bandwidth for most cases. We implement CORE and evaluate our prototype atop a Hadoop HDFS cluster testbed with up to 20 storage nodes. We demonstrate that our CORE prototype conforms to our theoretical findings and achieves recovery bandwidth saving when compared to the conventional recovery approach based on erasure codes.

关键词： regenerating codes failure recovery distributed storage systems coding theory experiments and implementation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：