检索结果-内蒙古大学图书馆

Achieving High Efficiency and High Throughput in erasure code-Based Distributed Storage for Blockchain

SENSORS 2025年第7期25卷 2161-2161页

作者： Park, So-Hyun Kim, So-Yeon Kim, So-Hui Lee, Il-Gu Sungshin Womens Univ Dept Future Convergence Technol Engn Seoul 02844 South Korea

A blockchain is a decentralized peer-to-peer network in which all nodes store data in copies, ensuring data integrity, as transactions cannot be changed or deleted. This can lead to duplicate data storage, resulting in high storage overhead, especially in storage-constrained environments, such as the Internet of Things (IoT) or sensor systems. Distributed storage techniques utilizing erasure code (EC) have been investigated to address this issue. Although EC-based blockchain storage increases storage efficiency, encoded chunks distributed across multiple nodes must be received to restore and access the original blocks. However, studies on increasing the data transmission efficiency of EC-based blockchain storage are limited. In this study, we propose a data transmission technique called trigger-based automatic repeat request (ARQ), enabling stable data recovery while ensuring low latency and high-throughput performance, even with frequent node failure. This technique increased the throughput efficiency by 8% while maintaining the decentralization of the blockchain. Furthermore, it maximized the storage efficiency of EC-based distributed blockchain storage by >99.8%, while solving the recovery overhead problem due to data transmission. Using the trigger-based ARQ scheme with an EC-based distribution technique, blockchains can reduce storage overhead while effectively accessing the original blocks, overcoming the limitations of conventional EC-based distributed storage.

关键词： automatic repeat request blockchain data recovery distributed storage erasure code wireless sensor networks trigger

来源：评论

学校读者我要写书评

暂无评论

A Novel Completely Local Repairable code Algorithm Based on erasure code

引用

TEHNICKI VJESNIK-TECHNICAL GAZETTE 2022年第5期29卷 1658-1664页

作者： Fang, Ying Tan, Hai Wang, Shuaifang Zhang, Xin Liao, Gejian Zhang, Jun Shangqiu Normal Univ Shangqiu Peoples R China Nanjing Audit Univ Nanjing Peoples R China East China Univ Technol Shanghai Peoples R China

Hadoop Distributed File System (HDFS) is widely used in massive data storage. Because of the disadvantage of the multi-copy strategy, the hardware expansion of HDFS cannot keep up with the continuous volume of big data. Now, the traditional data replication strategy has been gradually replaced by erasure code due to its smaller redundancy rate and storage overhead. However, compared with replicas, erasure code needs to read a certain amount of data blocks during the process of data recovery, resulting in a large amount of overhead for I/O and network. Based on the Reed-Solomon (RS) algorithm, we propose a novel Completely Local Repairable code (CLRC) algorithm. By grouping RS coded blocks and generating local check blocks, CLRC algorithm can optimize the locality of the RS algorithm, which can reduce the cost of data recovery. Evaluations show that the CLRC algorithm can reduce the bandwidth and I/O consumption during the process of data recovery when a single block is damaged. What's more, the cost of decoding time is only 59% of the RS algorithm.

关键词： data recovery erasure code HDFS RS algorithm storage overhead

来源：评论

学校读者我要写书评

暂无评论

On designing endurance aware erasure code for SSD-based storage systems

引用

MICROPROCESSORS AND MICROSYSTEMS 2016年第PartB期45卷 283-296页

作者： Chamazcoti, Saeideh Alinezhad Miremadi, Seyed Ghassem Sharif Univ Technol Dept Comp Engn Tehran Iran

erasure codes are applied in both HDD and SSD storage systems to improve the reliability. The design of erasure codes for SSD-based systems should be performed with respect to a specific feature of SSDs, i.e., endurance. Endurance is defined as the number of Program/Erase (P/E)-cycles that one SSD can endure for reliable operation. The common metric for comparing the endurance of two systems is the number of PIE -cycles, which is yielded by time-consuming simulations. This paper proposes two new metrics called DPD-factor and GDP-pattern, for comparing the effect of erasure codes on the endurance of systems based on their encoding designs, without simulation. With respect to the endurance, EA-EO is designed as the modification of EVENODD with smaller DPD-factor. The endurance of EVENODD and EA-EO are compared regarding the system configurations: the size of stripe unit, the number of disks, and the sizes of request. The results of comparison show that the best configurations of system for enhanced, endurance are: 1) a large number of disks are applied in systems, or 2) the size of request is equal to the stripe unit size. Furthermore, it concludes that a code with smaller DPD-factor and a sequential GDP-pattern can provide higher endurance for systems. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Solid state drive (SSD) erasure code Endurance Program/Erase (P/E) cycles Storage system

来源：评论

学校读者我要写书评

暂无评论

An Adaptive erasure code for JointCloud Storage of Internet of Things Big Data

引用

IEEE INTERNET OF THINGS JOURNAL 2020年第3期7卷 1613-1624页

作者： Bao, Han Wang, Yijie Xu, Fangliang Natl Univ Def Technol Natl Lab Parallel & Distributed Proc Changsha 410073 Peoples R China Natl Univ Def Technol Coll Comp Sci Changsha 410073 Peoples R China

JointCloud is a cross-cloud cooperation architecture for integrated Internet service customization. The customized cross-cloud storage service based on this architecture is called JointCloud storage. Storing the Internet of Things (IoT) big data in erasure-coded JointCloud storage systems ensures that data can be accessed when several cloud services interrupt. However, because existing erasure codes cannot adapt the generator matrix and data placement scheme to different network environments and encoding parameters, they usually incur a large network resource consumption (NRC) for repairing data in JointCloud storage systems. As a result, the availability of IoT applications running on JointCloud storage systems is impaired. In this article, to minimize the NRC of repairing data, we propose an adaptive erasure code for JointCloud storage of IoT big data called ACIoT. Specifically, we first propose the concept of average weighted locality (AWL) of a stripe of erasure-coded data, which is proportional to the average NRC of repairing this stripe in JointCloud storage systems. Then, we propose an active parallel trial-and-error algorithm to calculate the optimal generator matrix and data placement scheme to achieve the lowest AWL, under different network environments and encoding parameters. By encoding and placing each stripe of data with the optimal generator matrix and data placement scheme, ACIoT can achieve the minimum NRC. The experiments show that, compared with several state-of-the-art erasure codes, ACIoT reduces the NRC by 26.4%-44.7%.

关键词： Maintenance engineering Internet of Things Cloud computing Encoding Generators Big Data Network topology Cloud datacenter (DC) disaster tolerance erasure code fault tolerance Internet of Things (IoT) JointCloud storage

来源：评论

学校读者我要写书评

暂无评论

Cost-Effective Data Placement in Edge Storage Systems With erasure code

引用

IEEE TRANSACTIONS ON SERVICES COMPUTING 2023年第2期16卷 1039-1050页

作者： Jin, Hai Luo, Ruikun He, Qiang Wu, Song Zeng, Zilai Xia, Xiaoyu Huazhong Univ Sci & Technol Natl Engn Res Ctr Big Data Technol & Syst Sch Comp Sci & Technol Serv Comp Technol & Syst LabCluster & Grid Comp L Wuhan 430074 Hubei Peoples R China Swinburne Univ Technol Dept Comp Technol Melbourne Vic 3122 Australia Deakin Univ Sch Informat Technol Geelong Vic 3125 Australia

Edge computing, as a new computing paradigm, brings cloud computing's computing and storage capacities to network edge for providing low latency services for users. The networked edge servers in a specific area constitute edge storage systems (ESSs), where popular data can be stored to serve the users in the area. The novel ESSs raise many new opportunities as well as unprecedented challenges. Most existing studies of ESSs focus on the storage of data replicas in the system to ensure low data retrieval latency for users. However, replica-based edge storage strategies can easily incur high storage costs. It is not cost-effective to store massive replicas of large-size data, especially those that do not require real-time access at the edge, e.g., system upgrade files, popular app installation files, videos in online games. It may not even be possible due to the constrained storage resources on edge servers. In this article, we make the first attempt to investigate the use of erasure codes in cost-effective data storage at the edge. The focus is to find the optimal strategy for placing coded data blocks on the edge servers in an ESS, aiming to minimize the storage cost while serving all the users in the system. We first model this novel erasure Coding based Edge Data Placement (EC-EDP) problem as an integer linear programming problem and prove its $\mathcal {NP}$NP-hardness. Then, we propose an optimal approach named EC-EDP-O based on integer programming. Another approximation algorithm named EC-EDP-V is proposed to address the high computation complexity of large-scale EC-EDP scenarios efficiently. The extensive experimental results demonstrate that EC-EDP-O and EC-EDP-V can save an average of 68.58% (and up to 81.16% in large-scale scenarios) storage cost compared with replica-based storage approaches.

关键词： Servers Encoding Costs codes Cloud computing Edge computing Distributed databases erasure code data placement approximation algorithm

来源：评论

学校读者我要写书评

暂无评论

NADE: nodes performance awareness and accurate distance evaluation for degraded read in heterogeneous distributed erasure code-based storage

引用

JOURNAL OF SUPERCOMPUTING 2020年第7期76卷 4946-4975页

作者： Zhang, Xingjun Cai, Yi Liu, Yunfei Xu, Zhiwei Dong, Xiaoshe Xi An Jiao Tong Univ Sch Elect & Informat Engn Dept Comp Sci & Technol 28 XianNing West Rd Xian 710049 Peoples R China

To ensure data availability and save storage space, storage systems usually save data across multiple storage nodes (or servers) using erasure codes. Storage systems need to reconstruct the complete data to respond to reading requests in the case of the loss of some data blocks when node failure occurs. However, a degraded read in erasure code-based storage systems does not fully utilize node resources and ignores the node's topology. In this paper, we propose a real-time performance evaluation model for storage nodes to evaluate the performance of each node combining a metrics choice and an analytic hierarchy process. We also design a cost evaluation method to calculate the transmission cost by considering the node's topology. By combining the node evaluation method and a distance calculation, we propose an adaptive degraded read optimization strategy, NADE. We further implement the node selection method NADE in Ceph. The evaluation results show the efficiency of the proposed method.

关键词： Distributed storage erasure code Degraded read Performance evaluation Ceph

来源：评论

学校读者我要写书评

暂无评论

erasure code of Small File in a Distributed File System 3

Erasure Code of Small File in a Distributed File System

引用

3rd IEEE International Conference on Computer and Communications (ICCC)

作者： Chen, Xinhai Liu, Jie Xie, Peizhen Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha Hunan Peoples R China

ISBN: (纸本)9781509063529

With the development of Internet applications, the data storage of many applications has some new characters. There always exists a huge amount of pictures in distributed file systems, and the size of these pictures is usually no more than 1M bytes. To meet the demands in small file storage system which contains so many pictures, we provide a kind of distributed file system to store large amounts of small files. In our system, small files (actual data files) are merged into large files (default 64M) which we call block, and we use two-indexed way to reduce the pressure of nameserver, which maintains the information of correspondence between blocks and dataservers. The result shows that the memory usage of block information in nameserver is less than 2G when the system capacity is 1PB. It shows good scalability. Meanwhile, to reduce the cost of storage, we introduce the technique of erasure code which is an alternative offers the same data protection but reduces significantly the storage consumption. After grouping, the cost of storage dropped by 25% compared with 2 replications.

关键词： Small file index file erasure code block distributed storage system

来源：评论

学校读者我要写书评

暂无评论

Content-access QoS in peer-to-peer networks using a fast MDS erasure code

引用

COMPUTER COMMUNICATIONS 2005年第15期28卷 1778-1790页

作者： Dairaine, L Lacan, M Lancérica, L Fimes, J ENSICA F-31056 Toulouse France CNRS LAAS F-31077 Toulouse France TeSA F-31071 Toulouse France ICT Australia NICTA Eveleigh NSW 1430 Australia

This paper describes an enhancement of content access Quality of Service in peer to peer (P2P) networks. The main idea is to use an erasure code to distribute the information over the peers. This distribution increases the users' choice on disseminated encoded data and therefore, statistically enhances the overall throughput of the transfer. A performance evaluation based on an original model using the results of a measurement campaign of sequential and parallel downloads in a real P2P network over Internet is presented. Based on a bandwidth distribution, statistical content-access QoS are guaranteed in function of both the content replication level in the network and the file dissemination strategies. A simple application in the context of media streaming is proposed. Finally, the constraints on the erasure code related to the proposed system are analysed and a new fast MDS erasure code is proposed, implemented and evaluated. (c) 2005 Elsevier B.V. All rights reserved.

关键词： peer-to-peer quality of service data storage performance evaluation erasure code

来源：评论

学校读者我要写书评

暂无评论

CLEC:Combination Locality Based erasure code for Permissioned Blockchain Storage

引用

Computers, Materials & Continua 2022年第12期73卷 5137-5150页

作者： Jiabin Wu Boai Yang Yang Liu Fang Liu Nong Xiao Shuo Li National University of Defense Technology Changsha410073China Hunan University Changsha410082China Sun Yat-sen University Guangzhou510275China The University of Edinburgh EdinburghEH89BTThe United Kingdom

Building a new decentralized domain name system based on blockchain technology is helping to solve problems,such as load imbalance and over-dependence on the trust of the central ***,in the existing blockchain storage system,the storage overhead is very high due to its fullreplication data storage *** total storage consumption for each block is up to O(n)with n *** code applied to blockchains can significantly reduce the storage overhead,but also greatly lower the read *** this study,we propose a novel coding scheme for blockchain storage,Combination Locality based erasure code for Permissioned blockchain storage(CLEC).CLEC uses erasure code,parity locality,and topology locality in blockchain storage,greatly reducing reading latency and repair *** CLEC,the storage consumption per block can be reduced to O(1),and the repair penalty can also be lowered to O(1).Experiments in an open-source permissioned blockchain Tendermint show that CLEC has a maximum repair speed of 6 times and a read speed of nearly 1.7 times with storage overhead of only 1.17 times compared to the current work,a great improvement in reading performance and repair performance with slightly increased storage overhead via implementation.

关键词： CLEC blockchain erasure code delay circle read performance throughput repair performance

来源：评论

学校读者我要写书评

暂无评论

Joint video adaptation and erasure code for video broadcast in wireless networks

引用

Science China(Information Sciences) 2013年第10期56卷 102-116页

作者： HE NengQiang ZHANG XueDan LI YiPeng CAO JianNong LI Zhu REN Yong Department of Electronic Engineering Tsinghua University Department of Automation Tsinghua University Department of Computing Hong Kong Polytechnic University

Advances in wireless access technologies such as WiMax and LTE have accelerated the development of wireless video *** existing erasure codes are designed for wireless data broadcasting,their potential can only be realized for wireless video broadcasting if the distortions associated with video packets and their complex decoding dependency are taken into *** this paper,we propose the greedy video erasure code(GVEC)to minimize average video distortion by accounting for the temporal scalable decoding dependency among video frames e.g.I,P and B frames in *** is a greedy algorithm and sufers from high computational *** practical implementation,we have designed the joint video adaptation and erasure code(JVEC)with lower complexity,which can generate erasure codes on the fly to guarantee the decoding of important video *** results show that the performance of JVEC is very close to that of GVEC,and is superior to that of existing erasure *** results also show the existence of a performance tradeof between video distortion and video packet throughput.

关键词： wireless video broadcast erasure code video analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：