In lossy networks, including optical packet switching (OPS) networks and wireless networks, data recovery from frequent packet loss is essential. erasure-coding, a type of forward error correction (FEC), is one of eff...
详细信息
ISBN:
(纸本)9781665497343
In lossy networks, including optical packet switching (OPS) networks and wireless networks, data recovery from frequent packet loss is essential. erasure-coding, a type of forward error correction (FEC), is one of effective solutions, and furthermore, it could be combined with a technique that feeds back decoding progress information from the receiver to the sender for efficient data transmission. However, conventional feedback methods cannot inform the sender of the decoding progress information at the receiver when acknowledgment (ACK) packets are lost. Therefore, we propose a new data transmission method that is robust to high packet loss by feeding back accumulated information of decoded symbols at the receiver, and furthermore that smartly reduces the calculation overhead of encoding/decoding process. The proposed method can reduce the redundancy by up to 5.84% compared to conventional methods in environments where random packet loss occurs.
In this work, we propose an erasurecoding-based protocol that implements a key-value store with atomicity and near-optimal storage cost. Our protocol supports concurrent read and write operations while tolerating asy...
详细信息
ISBN:
(纸本)9781728183268
In this work, we propose an erasurecoding-based protocol that implements a key-value store with atomicity and near-optimal storage cost. Our protocol supports concurrent read and write operations while tolerating asynchronous communication and crash failures of any client and some fraction of servers. One novel feature is a tunable knob between the number of supported concurrent operations, availability, and storage cost. We implement our protocol into Cassandra, namely CassandrEAS (Cassandra + erasure-coding Atomic Storage). Extensive evaluation using YCSB on Google Cloud Platform shows that CassandrEAS incurs moderate penalty on latency and through-put, yet saves significant amount of storage space.
Gradient coding is a method for mitigating straggling servers in a centralized computing network that uses erasure-coding techniques to distributively carry out first-order optimization methods. Randomized numerical l...
详细信息
Gradient coding is a method for mitigating straggling servers in a centralized computing network that uses erasure-coding techniques to distributively carry out first-order optimization methods. Randomized numerical linear algebra uses randomization to develop improved algorithms for large-scale linear algebra computations. In this paper, we propose a method for distributed optimization that combines gradient coding and randomized numerical linear algebra. The proposed method uses a randomized $\ell _{2}$ -subspace embedding and a gradient coding technique to distribute blocks of data to the computational nodes of a centralized network, and at each iteration the central server only requires a small number of computations to obtain the steepest descent update. The novelty of our approach is that the data is replicated according to importance scores, called block leverage scores, in contrast to most gradient coding approaches that uniformly replicate the data blocks. Furthermore, we do not require a decoding step at each iteration, avoiding a bottleneck in previous gradient coding schemes. We show that our approach results in a valid $\ell _{2}$ -subspace embedding, and that our resulting approximation converges to the optimal solution.
Device-level interference is recognized as a major cause of the performance degradation in distributed file systems. Although the approaches of mitigating interference through coordination at application-level, middle...
详细信息
Device-level interference is recognized as a major cause of the performance degradation in distributed file systems. Although the approaches of mitigating interference through coordination at application-level, middleware-level, and server-level have shown beneficial results in previous studies, we find their effectiveness is largely reduced since I/O requests are re-arranged by underlying object file systems. In this research study, we prove that object-level coordination is critical and often the key to address the interference issue, as the scheduling of object requests determines the device-level accesses and thus determines the actual I/O bandwidth and latency. This article proposes an object-level coordination system, LoomIO, which uses an OBOP (One-Broadcast-One-Propagate) method and a time-limited coordination process to deliver highly efficient coordination service. Specifically, LoomIO enables object requests to achieve an optimized scheduling decision within a few milliseconds and largely mitigates the device-level interference. We have implemented a LoomIO prototye and integrated it into Ceph file system. The evaluation results show that LoomIO achieved the considerable improvements in resource utilization (by up to 35%), in I/O throughput (by up to 31%), and in 99th percentile latency (by up to 54%) compared to the K-optimal method which uses the same scheduling algorithm as LoomIO but does not have the coordination support.
In recent years cloud storage services have gained much attention and become a commodity to companies and private users. Nevertheless, cloud storage services have some limitations, especially related to privacy, laten...
详细信息
ISBN:
(数字)9781665491150
ISBN:
(纸本)9781665491150
In recent years cloud storage services have gained much attention and become a commodity to companies and private users. Nevertheless, cloud storage services have some limitations, especially related to privacy, latency, and availability. In this work we propose a distributed storage system which tackles the major limitations of classical cloud storage services. To this end, we design and implement a storage system which combines classical cloud storage services with the approach of fog computing by using resources at the edge of the network. At the core of our system lies a placement strategy which distributes the data to different storage components. Our implementation is based on well-established methods and techniques from information theory and cryptography. Our empirical analysis shows that our system preserves privacy, provides low latency, and offers high availability. Most notably, we reduce the latency by up to 42% in Upload Mode and even by up to 76% in Download Mode compared to a cloud-only solution.
Users entrust an increasing amount of data to online cloud systems for archival purposes. Existing storage systems designed to preserve user data unaltered for decades do not, however, provide strong security guarante...
详细信息
ISBN:
(纸本)9781538655962
Users entrust an increasing amount of data to online cloud systems for archival purposes. Existing storage systems designed to preserve user data unaltered for decades do not, however, provide strong security guarantees-at least at a reasonable cost. This paper introduces RECAST, an anti-censorship data archival system based on random data entanglement. Documents are mixed together using an entanglement scheme that exploits erasure codes for secure and tamper-proof long-term archival. Data is intertwined in such a way that it becomes virtually impossible to delete a specific document that has been stored long enough in the system, without also erasing a substantial fraction of the whole archive, which requires a very powerful adversary and openly exposes the attack. We validate RECAST's entanglement approach via simulations and we present and evaluate a full-fledged prototype deployed in a local cluster. In one of our settings, we show that RECAST, configured with the same storage overhead as triple replication, can withstand 10% of storage node failures without any data loss. Furthermore, we estimate that the effort required from a powerful censor to delete a specific target document is two orders of magnitude larger than for triple replication.
Distributed wide-area storage systems must tolerate both physical failure and logic errors. In particular, these functions are needed to enable the storage system to support remote disaster recovery. There are several...
详细信息
Distributed wide-area storage systems must tolerate both physical failure and logic errors. In particular, these functions are needed to enable the storage system to support remote disaster recovery. There are several solutions for distributed wide-area backup/ archive systems implemented at application level, file system level or at storage subsystem level. However, they suffer from high deployment cost and security issues. Moreover, previous researches in literature only focus on any diskrelated failures and ignore the fact that storage server linked predominantly to a Wide-Area-Network (WAN) which may be unavailable or owing to network failures. In this paper, we first model the efficiency and reliability of distributed wide area storage systems for all media, taking both network failures and disk failures into consideration. To provide higher performance, efficiency, reliability, and security to the wide-area disaster recovery storage systems, we present a configurable RAID-like data erasure-coding scheme referred to as Replication-based Snapshot Redundant Array of Independent Imagef iles (RSRAII). We argue that this scheme has benefits resulting from the consolidation of both erasure-coding and replication strategies. To this end, we propose a novel algorithm to improve the snapshot performance referred to as SMPDP (Snapshot based on Multi-Parallel Degree Pipeline). We also extend this study towards implementing a prototype system, called as SeWDReSS, which is shown to strike a tradeoff between reliability, storage space, security, and performance for distributed wide-area disaster recovery.
暂无评论