Providing data availability in a high performance computing environment is very important, especially in this data-intensive world. Most clusters either equip with RAID (Redundant Array of Independent Disks) devices o...
详细信息
Providing data availability in a high performance computing environment is very important, especially in this data-intensive world. Most clusters either equip with RAID (Redundant Array of Independent Disks) devices or use redundant nodes to protect data from loss. However, neither of these can really solve the reliability problem incurred in a striped filesystem. Striping provides an efficient way to increase I/O throughput both in the distributed and parallel paradigms. But it also reduces the overall reliability of a disk system by N fold, where N is the number of independent disks in the system. parallel virtual file system (PVFS) is an open source parallelfilesystem which has been widely used in the Linux environment. Its striping structure is good for performance but provides no fault tolerance. We implement Reliable parallelfilesystem (RPFS) based on PVFS but with reliability support. Our quantitative analysis shows that MTTF (Mean Time To Failure) of our RPFS is better than that of PVFS. Besides, we propose a parity cache table (PCT) to alleviate the penalty of parity updating. The evaluation of our RPFS shows that its read performance is almost the same as that of PVFS (2% to 13% degradation). As to the write performance, 28% to 45% improvement can be achieved depending on the behavior of the operations.
Linux clusters of commodity computer systems and interconnects have become the fastest growing choice for building cost-effective high-performance parallel computing systems. The parallel virtual file system (PVFS) co...
详细信息
ISBN:
(纸本)0769517609
Linux clusters of commodity computer systems and interconnects have become the fastest growing choice for building cost-effective high-performance parallel computing systems. The parallel virtual file system (PVFS) could potentially fulfill the requirements of large I/O-intensive parallel applications. It provides a high-performance parallelfilesystem by striping file data across multiple cluster nodes, called I/O nodes. Therefore, the choice of storage devices on I/O nodes is crucial to PVFS. In this paper we study the impact of software RAIDs and hardware RAIDs on the performance of PVFS when they are used on I/O node's. We first establish a baseline performance of both RAIDs in a stand-alone configuration. We then present the performance of PVFS for a workload comprising concurrent reads and writes using ROMIO MPI-IO, andfor the BTIO benchmark with a noncontiguous access pattern. We found that software RAIDs have a comparable performance to hardware RAIDs, except for write ope I rations that require file synchronization.
Striped filesystems such as the parallel virtual file system (PVFS) deliver high-bandwidth I/O to applications running on clusters. An open problem in the design of striped filesystems is how to reduce their vulnera...
详细信息
ISBN:
(纸本)0769520170
Striped filesystems such as the parallel virtual file system (PVFS) deliver high-bandwidth I/O to applications running on clusters. An open problem in the design of striped filesystems is how to reduce their vulnerability to disk failures with the minimum performance penalty. In this paper we describe a novel data redundancy scheme designed specifically to address the performance issue. We demonstrate the new scheme within CSAR, a proof-of-concept implementation based on PVFS. By dynamically switching between RAID1 and RAID5 redundancy based on write size, CSAR consistently achieves the best of two worlds - RAID1 performance on small writes, and RAID5 efficiency on large writes. Using the popular parallel I/O benchmark BTIO, our scheme achieves 82% of the write bandwidth of the unmodified PVFS. We describe the issues in implementing our new scheme in a popular striped filesystem such as PVFS on a Linux cluster
暂无评论