检索结果-内蒙古大学图书馆

parallel file system performances in fusion data storage

FUSION ENGINEERING AND DESIGN 2012年第12期87卷 2063-2067页

作者： Iannone, F. Podda, S. Bracco, G. Manduchi, G. Maslennikov, A. Migliori, S. Wolkersdorfer, K. CRENEA Frascati Assoc EURATOM ENEA Fus I-00044 Rome Italy ENEA Informat Commun Tecnol I-00196 Rome Italy Consorzio RFX Assoc EURATOM ENEA Fus I-35127 Padua Italy CASPUR Interuniv Consortium Applicat Super Comp R I-00185 Rome Italy Julich Supercomp Ctr FZJ D-52425 Julich Germany

High I/O flow rates, up to 10 GB/s, are required in large fusion Tokamak experiments like ITER where hundreds of nodes store simultaneously large amounts of data acquired during the plasma discharges. Typical network topologies such as linear arrays (systolic), rings, meshes (2-D arrays), tori (3-D arrays), trees, butterfly, hypercube in combination with high speed data transports like Infiniband or 10G-Ethernet, are the main areas in which the effort to overcome the so-called parallel I/O bottlenecks is most focused. The high I/O flow rates were modelled in an emulated testbed based on the parallel file systems such as Lustre and GPFS, commonly used in High Performance Computing. The test runs on High-Performance Computing-For Fusion (8640 cores) and ENEA CRESCO (3392 cores) supercomputers. Message Passing Interface based applications were developed to emulate parallel I/O on Lustre and GPFS using data archival and access solutions like MDSPLUS and Universal Access Layer. These methods of data storage organization are widely diffused in nuclear fusion experiments and are being developed within the EFDA Integrated Tokamak Modelling - Task Force;the authors tried to evaluate their behaviour in a realistic emulation setup. (c) 2012 Elsevier B.V. All rights reserved.

关键词： High performance computing (HPC) parallel file system Nuclear fusion

来源：评论

学校读者我要写书评

暂无评论

Boosting parallel file system Performance via Heterogeneity-Aware Selective Data Layout

引用

IEEE TRANSACTIONS ON parallel AND DISTRIBUTED systemS 2016年第9期27卷 2492-2505页

作者： He, Shuibing Wang, Yang Sun, Xian-He Wuhan Univ Comp Sch State Key Lab Software Engn Wuhan 430072 Hubei Peoples R China Natl Univ Def Technol State Key Lab High Performance Comp Changsha 410073 Hunan Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Xueyuan Ave 1068 Shenzhen 518055 Peoples R China IIT Dept Comp Sci Chicago IL 60616 USA

Hybrid parallel file systems (PFS) that combine HDD servers with SSD servers provide a promising solution for data intensive applications. The efficiency of a hybrid PFS relies on the data layout schemes. However, most current layout strategies are designed for homogeneous servers, which neither address the heterogeneity of servers nor the varying access patterns of applications. In this paper, we propose HAS, a novel heterogeneity-aware selective data layout scheme for hybrid PFSs. HAS alleviates inter-server load imbalance through skewing data distribution on heterogeneous servers based on their storage performance. Furthermore, to obtain the optimal performance for a specific access pattern, HAS selects one static data layout policy with lowest access cost from three typical layout candidates as the final file data layout method. To adapt to the mixed access patterns within an application, HAS uses a dynamic data layout scheme, which stores file with multiple copies, each using a different data layout policy, and then selects the copy with the lowest access cost to serve file requests. We have implemented HAS within MPICH2 and OrangeFS. Experimental results show that HAS can significantly increase the I/O throughput of hybrid PFSs, compared to existing data layout optimization methods.

关键词： parallel I/O system parallel file system data layout solid state drive

来源：评论

学校读者我要写书评

暂无评论

Design and Implementation of an Asymmetric Block-Based parallel file system

引用

IEEE TRANSACTIONS ON COMPUTERS 2014年第7期63卷 1723-1735页

作者： Yi, Letian Shu, Jiwu Zhao, Ying Qian, Yingjin Lu, Youyou Zheng, Weimin Tsinghua Univ Dept Comp Sci Beijing 100084 Peoples R China

Existing block-based parallel file systems, which are deployed in the storage area network (SAN), blend metadata with data in underlying disks. Unfortunately, such symmetric architecture is prone to system-level failures, as metadata on shared disks can be damaged by a malfunctioning client. In this paper, we present an asymmetric block-based parallel file system, Redbud, which isolates the metadata storage in the metadata server (MDS) access domain. Although centralized metadata management can effectively improve the reliability of the system, it faces some challenges in providing high performance and availability. Towards this end, we introduce an embedded directory mechanism to explore the disk bandwidth of the metadata storage;we also introduces adaptive layout operations to deliver high I/O throughput for various file access pattern. Besides, by taking the MDS's load into consideration, we propose an adaptive timeout algorithm to make the MDS failure detection adaptive to the evolving workloads, improving the system availability. Measurements of a wide range of workloads demonstrate the benefit of our design and that Redbud gains good scalability.

关键词： parallel file system asymmetric

来源：评论

学校读者我要写书评

暂无评论

A high-performance distributed parallel file system for data-intensive computations

引用

JOURNAL OF parallel AND DISTRIBUTED COMPUTING 2004年第10期64卷 1157-1167页

作者： Shen, XH Choudhary, A Northwestern Univ Dept Elect & Comp Engn Ctr Parallel & Distributed Comp Evanston IL 60208 USA

One of the challenges brought by large-scale scientific applications is how to avoid remote storage access by collectively using sufficient local storage resources to hold huge amounts of data generated by the simulation while providing high-performance I/O. DPFS, a distributed parallel file system, is designed and implemented to address this problem. DPFS collects locally distributed and unused storage resources as a supplement to the internal storage of parallel computing systems to satisfy the storage capacity requirement of large-scale applications. In addition, like parallel file systems, DPFS provides striping mechanisms that divide a file into small pieces and distributes them across multiple storage devices for parallel data access. The unique feature of DPFS is that it provides three file levels with each file level corresponding to a file striping method. In addition to the traditional linear striping method, DPFS also provides a novel Multidimensional striping method that can solve performance problems of linear striping for many popular access patterns. Other issues such as load-balancing and user interface are also addressed in DPFS. (C) 2004 Elsevier Inc. All rights reserved.

关键词： distributed file system parallel file system striping data intensive applications I/O

来源：评论

学校读者我要写书评

暂无评论

PeakFS: An Ultra-High Performance parallel file system via Computing-Network-Storage Co-Optimization for HPC Applications

引用

IEEE TRANSACTIONS ON parallel AND DISTRIBUTED systemS 2024年第12期35卷 2578-2595页

作者： Chen, Yixiao Yang, Haomai Lu, Kai Huang, Wenlve Wang, Jibin Wan, Jiguang Zhou, Jian Wu, Fei Xie, Changsheng Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect Wuhan 430074 Peoples R China Qilu Univ Technol Shandong Acad Sci Natl Supercomp Ctr Jinan Key Lab Comp Power Network & Informat SecurMinist Jinan 250353 Peoples R China Shandong Fundamental Res Ctr Comp Sci Shandong Prov Key Lab Comp Power Internet & Serv C Jinan 250000 Peoples R China

Emerging high-performance computing (HPC) applications with diverse workload characteristics impose greater demands on parallel file systems (PFSs). PFSs also require more efficient software designs to fully utilize the performance of modern hardware, such as multi-core CPUs, Remote Direct Memory Access (RDMA), and NVMe SSDs. However, existing PFSs expose great limitations under these requirements due to limited multi-core scalability, unaware of HPC workloads, and disjointed network-storage optimizations. In this article, we present PeakFS, an ultra-high performance parallel file system via computing-network-storage co-optimization for HPC applications. PeakFS designs a shared-nothing scheduling system based on link-reduced task dispatching with lock-free queues to reduce concurrency overhead. Besides, PeakFS improves I/O performance with flexible distribution strategies, memory-efficient indexing, and metadata caching according to HPC I/O characteristics. Finally, PeakFS shortens the critical path of request processing through network-storage co-optimizations. Experimental results show that the metadata and data performance of PeakFS reaches more than 90% of the hardware limits. For metadata throughput, PeakFS achieves a 3.5-19x improvement over GekkoFS and outperforms BeeGFS by three orders of magnitude.

关键词： Metadata Message systems file systems Hardware Optimization Semantics Scalability Servers Protocols Program processors parallel file system high-performance computing remote direct memory access performance optimization

来源：评论

学校读者我要写书评

暂无评论

The Vesta parallel file system

引用

ACM TRANSACTIONS ON COMPUTER systemS 1996年第3期14卷 225-264页

作者： Corbett, PF Feitelson, DG IBM T. J. Watson Research Center

The Vests parallel file system is designed to provide parallel file access to application programs running on multicomputers with parallel I/O subsystems. Vesta uses a new abstraction of files: a file is not a sequence of bytes, but rather it can be partitioned into multiple disjoint sequences that are accessed in parallel. The partitioning-which can also be changed dynamically-reduces the need for synchronization and coordination during the access. Some control over the layout of data is also provided, so the layout can be matched with the anticipated access patterns. The system is fully implemented and forms the basis for the AM parallel I/O file system on the IBM SP2. The implementation does not compromise scalability or parallelism. In fact, all data accesses are done directly to the I/O node that contains the requested data, without any indirection or access to shared metadata. Disk mapping and caching functions are confined to each I/O node, so there is no need to keep data coherent across nodes. Performance measurements show good scalability with increased resources. Moreover, different access patterns are shown to achieve similar performance.

关键词： design performance data partitioning parallel computing parallel file system

来源：评论

学校读者我要写书评

暂无评论

Dynamic file prefetching scheme based on file access patterns in VIA-based parallel file system

引用

IEICE TRANSACTIONS ON INFORMATION AND systemS 2002年第4期E85D卷 714-721页

作者： Lee, YY Kim, CY Seo, DW Kyungpook Natl Univ Dept Elect Engn Taegu South Korea ETRI Comp Syst Res Dept Taejon South Korea

A parallel file system is normallly used to support excessive file requests from parallel applications in a cluster system, whereas prefetching is useful for improving the file system performance. This paper proposes dynamic file prefetching scheme based on file access patterns, named, table-comparison prefetching policy, that is particularly suitable for parallel scientific applications and multimedia web services in a VIA-based parallel file system. VIA relieves the communication overhead of traditional communication protocols, such as TCP/IP. The proposed policy introduces a table-comparison method to predict data for prefetching. In addition, it includes an algorithm to determine whether and when prefetching is performed using the current available I/O bandwidth. Experimental results confirmed that the use of the proposed prefetching policy in a VIA-based parallel HIP system produced a higher file system performance for various file access pattern.

关键词： parallel file system prefetching virtual interface architecture cluster systems

来源：评论

学校读者我要写书评

暂无评论

Enabling dynamic file I/O path selection at runtime for parallel file system

引用

JOURNAL OF SUPERCOMPUTING 2014年第2期68卷 996-1021页

作者： Li, Xiuqiao Xiao, Limin Qiu, Meikang Dong, Bin Ruan, Li State Key Lab Software Dev Environm Beijing 100191 Peoples R China Beihang Univ Sch Comp Sci & Engn Beijing 100191 Peoples R China San Jose State Univ Dept Comp Engn San Jose CA 95192 USA

parallel file systems are experiencing more and more applications from various fields. Various applications have different I/O workload characteristics, which have diverse requirements on accessing storage resources. However, parallel file systems often adopt the "one-size-fits-all" solution, which fails to meet specific application needs and hinders the full exploitation of potential performance. This paper presents a framework to enable dynamic file I/O path selection with fine granularity at runtime. The framework adopts a file handle-rich scheme to allow file systems choose corresponding optimizations to serve I/O requests. Consistency control algorithms are proposed to ensure data consistency while changing optimizations at runtime. One case study on our prototype shows that choosing proper optimizations can improve the I/O performance for small files and large files by up to 40 and 64.4 %, respectively. Another case study shows that the data prefetch performance for real-world application traces can be improved by up to 193 % by selecting correct prefetch patterns. Simulations in large-scale environment also show that our method is scalable and both the memory consumption and the consistency control overhead can be negligible.

关键词： parallel file system Rich file handle I/O path selection Consistency control Small file optimization Data prefetching

来源：评论

学校读者我要写书评

暂无评论

A parallel file system with Application-Aware Data Layout Policies for Massive Remote Sensing Image Processing in Digital Earth

引用

IEEE TRANSACTIONS ON parallel AND DISTRIBUTED systemS 2015年第6期26卷 1497-1508页

作者： Wang, Lizhe Ma, Yan Zomaya, Albert Y. Ranjan, Rajiv Chen, Dan Chinese Acad Sci Inst Remote Sensing & Digital Earth Beijing 100094 Peoples R China Univ Sydney Sch Informat Technol Sydney NSW 2006 Australia ICT CSIRO Marsfield NSW Australia Wuhan Univ Sch Comp Wuhan 430072 Hubei Peoples R China

Remote sensing applications in Digital Earth are overwhelmed with vast quantities of remote sensing (RS) image data. The intolerable I/O burden introduced by the massive amounts of RS data and the irregular RS data access patterns has made the traditional cluster based parallel I/O systems no longer applicable. We propose a RS data object-based parallel file system for remote sensing applications and implement it with the OrangeFS file system. It provides application-aware data layout policies, together with RS data object based data I/O interfaces, for efficient support of various data access patterns of RS applications from the server side. With the prior knowledge of the desired RS data access patterns, HPGFS could offer relevant space-filling curves to organize the sliced 3-D data bricks and distribute them over I/O servers. In this way, data layouts consistent with expected data access patterns could be created to explore data locality and achieve performance improvement. Moreover, the multi-band RS data with complex structured geographical metadata could be accessed and managed as a single data object. Through experiments on remote sensing applications with different access patterns, we have achieved performance improvement of about 30 percent for I/O and 20 percent overall.

关键词： parallel file system parallel I/O data-intensive computing digital earth big data remote sensing image processing

来源：评论

学校读者我要写书评

暂无评论

Enhancing hybrid parallel file system through performance and space-aware data layout

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS 2016年第4期30卷 396-410页

作者： He, Shuibing Liu, Yan Wang, Yang Sun, Xian-He Huang, Chuanhe Wuhan Univ State Key Lab Software Engn Comp Sch Wuhan Hubei Peoples R China Natl Univ Def Technol State Key Lab High Performance Comp Changsha Hunan Peoples R China Hunan Univ Coll Comp Sci & Elect Engn Changsha Hunan Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Shenzhen Peoples R China IIT Dept Comp Sci Chicago IL 60616 USA

Hybrid parallel file systems (PFSs), which consist of solid-state drive servers (SServer) and hard disk drive servers (HServer), have recently attracted growing attention. Compared to a traditional HServer, an SServer consistently provides improved storage performance but lacks storage space. However, most current data layout schemes do not consider the differences in performance and space between heterogeneous servers and may significantly degrade the performance of the hybrid PFSs. In this article, we propose performance and space-aware (PSA) scheme, a novel data layout scheme, which maximizes the hybrid PFSs' performance by applying adaptive varied-size file stripes. PSA dispatches data on heterogeneous file servers not only based on storage performance but also storage space. We have implemented PSA within OrangeFS, a popular PFS in the high-performance computing domain. Our extensive experiments with representative benchmarks, including IOR, HPIO, MPI-TILE-IO, and BTIO, show that PSA provides superior I/O throughput than the default and performance-aware file data layout schemes.

关键词： parallel I O system parallel file system data layout solid-state drive hybrid I O system

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：