检索结果-内蒙古大学图书馆

A prediction-based dynamic file assignment strategy for parallel file systems

parallel COMPUTING 2015年第Jan.期41卷 1-13页

作者： Long, Saiqin Zhao, Yuelong Chen, Wei Tang, Yuanbin S China Univ Technol Sch Comp Sci & Engn Guangzhou 510006 Guangdong Peoples R China Sichuan Univ Arts & Sci Modern Educ Technol Ctr Dazhou 635000 Peoples R China

Nowadays, the rapid development of the internet calls for a high performance file system, and a lot of efforts have already been devoted to the issue of assigning nonpartitioned files in a parallel file system with the aim of pursuing a prompt response to requests. Yet most of the existing strategies still fail to bring about an optimal performance on system mean response time metrics, and new strategies which can achieve better performance in terms of mean response time become indispensable for parallel file systems. This paper, while addressing the issue of assigning nonpartitioned files in parallel file systems where the file accesses exhibit Poisson arrival rates and fixed service times, presents an on-line file assignment strategy, named prediction-based dynamic file assignment (PDFA), to minimize the mean response time among disks under different workload conditions, and a comparison of the PDFA with the well-known file assignment algorithms, such as HP and SOR. Comprehensive experimental results show that PDFA is able to improve the performance consistently in terms of mean response time among all algorithms for comparison. (C) 2015 Published by Elsevier B.V.

关键词： file assignment parallel file systems Load prediction Genetic algorithm Mean response time

来源：评论

学校读者我要写书评

暂无评论

Cost-intelligent application-specific data layout optimization for parallel file systems

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2013年第2期16卷 285-298页

作者： Song, Huaiming Yin, Yanlong Chen, Yong Sun, Xian-He Dawning Informat Ind Co Ltd R&D Ctr Beijing 100084 Peoples R China IIT Dept Comp Sci Chicago IL 60616 USA Texas Tech Univ Dept Comp Sci Lubbock TX 79409 USA

parallel file systems have been developed in recent years to ease the I/O bottleneck of high-end computing system. These advanced file systems offer several data layout strategies in order to meet the performance goals of specific I/O workloads. However, while a layout policy may perform well on some I/O workload, it may not perform as well for another. Peak I/O performance is rarely achieved due to the complex data access patterns. Data access is application dependent. In this study, a cost-intelligent data access strategy based on the application-specific optimization principle is proposed. This strategy improves the I/O performance of parallel file systems. We first present examples to illustrate the difference of performance under different data layouts. By developing a cost model which estimates the completion time of data accesses in various data layouts, the layout can better match the application. Static layout optimization can be used for applications with dominant data access patterns, and dynamic layout selection with hybrid replications can be used for applications with complex I/O patterns. Theoretical analysis and experimental testing have been conducted to verify the proposed cost-intelligent layout approach. Analytical and experimental results show that the proposed cost model is effective and the application-specific data layout approach can provide up to a 74% performance improvement for data-intensive applications.

关键词： Data layout I/O performance modeling parallel file systems parallel I/O Data-intensive computing

来源：评论

学校读者我要写书评

暂无评论

Symmetric active/active metadata service for high availability parallel file systems

引用

JOURNAL OF parallel AND DISTRIBUTED COMPUTING 2009年第12期69卷 961-973页

作者： He, Xubin Ou, Li Engelmann, Christian Chen, Xin Scott, Stephen L. Tennessee Technol Univ Dept Elect & Comp Engn Cookeville TN 38505 USA Oak Ridge Natl Lab Div Math & Comp Sci Oak Ridge TN 37831 USA

High availability data storage systems are critical for many applications as research and business become more data driven. Since metadata management is essential to system availability, multiple metadata services are used to improve the availability of distributed storage systems. Past research has focused on the active/standby model, where each active service has at least one redundant idle backup. However, interruption of service and even some loss of service state may occur during a fail-over depending oil the replication technique used. In addition, the replication overhead for multiple metadata services call be very high. The research in this paper targets the symmetric active/active replication model, which uses Multiple redundant service nodes running in virtual synchrony. In this model, service node failures do not Cause a fail-over to a backup and there is no disruption of service or loss of service state. A fast delivery protocol is further discussed to reduce the latency of the total order broadcast needed. The prototype implementation shows that metadata service high availability call be achieved with all acceptable performance trade-off using the symmetric active/active metadata service Solution. (C) 2009 Elsevier Inc. All rights reserved

关键词： Metadata management Fault tolerance High availability parallel file systems Group communication

来源：评论

学校读者我要写书评

暂无评论

Client-side straggler-aware I/O scheduler for object-based parallel file systems

引用

parallel COMPUTING 2019年 82卷 3-18页

作者： Tavakoli, Neda Dai, Dong Chen, Yong Texas Tech Univ Comp Sci Dept Lubbock TX 79409 USA

Object-based parallel file systems have emerged as promising storage solutions for high-performance computing (HPC) systems. Despite the fact that object storage provides a flexible interface, scheduling highly concurrent I/O requests that access a large number of objects still remains as a challenging problem, especially in the case when stragglers (storage servers that are significantly slower than others) exist in the system. An efficient I/O scheduler needs to avoid possible stragglers to achieve low latency and high throughput. In this paper, we introduce a log-assisted straggler-aware I/O scheduling to mitigate the impact of storage server stragglers. The contribution of this study is threefold. First, we introduce a client-side, log-assisted, straggler-aware I/O scheduler architecture to tackle the storage straggler issue in HPC systems. Second, we present three scheduling algorithms that can make efficient decision for scheduling I/Os while avoiding stragglers based on such an architecture. Third, we evaluate the proposed I/O scheduler using simulations, and the simulation results have confirmed the promise of the newly introduced straggler-aware I/O scheduler. (C) 2018 Elsevier B.V. All rights reserved.

关键词： parallel file systems I/O scheduler Straggler High performance computing

来源：评论

学校读者我要写书评

暂无评论

PFAULT: A General Framework for Analyzing the Reliability of High-Performance parallel file systems 18

PFAULT: A General Framework for Analyzing the Reliability of...

引用

32nd International Conference on Supercomputing (ICS)

作者： Cao, Jinrui Gatla, Om Rameshwar Zheng, Mai Dai, Dong Eswarappa, Vidya Mu, Yan Chen, Yong New Mexico State Univ Las Cruces NM 88003 USA Texas Tech Univ Lubbock TX 79409 USA

ISBN: (纸本)9781450357838

High-performance parallel file systems (PFSes) are of prime importance today. However, despite the importance, their reliability is much less studied compared with that of local storage systems, largely due to the lack of an effective analysis methodology. In this paper, we introduce PFAULT, a general framework for analyzing the failure handling of PFSes. PFAULT automatically emulates the failure state of each storage device in the target PFS based on a set of well-defined fault models, and enables analyzing the recoverability of the PFS under faults systematically. To demonstrate the practicality, we apply PFAULT to study Lustre, one of the most widely used PFSes. Our analysis reveals a number of cases where Lustre's checking and repairing utility LFSCK fails with unexpected symptoms (e.g., I/O error, hang, reboot). Moreover, with the help of PFAULT, we are able to identify a resource leak problem where a portion of Lustre's internal namespace and storage space become unusable even after running LFSCK. On the other hand, we also verify that the latest Lustre has made noticeable improvement in terms of failure handling comparing to a previous version. We hope our study and framework can help improve PFSes for reliable high-performance computing.

关键词： parallel file systems reliability high performance computing

来源：评论

学校读者我要写书评

暂无评论

A Statistical Analysis of the Performance Variability of Read/Write Operations on parallel file systems

A Statistical Analysis of the Performance Variability of Rea...

引用

International Conference on Computational Science (ICCS)

作者： Inacio, Eduardo C. Barbetta, Pedro A. Dantas, Mario A. R. Univ Fed Santa Catarina Florianopolis SC Brazil

This paper reports a statistical analysis about the performance variability of read and write operations on parallel file systems. To properly account for the inherent system variability and to obtain statistically significant results, formal experimental design and analysis methods were employed in this study. This research reveals that in the evaluated conditions six effects dominate I/O time, responding for 99.32% of the performance variability. Further, some factors traditionally explored in I/O optimization proposals presented no statistical evidence of significant effects in this study. Moreover, high-level effects were identified by the interpretation of the set of statistically significant factors, providing a case for further research in the subject. (C) 2017 The Authors. Published by Elsevier B.V.

关键词： parallel file systems performance variability experimental design statistical analysis

来源：评论

学校读者我要写书评

暂无评论

AGIOS: Application-guided I/O Scheduling for parallel file systems

AGIOS: Application-guided I/O Scheduling for Parallel File S...

引用

19th IEEE International Conference on parallel and Distributed systems (ICPADS)

作者： Boito, Francieli Zanon Kassick, Rodrigo Virote Navaux, Philippe O. A. Denneulin, Yves Univ Fed Rio Grande do Sul Inst Informat Porto Alegre RS Brazil Univ Grenoble INRIA LIG Lab Grenoble France

ISBN: (纸本)9781479920815

In this paper, we improve the performance of server-side I/O scheduling on parallel file systems by transparently including information about the applications' access patterns. Server-side I/O scheduling is a valuable tool on multi-application scenarios, where the applications' spatial locality suffers from interference caused by concurrent accesses to the file system. We present AGIOS, an I/O scheduling library for parallel file systems. We guide scheduler's decisions by including information about the applications' future requests. This information is obtained from traces generated by the scheduler itself, without changes in application or file system. Our approach shows performance improvements under different workloads of 46.3% on average when compared to a scenario without an I/O scheduler, and of 25.1% when compared to a scheduler which does not use information about future accesses.

关键词： Disk Access I/O I/O Scheduling parallel file systems

来源：评论

学校读者我要写书评

暂无评论

Performance Impact of Operating systems' Caching Parameters on parallel file systems 15

Performance Impact of Operating Systems' Caching Parameters ...

引用

30th ACM Symposium on Applied Computing (SAC)

作者： Inacio, Eduardo C. Dantas, Mario A. R. Boito, Francieli Z. Navaux, Philippe O. A. de Macedo, Douglas D. J. Univ Fed Santa Catarina Dept Informat & Stat Florianopolis SC Brazil Univ Fed Rio Grande do Sul Inst Informat Porto Alegre RS Brazil Univ Fed Sergipe Dept Comp Sao Cristovao SE Brazil

ISBN: (纸本)9781450331968

This research work investigates the peformance impact of operating systems' (OS) caching parameters on a parallel file system (PFS). Through an extensive experimental analysis, an analytical performance model is proposed in order to reflect caching effects on performance of file write operations. A qualitative and quantitative evaluation of 855 test cases over 3 different platforms was performed. Results indicate that the proposed model is effective in representing caching effects, identifying both situation and intensity of performance degradation. Observed mean absolute percentage error (MAPE) of predicted values is less than 36%.

关键词： Performance analysis operating systems' caching parameters parallel file systems performance model

来源：评论

学校读者我要写书评

暂无评论

A Cost-intelligent Application-specific Data Layout Scheme for parallel file systems 11

A Cost-intelligent Application-specific Data Layout Scheme f...

引用

20th International Symposium on High Performance Distributed Computing

作者： Song, Huaiming Yin, Yanlong Chen, Yong Sun, Xian-He IIT Dept Comp Sci Chicago IL 60616 USA

ISBN: (纸本)9781450305525

I/O data access is a recognized performance bottleneck of high-end computing. Several commercial and research parallel file systems have been developed in recent years to ease the performance bottleneck. These advanced file systems perform well on some applications but may not perform well on others. They have not reached their full potential in mitigating the I/O-wall problem. Data access is application dependent. Based on the application-specific optimization principle, in this study we propose a cost-intelligent data access strategy to improve the performance of parallel file systems. We first present a novel model to estimate data access cost of different data layout policies. Next, we extend the cost model to calculate the overall I/O cost of any given application and choose an appropriate layout policy for the application. A complex application may consist of different data access patterns. Averaging the data access patterns may not be the best solution for those complex applications that do not have a dominant pattern. We then further propose a hybrid data replication strategy for those applications, so that a file can have replications with different layout policies for the best performance. Theoretical analysis and experimental testing have been conducted to verify the newly proposed cost-intelligent layout approach. Analytical and experimental results show that the proposed cost model is effective and the application-specific data layout approach achieved up to 74% performance improvement for data-intensive applications.

关键词： data layout data-access performance modeling data-intensive parallel file systems

来源：评论

学校读者我要写书评

暂无评论

Toward Comprehensive and Accurate Simulation Performance Prediction of parallel file systems

Toward Comprehensive and Accurate Simulation Performance Pre...

引用

42nd Annual IEEE/IFIP International Conference on Dependable systems and Networks (DSN)

作者： Erazo, Miguel A. Li, Ting Liu, Jason Eidenbenz, Stephan Florida Int Univ Sch Comp & Informat Sci Miami FL 33199 USA Los Alamos Natl Lab Informat Sci CCS 3 Los Alamos NM USA

ISBN: (纸本)9781467316255

We present the design and implementation of fileSim, a simulation framework with detailed models of parallel file systems, capable of reproducing the complex I/O behavior at scale. fileSim aims to support comprehensive and accurate end-to-end I/O performance prediction and evaluation of exascale high-end computing systems. To this end, fileSim provides several key features, including detailed, pluggable models of contemporary parallel file systems, the support of trace-driven simulation, and the capability of running large-scale I/O systems using parallel and distributed simulation. We conducted extensive validation and performance studies, through which we show that the simulator is capable of reproducing important I/O system behaviors comparable to those measured from the real systems. We demonstrate the capabilities of fileSim as a tool for exploring the parameter space and design alternatives of large-scale parallel file systems.

关键词： parallel file systems PanFS simulation and modeling parallel simulation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：