检索结果-内蒙古大学图书馆

Distributed and Scalable Directory Service in a parallel file system

IEICE TRANSACTIONS ON INFORMATION AND systemS 2016年第2期E99D卷 313-323页

作者： Wang, Lixin Lu, Yutong Zhang, Wei Lei, Yan Natl Univ Def Technol Coll Comp Changsha Hunan Peoples R China Natl Univ Def Technol Coll Comp Syst Software Lab Changsha Hunan Peoples R China

One of the patterns that the design of parallel file systems has to solve stems from the difficulty of handling the metadata-intensive I/O generated by parallel applications accessing a single large directory. We demonstrate a middleware design called SFS to support existing parallel file systems for distributed and scalable directory service. SFS distributes directory entries over data servers instead of metadata servers to offer increased scalability and performance. Firstly, SFS exploits an adaptive directory partitioning based on extendible hashing to support concurrent and unsynchronized partition splitting. Secondly, SFS describes an optimization based on recursive split-ordering that emphasizes speeding up the splitting process. Thirdly, SFS applies a write-optimized index structure to convert slow, small, random metadata updates into fast, large, sequential writes. Finally, SFS gracefully tolerates stale mapping at the clients while maintaining the correctness and consistency of the system. Our performance results on a cluster of 32-servers show our implementation can deliver more than 250,000 file creations per second on average.

关键词： parallel file system distributed and scalable directory service concurrent and unsynchronized splitting recursive split-ordering write-optimized index structure

来源：评论

学校读者我要写书评

暂无评论

RAMA: An easy-to-use, high-performance parallel file system

引用

parallel COMPUTING 1997年第4-5期23卷 419-446页

作者： Miller, EL Katz, RH UNIV CALIF BERKELEY DIV COMP SCIBERKELEYCA 94720

Modem massively parallel file systems provide high bandwidth file access by striping files across arrays of disks attached to a few specialized I/O nodes, However, these file systems are hard to use and difficult to integrate with workstations and tertiary storage. RAMA addresses these problems by providing a high-performance massively parallel file system with a simple interface, RAMA uses hashing to pseudo-randomly distribute data to all of its disks, insuring high bandwidth regardless of access pattern and eliminating bottlenecks in file block accesses, This flexibility does not cause a large loss of performance - RAMA's simulated performance is within 10-15% of the optimum performance of a similarly-sized striped file system, and is a factor of 4 or more better than a striped file system with poorly laid out data.

关键词： parallel file system parallel disk pseudo-random file block layout high-performance I/O scientific computing

来源：评论

学校读者我要写书评

暂无评论

Understanding the Effect of Multiple Factors on a parallel file system's Performance 24

Understanding the Effect of Multiple Factors on a Parallel F...

引用

IEEE 24th International Conference on Enabling Technologies Infrastructures for Collaborative Enterprises

作者： Inacio, Eduardo C. Pilla, Laercio L. Dantas, Mario A. R. Univ Fed Santa Catarina Dept Informat & Stat INE Florianopolis SC Brazil

ISBN: (纸本)9781467376921

This research work presents an investigation of the impact of a wide range of factors on the performance of parallel file systems (PFSs). It is the result of an extensive test campaign with three distinct computing platforms and value variations for eleven factors that advance the understanding of PFSs' behavior under different conditions. Our main contributions are the characterization of effects not fully explained or misunderstood in the literature previously. First, we demonstrate that no significant performance variation (approximate to 6%) is observed when choosing one among four TCP congestion-avoidance algorithms. Second, we detail the effect of the page cache of I/O nodes on a PFS's write throughput and how it relates to other factors.

关键词： Page cache parallel file system Performance TCP Congestion-avoidance Algorithms file systems multiple factor Research work congestion avoidance parallel Lines expanding test

来源：评论

学校读者我要写书评

暂无评论

Performance analysis of parallel file system for I/O load balancing in distributed applications

Performance analysis of parallel file system for I/O load ba...

引用

International Conference on parallel and Distributed Processing Techniques and Applications

作者： Deepalakshmi, S Baswaraj, K Rao, SSSP Jagdish, BS Sonavane, DD Dhekne, PS Kaura, HK Indian Inst Technol Dept Comp Sci & Engn Bombay 400076 Maharashtra India

ISBN: (纸本)1892512459

Cluster environment these days is gaining recognition in equivalence to complex parallel machines. This provides environment to utilize distributed resources in an efficient way. Complex applications can be run on clusters and at the same time have advantage of the power of high-end machines due to the availability of resources in multiples across the cluster. In order to provide transparency in data location we need some kind of interface, which would provide access to data that has been distributed across the nodes. parallel file system for Intel Networked Work-stations (PFSNIW) is one such parallel file system that has been developed by a team in Indian Institute of Technology, Bombay. In this paper we are discussing the effect of different network parameters on the performance of PFSNIW. We also discuss the results of the experiments that have been conducted to get optimal values for different parameters that influence the performance of the file system.

关键词： I/O load balancing parallel file system cluster computing performance analysis performance issues

来源：评论

学校读者我要写书评

暂无评论

Clusterfile: a flexible physical layout parallel file system

Clusterfile: a flexible physical layout parallel file system

引用

International Conference on Cluster Computing (Cluster 2001)

作者： Isaila, F Tichy, WF Univ Karlsruhe Fak Informat D-76128 Karlsruhe Germany

This paper presents Clusterfile, a parallel file system that provides parallel file access on a cluster of computers. We introduce a file partitioning model that has been used in the design of Clusterfile. The model uses a data representation that is optimized for multidimensional array partitioning while allowing arbitrary partitions. The paper shows how the file model can be employed for file partitioning into both physical subfiles and logical views. We also present how the conversion between two partitions of the same file is implemented using a general memory redistribution algorithm. We show how we use the algorithm to optimize non-contiguous read and write operations. The experimental results include performance comparisons with the parallel Virtual file system (PVFS) and an MPI-IO implementation for PVFS. Copyright (C) 2003 John Wiley Sons, Ltd.

关键词： parallel file system parallel I/O physical and logical partitioning non-contiguous I/O data redistribution mapping functions

来源：评论

学校读者我要写书评

暂无评论

Fault Tolerant in the Expand Ad-Hoc parallel file system 30th

Fault Tolerant in the Expand Ad-Hoc Parallel File System

引用

30th European Conference on parallel and Distributed Processing (Euro-Par)

作者： Munoz-Munoz, Dario Garcia-Carballeira, Felix Camarmas-Alonso, Diego Calderon-Mateos, Alejandro Carretero, Jesus Univ Carlos III Madrid Comp Sci & Engn Dept Ave Univ 30 Madrid 28911 Spain

ISBN: (纸本)9783031697654;9783031697661

In the last years, applications related to Artificial Intelligence and big data, among others, have been involved. There is a need to improve I/O operations to avoid bottlenecks in accessing a larger amount of data. For this purpose, the Expand Ad-Hoc parallel file system is being designed and developed. Since these applications have very long execution times, fault tolerance mechanisms in the file system are necessary to allow them to continue running in the presence of failures. This work introduces a fault-tolerant design based on data replication for the Expand Ad-Hoc parallel file system and an initial evaluation conducted on the HPC4AI Laboratory supercomputer in Torino. The evaluation of Expand Ad-Hoc with fault-tolerant found that, despite data replication, its performance and scalability are generally better than those of other parallel file systems without fault-tolerant.

关键词： Fault tolerance Expand Ad-Hoc parallel file system Ad-Hoc file system

来源：评论

学校读者我要写书评

暂无评论

Improving GridFTP transfers by means of a multiagent parallel file system

引用

MULTIAGENT AND GRID systemS 2007年第4期3卷 441-451页

作者： Sanchez, Alberto Perez, Maria S. Gueant, Pierre Montes, Jesus Herrero, Pilar Cortes, Toni UPM Fac Informat Campus Montegancedo S-N Madrid Spain Barcelona Supercomp Ctr BSC Barcelona Spain

Data transfer in a grid environment has become one critical activity in a large number of applications that require access to huge volumes of data. In these scenarios, characterized by large latencies, poor performance and complex dependencies, the use of approaches such as multiagents or parallel I/O can provide a great benefit. However, all the attempts to improve the performance of data transfer in grids should achieve the interoperability with already established data transfer schemes. GridFTP is one of the most known and used data grid transfer protocols. This paper describes MAPFS-DSI, a modification of the GridFTP server, based on a multiagent parallel file system. MAPFS-DSI increases the performance of data transfers, but keeping the interoperability with existing GridFTP servers.

关键词： Data transfer GridFTP parallel file system multiagent system MAPFS

来源：评论

学校读者我要写书评

暂无评论

On the Design and Implementation of a Simulator for parallel file system Research

On the Design and Implementation of a Simulator for Parallel...

引用

29th IEEE Symposium on Massive Storage systems and Technologies (MSST) - Research Track

作者： Liu, Yonggang Figueiredo, Renato Xu, Yiqi Zhao, Ming Univ Florida Dept Elect & Comp Engn Gainesville FL 32610 USA Florida Int Univ Sch Comp & Informat Sci Miami FL USA

ISBN: (纸本)9781479902170;9781479902187

Due to the popularity and importance of parallel file systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.

关键词： simulation parallel file system I/O scheduling

来源：评论

学校读者我要写书评

暂无评论

Performance Evaluation of A Infiniband-based Lustre parallel file system

Performance Evaluation of A Infiniband-based Lustre Parallel...

引用

2nd International Conference on Challenges in Environmental Science and Computer Engineering (CESCE)

作者： Wang, Yuan Lu, Yongquan Qiu, Chu Gao, Pengdong Wang, Jintao Commun Univ China Ctr High Performance Comp Beijing 100024 Peoples R China

As one of the most widely used parallel file systems, Lustre plays an important role in High Performance Computing (HPC). In this paper, an infiniband-based Lustre parallel file system is built firstly. And then a series of experiments are executed to evaluate the performance of luster with infiniaband. Some parameters have great influence on the result, such as size record, client count, stripe count and stripe size. Experimental results indicate that the higher is the speed of network the higher aggregation bandwidth luster could provide. Moreover, Client count and stripe count have more impact on the performance of luster than the record size and stripe size. Appropriate record size and stripe size could improve the performance a little, but not significantly. (C) 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the Intelligent Information Technology Application Research Association.

关键词： Lustre infiniband parallel file system high performance computing performance evaluation

来源：评论

学校读者我要写书评

暂无评论

StageFS: A parallel file system Optimizing Metadata Performance for SSD Based Clusters 15

StageFS: A Parallel File System Optimizing Metadata Performa...

引用

15th IEEE Int Conf on Trust, Security and Privacy in Comp and Commun / 10th IEEE Int Conf on Big Data Science and Engineering / 14th IEEE Int Symposium on parallel and Distributed Proc with Applicat (IEEE Trustcom/BigDataSE/ISPA)

作者： Wu, Huijun Zhu, Liming Wu, Dongyao Lu, Kai Li, Gen Univ New South Wales Data61 CSIRO Kensington NSW Australia Natl Univ Def Technol Changsha Hunan Peoples R China

ISBN: (纸本)9781509032051

parallel file systems are important infrastructures for both cloud and high performance computing. The performance of metadata operations is critical to achieve high scalability in parallel file systems. Nevertheless, traditional parallel file systems are lack of scalable metadata service. To alleviate these problems, some previous research distributes metadata to separated large-scale clusters and uses write-optimized techniques like log-structured merge tree (LSM-tree) to store metadata. However, LSM-tree design does not consider the features of solid state drive devices (SSD) which are widely deployed in modern parallel computing systems. The design of using LSM-trees to store metadata has not explored the potential benefits of SSD devices. In this paper, we present StageFS, which is a parallel file system optimized for SSD based clusters. StageFS stores both the metadata and small files in LSM-trees for fast indexing. For larger files, the file blocks are separately stored to reduce the write amplifications. In addition, the parallel I/O feature of SSD devices is used to improve the performance of accessing directories and large files. To avoid frequent small writes, StageFS uses buffering to better utilize the bandwidth of SSD devices. Experimental results show that StageFS provides better performance in metadata operations (up to 21.28x) and small file access (1.92x to two orders of magnitude) compared with Ceph and HDFS.

关键词： parallel file system metadata LSM-tree small file

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：