检索结果-内蒙古大学图书馆

AFPT: Accelerating Read Performance of in-memory file system Through Adaptive file Page Table

ENGINEERING LETTERS 2019年第2期27卷 364-373页

作者： Cui, Bingde Zhang, Huansheng Hebei Univ Water Resources & Elect Engn Dept Comp Sci Handan 061001 Peoples R China

Emerging non-volatile memory (NVM) technologies are expected to revolutionize storage systems by providing cheap, persistent and fast data accesses through memory bus interface. In oder to fully exploit NVM, many in-memory file systems are proposed to achieve excellent performance and strong consistency. Besides, to mitigate the read-write asymmetric problem of NVM, many optimization strategies are designed to hide the long write latency to NVM in critical path of file operations, such as path resolution. However, we find that the index structure of state-of-the-art in-memory file systems cannot provide fast read performance in various use scenarios. In this paper, we propose Adaptive file Page Table (AFPT), a novel index scheme that combines software search and MMU mapping to provide excellent read performance for different workloads. For small requests, software search routines are used to locate data pages by traversing the file index structure. For large requests, we allocate a continuous address space and build file page table to utilize hardware MMU for address translation. A Cost Model is proposed to determine when to build page table for a file. This model is 1.38-competitive against optimal solution. We implement AFPT in PMFS and NOVA and evaluate the performance with micro-benchmarks and application workloads. The experimental results show that AFPT improves file system performance by up to 55.62% and 41.78% for NOVA and PMFS, respectively.

关键词： in-memory file system file Page Table Virtual Address Space Performance Optimization

来源：评论

学校读者我要写书评

暂无评论

HMFS: A hybrid in-memory file system with version consistency

引用

JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING 2018年 117卷 18-36页

作者： Liu, Hao Huang, Linpeng Zhu, Yanmin Zheng, Shengan Shen, Yanyan Shanghai Jiao Tong Univ Dept Comp Sci & Engn Shanghai Peoples R China

Emerging non-volatile memory (NVM) such as PCM and STT-RAM has memory-like byte-addressability as well as disk-like persistent storage capability. It offers an opportunity to bring NVM into the existing computer architecture for constructing an efficient and high-performance in-memory file system. Several NVM-optimized file systems have been designed. However, most of them fail to exploit all important features of NVM, and can only guarantee the file system consistency to the data consistency level. In this paper, we present HMFS, a hybrid in-memory full-versioning file system. HMFS manages DRAM and NVM in a unified address space and adopts different updating mechanisms to them. Besides, HMFS achieves version consistency with a simple and efficient multi-version approach. Experimental results show that HMFS achieves significant throughput improvement comparing with the state-of-the-art NVM-optimized file systems, such as PMFS and NOVA, and 3.1x to 13.5x higher versioning efficiency compared to some other multi-versioned file system such as BTRFS and NILFS2. (C)2018 Published by Elsevier Inc.

关键词： Non-volatile memory Hybrid memory architecture in-memory file system Multi-versioning

来源：评论

学校读者我要写书评

暂无评论

Performance Optimization of in-memory file system in Distributed Storage system

Performance Optimization of In-Memory File System in Distrib...

引用

12th International Conference on Networking, Architecture, and Storage (NAS)

作者： Li, Zhaowei Yan, Yunlong Mo, Jintao Wen, Zhaocong Wu, Junmin Guangdong Power Grid Co Ltd Dongguan Power Supply Bur Guangzhou Guangdong Peoples R China Univ Sci & Technol China Suzhou Inst Adv Study Hefei Anhui Peoples R China

ISBN: (纸本)9781538634868

Hadoop as an open source framework for dealing with Big Data can be processed to calculate large amounts of data in parallel, which has attracted more and more attention in academia and industry. This paper analyzes the methods of in-memory file system using HDFS Lazy Persist strategy and Alluxio to upgrade system I/O efficiency. Besides, in order to avoid the problem that Lazy Persist strategy needs to be triggered manually each time, we propose HDFS Lazy Persist strategy automatic trigger mechanism based on the statistics of data access information.

关键词： in-memory file system Hadoop Distributed file system Alluxio Big Data

来源：评论

学校读者我要写书评

暂无评论

in-memory file system with Efficient Swap Support for Mobile Smart Devices

引用

IEEE TRANSACTIONS ON CONSUMER ELECTRONICS 2016年第3期62卷 275-282页

作者： Choi, Jungsik Ahn, Joonwook Kim, Jiwon Ryu, Sungtae Han, Hwansoo Sungkyunkwan Univ 2066 Seobu Ro Suwon 16419 South Korea Samsung Elect 1-1 Samsungjeonja Ro Hwaseong 18448 South Korea

As the number of file I/O intensive mobile applications increase, in-memory file systems have gained attention. in-memory file systems provide a fast access to file data, but the main memory of the corresponding mobile devices often suffers due to its capacity limit. To provide mobile applications with more free space, the swap mechanism has been an essential technique for the overcoming of this space insufficiency;however, the existing swap mechanism is inadequate for the file I/O of mobile applications. In this study, a new in-memory file system is devised to improve the current swap mechanism in the legacy in-memory file system, whereby strip-based layouts with separated file-swap partitions are used and unnecessary file-I/O overheads are eliminated. To evaluate the proposed in-memory file system, experiments with the file-I/O benchmark were conducted. The experiment results show that, compared to the current swap scheme, the proposed swap scheme improves the overall performance by ten times on average(1).

关键词： Mobile devices in-memory file system Swap mechanism file I/O performance

来源：评论

学校读者我要写书评

暂无评论

Overcoming data locality: An in-memory runtime file system with symmetrical data distribution

引用

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2016年 54卷 144-158页

作者： Uta, Alexandru Sandu, Andreea Kielmann, Thilo Vrije Univ Amsterdam Dept Comp Sci Amsterdam Netherlands

In many-task computing (MTC), applications such as scientific workflows or parameter sweeps communicate via intermediate files;application performance strongly depends on the file system in use. The state of the art uses runtime systems providing in-memory file storage that is designed for data locality: files are placed on those nodes that write or read them. With data locality, however, task distribution conflicts with data distribution, leading to application slowdown, and worse, to prohibitive storage imbalance. To overcome these limitations, we present MemFS, a fully symmetrical, in-memory runtime file system that stripes files across all compute nodes, based on a distributed hash function. Our cluster experiments with Montage and BLAST workflows, using up to 512 cores, show that MemFS has both better performance and better scalability than the state-of-the-art, locality-based file system, AMFS. Furthermore, our evaluation on a public commercial cloud validates our cluster results. On this platform MemFS shows excellent scalability up to 1024 cores and is able to saturate the 10G Ethernet bandwidth when running BLAST and Montage. (c) 2015 Elsevier B.V. All rights reserved.

关键词： Many-task computing in-memory file system Distributed hashing Scalability

来源：评论

学校读者我要写书评

暂无评论

MemEFS: A network-aware elastic in-memory runtime distributed file system

引用

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年 82卷 631-646页

作者： Uta, Alexandru Danner, Ove van der Weegen, Cas Oprescu, Ana-Maria Sandu, Andreea Costache, Stefania Kielmann, Thilo Vrije Univ Amsterdam Dept Comp Sci Amsterdam Netherlands Univ Amsterdam Informat Inst Amsterdam Netherlands

Scientific domains such as astronomy or bioinformatics produce increasingly large amounts of data that need to be analyzed. Such analyses are modeled as scientific workflows - applications composed of many individual tasks that exhibit data dependencies. Typically, these applications suffer from significant variability in the interplay between achieved parallelism and data footprint. To efficiently tackle the data deluge, cost effective solutions need to be deployed by extending private computing infrastructures with public cloud resources. To achieve this, two key features for such systems need to be addressed: elasticity and network adaptability. The former improves compute resource utilization efficiency, while the latter improves network utilization efficiency, since public clouds suffer from significant bandwidth variability. This paper extends our previous work on MemEFS, an in-memory elastic distributed file system by adding network adaptability. Our results show that MemEFS' elasticity increases the resource utilization efficiency by up to 65%. Regarding the network adaptation policy, MemEFS achieves up to 50% speedup compared to its network-agnostic counterpart. (C) 2017 Elsevier B.V. All rights reserved.

关键词： in-memory file system Distributed hashing Elasticity Scalable computing Network variability Network adaptation High-performance I/O Large-scale scientific computing Big data and HPC systems Big data for e-Science Large-scale systems for computational sciences

来源：评论

学校读者我要写书评

暂无评论

PERFORMANCE-EFFICIENT RECOMMENDATION AND PREDICTION SERVICE FOR BIG DATA FRAMEWORKS FOCUSING ON DATA COMPRESSION AND IN-memory DATA STORAGE INDICATORS

引用

SCALABLE COMPUTING-PRACTICE AND EXPERIENCE 2021年第4期22卷 401-412页

作者： Astsatryan, Hrachya Lalayan, Arthur Kocharyan, Aram Hagimont, Daniel Natl Acad Sci Armenia Inst Informat & Automat Problems 1 Paruyr Sevak Str Yerevan 0014 Armenia Natl Polytech Univ Armenia 105Teryan Str Yerevan 0009 Armenia Univ Fed Toulouse Midi Pyrenees Toulouse F-31000 Toulouse 7 France

The MapReduce framework manages Big Data sets by splitting the large datasets into a set of distributed blocks and processes them in parallel. Data compression and in-memory file systems are widely used methods in Big Data processing to reduce resource-intensive I/O operations and improve I/O rate correspondingly. The article presents a performance-efficient modular and configurable decision-making robust service relying on data compression and in-memory data storage indicators. The service consists of Recommendation and Prediction modules, predicts the execution time of a given job based on metrics, and recommends the best configuration parameters to improve Hadoop and Spark frameworks' performance. Several CPU and data -intensive applications and micro-benchmarks have been evaluated to improve the performance, including Log Analyzer, WordCount, and K-Means.

关键词： Hadoop Spark MapReduce data compression in-memory file system

来源：评论

学校读者我要写书评

暂无评论

DWARM: A wear-aware memory management scheme for in-memory file systems

引用

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年 88卷 1-15页

作者： Wu, Lin Zhuge, Qingfeng Sha, Edwin Hsing-Mean Chen, Xianzhang Cheng, Linfeng Chongqing Univ Dept Comp Sci Chongqing 400044 Peoples R China East China Normal Univ Sch Comp Sci & Software Engn Shanghai 200241 Peoples R China

Emerging non-volatile memories (NVMs) are promised to revolutionize storage systems by providing fast, persistent data accesses on memory bus. A hybrid NVM/DRAM architecture that combines faster, volatile DRAM with slightly slower, denser NVM can harness the characteristics of both technologies. In order to fully take advantage of NVM, state-of-the-art in-memory file systems are designed to provide high performance and strong consistency guarantees. However, the free space management schemes of existing in-memory file systems can easily cause "hot spots" when updating data structures on NVM, leading to significant skewness in terms of writes to each data page. In this paper, we propose dynamic wear-aware range management (DWARM) scheme, a novel free space management technique for in-memory file systems. This scheme achieves wear-leveling with high performance for allocationideallocation. The essential idea is to allocate less-written pages for each allocation request. Specifically, this scheme works by associating a write counter with each data page and updating the counters in the file write path. We build an "index" structure to fast locate the pages that have received less writes. The index divides the NVM pages into different subranges according to the write counters. Allocation always starts from the minimal subrange. Also, we propose Adaptive Wear Range Determination Algorithm to adjust the wear ranges dynamically. To accelerate lookup, we keep the index in DRAM and avoid the overhead of strong consistency by rebuilding the index in case of system failure. Experimental results show that this scheme can provide 4.9x to 158.1x wear-leveling improvement compared to the state-of-the-art memory management schemes. For application workloads, the DWARM strategy can improve the lifetime of NVM by up to 125 x, 39x, and 25x, compared with the standard memory management schemes of PMFS, NOVA and SIMFS. (C) 2018 Published by Elsevier B.V.

关键词： in-memory file system Wear leveling memory management Index

来源：评论

学校读者我要写书评

暂无评论

Accelerating big data analytics on HPC clusters using two-level storage

引用

PARALLEL COMPUTING 2017年 61卷 18-34页

作者： Xuan, Pengfei Ligon, Walter B. Srimani, Pradip K. Ge, Rong Luo, Feng Clemson Univ Sch Comp Clemson SC 29634 USA Clemson Univ Elect & Comp Engn Clemson SC 29634 USA

Data-intensive applications that are inherently I/O bound have become a major workload on traditional high-performance computing (HPC) clusters. Simply employing data intensive computing storage such as HDFS or using parallel file systems available on HPC clusters to serve such applications incurs performance and scalability issues. In this paper, we present a novel two-level storage system that integrates an upper-level in-memory file system with a lower-level parallel file system. The former renders memory-speed high I/O performance and the latter renders consistent storage with large capacity. We build a two level storage system prototype with Tachyon and OrangeFS, and analyze the resulting I/O throughput for typical MapReduce operations. Theoretical modeling and experiments show that the proposed two-level storage delivers higher aggregate I/O throughput than HDFS and OrangeFS and achieves scalable performance for both read and write. We expect this two-level storage approach to provide insights on system design for big data analytics on HPC clusters. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Two-level storage in-memory file system Parallel file system Data-intensive computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：