检索结果-内蒙古大学图书馆

27th Euromicro International Conference on Parallel, distributed and Network-Based Processing (PDP)

作者： Jeong, Kisik Duffy, Carl Kim, Jin-Soo Lee, Joonwon Sungkyunkwan Univ Suwon South Korea Seoul Natl Univ Seoul South Korea

ISBN: (纸本)9781728116440

With increasing demand for running big data analytics and machine learning workloads with diverse data types, high performance computing ( HPC) systems consequently need to support diverse types of storage services. Ceph is one possible candidate for such HPC environments, as Ceph provides interfaces for object, block, and file storage. Ceph, however, is not designed for HPC environments, thus it needs to be optimized for HPC workloads. In this paper, we find and analyze problems that arise when running HPC workloads on Ceph, and propose a novel optimization technique called F2FS-split, based on the F2FS file system and several other optimizations. We measure the performance of Ceph in HPC environments, and show that F2FS-split outperforms both F2FS and XFS by 39% and 59%, respectively, in a write dominant workload. We also observe that modifying the Ceph RADOS object size can improve read speed further.

关键词： Ceph distributed file system high performance computing

来源：评论

学校读者我要写书评

暂无评论

BlinkFS: A distributed file system for intermittent power

BlinkFS: A distributed file system for intermittent power

引用

International Green Computing Conference (IGCC) of Sustainable Computing - Informatics and systems (SUSCOM)

作者： Sharma, Navin Irwin, David Shenoy, Prashant Univ Massachusetts CS Dept Amherst MA 01003 USA Univ Massachusetts ECE Dept Amherst MA 01003 USA

The ability to use intermittent power in data centers introduces numerous new opportunities for optimization, including (i) using real-time markets to buy more power when it is cheap, (ii) increasing the fraction of clean, but intermittent, renewable power, (iii) capping power for long periods to extend UPS lifetime during blackouts, and (iv) fully utilizing a data center's power delivery infrastructure. The capability to run off intermittent power also moves us closer to the vision of a net-zero data center that consumes no net energy from the electric grid and has a small carbon footprint. However, designing systems to operate under intermittent power is challenging, since applications often access persistent distributed state, where power fluctuations can impact data availability and I/O performance. To address the problem, we design and implement BlinkFS, which combines blinking with a power-balanced data layout and popularity-based replication/reclamation to optimize I/O throughput and latency as power varies. Our experiments show that BlinkFS outperforms approaches that co-opt existing energy-proportional distributed file systems (DFSs) for intermittent power, particularly at low steady power levels and high levels of intermittency. For example, we show that BlinkFS reduces completion time for MapReduce-style jobs by 42% at 50% full power compared to existing energy-proportional DFSs. (C) 2014 Elsevier Inc. All rights reserved.

关键词： distributed file system Intermittent power

来源：评论

学校读者我要写书评

暂无评论

Analytical Review on Hadoop distributed file system 5

Analytical Review on Hadoop Distributed File System

引用

5th International Conference on Confluence - The Next Generation Information Technology Summit (Confluence)

作者： Dwivedi, Kalpana Dubey, Sanjay Kumar Amity Univ Amity Sch Engn & Technol Noida Uttar Pradesh India

ISBN: (纸本)9781479942367

Hadoop distributed file system is used for processing, storing and analyzing very large amount of unstructured data. It stores the data reliably and provides fault tolerance, fast and scalable access to the information. It is used with MapReduce, which is a programming model. HDFS and Map Reduce are the core components of Hadoop. Hadoop is a framework of tools for large scale computation and data processing of large data sets. As we know data and information is exponentially increasing in current era therefore the technology like Hadoop, Cassandra file system, etc became the preferred choice among the IT professionals and business communities. Hadoop distributed file system is rapidly growing and proving itself as cutting edge technology in dealing with huge amount of structured and unstructured data. This paper includes step by step introduction to data management using file system, data management using RDBMS then need of Hadoop distributed file system, and its working process.

关键词： Hadoop HDFS distributed file system RDBMS

来源：评论

学校读者我要写书评

暂无评论

Design and Implementation of Metadata Cache Management Strategy for the distributed file system

引用

INTERNATIONAL JOURNAL OF GRID AND distributed COMPUTING 2014年第3期7卷 89-102页

作者： Liu, Jingning Chen, Junjian Tong, Wei Tian, Lei Chen, Cong Huazhong Univ Sci & Technol Wuhan Natl Lab Optoelect Sch Comp Sci & Technol Wuhan Hubei Peoples R China Foshan Univ Sch Elect & Informat Engn Foshan Peoples R China

Caching strategies and the resulting cache coherence control technique have become the key techniques for the system development. Based on the study of the cache technology of the home and abroad distributed file systems, and combined with the design requirements and characteristics for our multi-user parallel file system Cappella, this paper proposes a new meta-data cache management strategy. And this strategy not only improves the performance of metadata services, reducing the user access latency, but also controls metadata cache consistency very well.

关键词： distributed file system Metadata Cache Management Strategy

来源：评论

学校读者我要写书评

暂无评论

LineFS: Efficient SmartNIC Offload of a distributed file system with Pipeline Parallelism 21

LineFS: Efficient SmartNIC Offload of a Distributed File Sys...

引用

28th ACM Symposium on Operating systems Principles (SOSP)

作者： Kim, Jongyul Jang, Insu Reda, Waleed Im, Jaeseong Canini, Marco Kostic, Dejan Kwon, Youngjin Peter, Simon Witchel, Emmett Korea Adv Inst Sci & Technol Daejeon South Korea Univ Michigan Ann Arbor MI 48109 USA Univ Catholic Louvain KTH Royal Inst Technol Louvain Belgium KAUST Daejeon South Korea KTH Royal Inst Technol Daejeon South Korea Univ Texas Austin Austin TX 78712 USA Katana Graph Austin TX USA

ISBN: (纸本)9781450387095

In multi-tenant systems, the CPU overhead of distributed file systems (DFSes) is increasingly a burden to application performance. CPU and memory interference cause degraded and unstable application and storage performance, in particular for operation latency. Recent client-local DFSes for persistent memory (PM) accelerate this trend. DFS offload to SmartNICs is a promising solution to these problems, but it is challenging to fit the complex demands of a DFS onto simple SmartNIC processors located across PCIe. We present LineFS, a SmartNIC-offloaded, high-performance DFS with support for client-local PM. To fully leverage the SmartNIC architecture, we decompose DFS operations into execution stages that can be offloaded to a parallel datapath execution pipeline on the SmartNIC. LineFS offloads CPU-intensive DFS tasks, like replication, compression, data publication, index and consistency management to a SmartNIC. We implement LineFS on the Mellanox BlueField SmartNIC and compare it to Assise, a state-of-the-art PM DFS. LineFS improves latency in LevelDB up to 80% and through-put in filebench up to 79%, while providing extended DFS availability during host system failures.

关键词： distributed file system SmartNIC offload

来源：评论

学校读者我要写书评

暂无评论

A Direct Data Fetch Technology Applied in Disaster-Tolerant Model of distributed file system 4

A Direct Data Fetch Technology Applied in Disaster-Tolerant ...

引用

4th International Conference on Computer Science and Network Technology (ICCSNT)

作者： Li, Xigao Qian, Lin State Grid Elect Power Res Inst Nanjing Jiangsu Peoples R China

ISBN: (纸本)9781467381734

A geographically isolated disaster-tolerant system for distributed file system (DFS) is essential for its enhancement in data security. However, the plain disaster tolerant model performs poor quality while a switch happens in hot-backup. In this study, we proposed a Direct Data Fetch (DDF) technology for the hybrid model. The DDF can fetch data directly from the data chunk server while the main server is temporarily unavailable. Then, we implemented the model on MooseFS, and compared it with IBM Peer-to-Peer Remote Copy (PPRC) technology. The test results indicate that DDF technology enhanced the stability and performance during the switch process of disaster tolerant model, and performs best in small quantities of files.

关键词： MooseFS Direct Data Fetch distributed file system Disaster Tolerant

来源：评论

学校读者我要写书评

暂无评论

A Novel Algorithm Of Load Balancing In distributed file system For Cloud 2

A Novel Algorithm Of Load Balancing In Distributed File Syst...

引用

2015 International Conference on Innovations in Information, Embedded and Communication systems (ICIIECS)

作者： Pius, Shilpa V. Suresh, Shruthi Royal Coll Engn & Technol Comp Sci & Engn Akkikavu India

ISBN: (纸本)9781479968183

distributed file systems are building blocks for cloud computing. Here, nodes simultaneously serve computing and storage functions. file is partitioned into number of chunks allocated with distinct nodes so that tasks can be performed in parallel over the nodes. In a cloud computing environment, failure can occur, and nodes can be upgraded, replaced, and added to the system. files can be dynamically created, deleted, or appended, that results in load imbalance in the distributed file system. Emerging distributed file systems in production systems strongly depend on a central node for chunk reallocation. The dependence is clearly inadequate in large-scale environment because the central load balancer is put under considerable workload that is linearly scaled with the system size, and thus leads to performance bottleneck and the single point of failure. Here in this paper, a fully distributed load balancing algorithm is presented to cope with the load imbalance problem.

关键词： Cloud Computing distributed file system Load Balancing Map Reduce

来源：评论

学校读者我要写书评

暂无评论

OctopusFS: A distributed file system with Tiered Storage Management 17

OctopusFS: A Distributed File System with Tiered Storage Man...

引用

ACM International Conference on Management of Data

作者： Kakoulli, Elena Herodotou, Herodotos Cyprus Univ Technol 30 Archbishop Kyprianou Str CY-3036 Limassol Cyprus

ISBN: (纸本)9781450341974

The ever-growing data storage and I/O demands of modern large-scale data analytics are challenging the current distributed storage systems. A promising trend is to exploit the recent improvements in memory, storage media, and networks for sustaining high performance and low cost. While past work explores using memory or SSDs as local storage or combine local with network-attached storage in cluster computing, this work focuses on managing multiple storage tiers in a distributed setting. We present OctopusFS, a novel distributed file system that is aware of heterogeneous storage media (e.g., memory, SSDs, HDDs, NAS) with different capacities and performance characteristics. The system offers a variety of pluggable policies for automating data management across the storage tiers and cluster nodes. The policies employ multi-objective optimization techniques for making intelligent data management decisions based on the requirements of fault tolerance, data and load balancing, and throughput maximization. At the same time, the storage media are explicitly exposed to users and applications, allowing them to choose the distribution and placement of replicas in the cluster based on their own performance and fault tolerance requirements. Our extensive evaluation shows the immediate benefits of using OctopusFS with data-intensive processing systems, such as Hadoop and Spark, in terms of both increased performance and better cluster utilization.

关键词： distributed file system tiered storage management

来源：评论

学校读者我要写书评

暂无评论

Design and Performance Analysis of file Replication Strategy on distributed file system Using GridSim

引用

48th Annual Convention of Computer Society of India (CSI)

作者： Singh, Nirmal Singh, Sarbjeet Panjab Univ UIET Chandigarh 160014 India

ISBN: (纸本)9783319031071

distributed Computing systems like Peer-to-Peer, Web and Cloud are becoming popular day by day and are being used in a wide variety of data intensive applications. Data replication is an important concept which increases the availability and reliability of data in these systems. This paper presents the design and performance analysis of simulated implementation of data replication strategy on a distributed file system using GridSim toolkit. The parameters taken for the performance analysis are aggregate bandwidth, successful execution rate and system byte effective rate. The results indicate that the distributed file system making use of data replication strategy performs better, with respect to the parameters mentioned above, than the distributed file system which is not making use of data replication strategy. The integration of data replication scheme with distributed file system can greatly improve the availability and reliability of data.

关键词： distributed file system Replication Strategy Web Cloud Peer-to-Peer GridSim

来源：评论

学校读者我要写书评

暂无评论

Mining infrequently-accessed file correlations in distributed file system

Mining infrequently-accessed file correlations in distribute...

引用

9th Asia-Pacific Web Conference/8th International Conference on Web-Age Information Management (APWeb/WAIM 2007)

作者： Yu, Lihua Chen, Gang Dong, Jinxiang Zhejiang Univ Coll Comp Sci Hangzhou 310027 Peoples R China

ISBN: (纸本)9783540724834

file correlation mining, as a technique to enhance file system performance, can usually be exploited for many purposes such as to improve the effectiveness of cache, to optimize file layout, as well as to enable disk file prefetching. While most research works on file correlations focus on traditional stand-alone file systems, this paper investigates the problem of mining file correlations in a distributed environment. We present a parallel data mining algorithm called PFC-Miner (Parallel file Correlation Miner), which is based on Locality Sensitive Hashing. PFC-Miner can efficiently discover correlations between infrequently- accessed files which are more valuable for web applications. Experimental results show that PFC-Miner can efficiently discover file correlations in distributed file systems without compromiSmg the accuracy, and that the proposed approach has good scalability.

关键词： file correlation parallel data mining distributed file system

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：