检索结果-内蒙古大学图书馆

15th European Conference on Computer systems (EuroSys)

作者： Kogan, Alex Dice, Dave Issa, Shady Oracle Labs Burlington MA 01803 USA Univ Lisbon Lisbon Portugal INESC ID Lisbon Portugal

ISBN: (纸本)9781450368827

Range locks are a synchronization construct designed to provide concurrent access to multiple threads (or processes) to disjoint parts of a shared resource. Originally conceived in the file system context, range locks are gaining increasing interest in the Linux kernel community seeking to alleviate bottlenecks in the virtual memory management subsystem. The existing implementation of range locks in the kernel, however, uses an internal spin lock to protect the underlying tree structure that keeps track of acquired and requested ranges. This spin lock becomes a point of contention on its own when the range lock is frequently acquired. Furthermore, where and exactly how specific (refined) ranges can be locked remains an open question. In this paper, we make two independent, but related contributions. First, we propose an alternative approach for building range locks based on linked lists. The lists are easy to maintain in a lock-less fashion, and in fact, our range locks do not use any internal locks in the common case. Second, we show how the range of the lock can be refined in the mprotect operation through a speculative mechanism. This refinement, in turn, allows concurrent execution of mprotect operations on non-overlapping memory regions. We implement our new algorithms and demonstrate their effectiveness in user-space and kernel-space, achieving up to 9x speedup compared to the stock version of the Linux kernel. Beyond the virtual memory management subsystem, we discuss other applications of range locks in parallel software. As a concrete example, we show how range locks can be used to facilitate the design of scalable concurrent data structures, such as skip lists.

关键词： reader-writer locks semaphores scalable synchronization lock-less Linux kernel parallel file systems

来源：评论

学校读者我要写书评

暂无评论

On the Use of Containers in High Performance Computing Environments 13

On the Use of Containers in High Performance Computing Envir...

引用

IEEE 13th International Conference on Cloud Computing (CLOUD)

作者： Abraham, Subil Paul, Arnab K. Khan, Redwan Ibne Seraj Butt, Ali R. Virginia Tech Blacksburg VA 24061 USA

ISBN: (纸本)9781728187808

The lightweight nature, application portability, and deployment flexibility of containers is driving their widespread adoption in cloud solutions. Data analysis and deep learning (DL)/machine learning (ML) applications have especially benefited from containerization. As such data analysis is adopted in high performance computing (HPC), the need for container support in HPC has become paramount. However, containers face crucial performance and I/O challenges in HPC. One obstacle is that while there have been HPC containers, such solutions have not been thoroughly investigated, especially from the aspect of their impact on the crucial HPC I/O throughput. To this end, this paper provides a first-of-its-kind empirical analysis of state-of-the-art representative container solutions (Docker, Podman, Singularity, and Charliecloud) in HPC environments. We also explore how containers interact with an HPC parallel file system like Lustre. We present the design of an analysis framework that is deployed on all nodes in an HPC environment, and captures CPU, memory, network, and file I/O statistics from the nodes and the storage system. We are able to garner key insights from our analysis, e.g., Charliecloud outperforms other container solutions in terms of container start-up time, while Singularity and Charliecloud are equivalent in I/O throughput. But this comes at a cost, as Charliecloud invokes the most metadata and I/O operations on the underlying Lustre file system. By identifying such trade-offs and optimization opportunities, we can enhance HPC containers performance and the ML/DL applications that increasingly rely on them.

关键词： Container Performance High Performance Computing parallel file systems HPC Storage and I/O

来源：评论

学校读者我要写书评

暂无评论

On the Use of Containers in High Performance Computing

On the Use of Containers in High Performance Computing

引用

作者： Abraham, Subil Virginia Tech | University

The lightweight, portable, and flexible nature of containers is driving their widespread adoption in cloud solutions. Data analysis and deep learning applications have especially benefited from containerized solutions. As such data analysis is also being utilized in the high performance computing (HPC) domain, the need for container support in HPC has become paramount. However, container adoption in HPC face crucial performance and I/O challenges. One obstacle is that while there have been container solutions for HPC, such solutions have not been thoroughly investigated, especially from the aspect of their impact on the crucial I/O throughput needs of HPC. To this end, this paper provides a first-of-its-kind empirical analysis of state-of-the-art representative container solutions (Docker, Podman, Singularity, and Charliecloud) in HPC environments, especially how containers interact with the HPC storage systems. We present the design of an analysis framework that is deployed on all nodes in an HPC environment, and captures aspects such as CPU, memory, network, and file I/O statistics from the nodes and the storage system. We are able to garner key insights from our analysis, e.g., Charliecloud outperforms other container solutions in terms of container start-up time, while Singularity and Charliecloud are equivalent in I/O throughput. But this comes at a cost, as Charliecloud invokes the most metadata and I/O operations on the underlying Lustre file system. By identifying such optimization opportunities, we can enhance performance of containers atop HPC and help the aforementioned applications.

关键词： Container Performance High Performance Computing parallel file systems HPC Storage and I/O Thesis

来源：评论

学校读者我要写书评

暂无评论

Optimizing I/O Performance of HPC Applications with Autotuning

引用

ACM TRANSACTIONS ON parallel COMPUTING 2019年第4期5卷 15.1-15.27页

作者： Behzad, Babak Byna, Surendra Prabhat Snir, Marc Univ Illinois Champaign IL 61801 USA Lawrence Berkeley Natl Lab 1 Cyclotron RdMail Stop 50B-3238 Berkeley CA 94720 USA Lawrence Berkeley Natl Lab 1 Cyclotron RdMail Stop 59R4010A Berkeley CA 94720 USA Argonne Natl Lab Argonne IL 60439 USA Univ Illinois Dept Comp Sci 201 N Goodwin Ave Urbana IL 61801 USA 66 E 40th Ave San Mateo CA 94403 USA

parallel Input output is an essential component of modern high-performance computing (HPC). Obtaining good I/O performance for a broad range of applications on diverse HPC platforms is a major challenge, in part, because of complex inter dependencies between I/O middleware and hardware. The parallel file system and I/O middleware layers all offer optimization parameters that can, in theory, result in better I/O performance. Unfortunately, the right combination of parameters is highly dependent on the application, HPC platform, problem size, and concurrency. Scientific application developers do not have the time or expertise to take on the substantial burden of identifying good parameters for each problem configuration. They resort to using system defaults, a choice that frequently results in poor I/O performance. We expect this problem to be compounded on exascale-class machines, which will likely have a deeper software stack with hierarchically arranged hardware resources. We present as a solution to this problem an autotuning system for optimizing I/O performance, I/O performance modeling, I/O tuning, and I/O patterns. We demonstrate the value of this framework across several HPC platforms and applications at scale.

关键词： HPC storage I/O autotuning performance optimization parallel file systems

来源：评论

学校读者我要写书评

暂无评论

Toward Comprehensive and Accurate Simulation Performance Prediction of parallel file systems

Toward Comprehensive and Accurate Simulation Performance Pre...

引用

42nd Annual IEEE/IFIP International Conference on Dependable systems and Networks (DSN)

作者： Erazo, Miguel A. Li, Ting Liu, Jason Eidenbenz, Stephan Florida Int Univ Sch Comp & Informat Sci Miami FL 33199 USA Los Alamos Natl Lab Informat Sci CCS 3 Los Alamos NM USA

ISBN: (纸本)9781467316255

We present the design and implementation of fileSim, a simulation framework with detailed models of parallel file systems, capable of reproducing the complex I/O behavior at scale. fileSim aims to support comprehensive and accurate end-to-end I/O performance prediction and evaluation of exascale high-end computing systems. To this end, fileSim provides several key features, including detailed, pluggable models of contemporary parallel file systems, the support of trace-driven simulation, and the capability of running large-scale I/O systems using parallel and distributed simulation. We conducted extensive validation and performance studies, through which we show that the simulator is capable of reproducing important I/O system behaviors comparable to those measured from the real systems. We demonstrate the capabilities of fileSim as a tool for exploring the parameter space and design alternatives of large-scale parallel file systems.

关键词： parallel file systems PanFS simulation and modeling parallel simulation

来源：评论

学校读者我要写书评

暂无评论

End-to-end I/O Portfolio for the Summit Supercomputing Ecosystem 19

End-to-end I/O Portfolio for the Summit Supercomputing Ecosy...

引用

International Conference for High Performance Computing, Networking, Storage and Analysis (SC)

作者： Oral, Sarp Vazhkudai, Sudharshan S. Wang, Feiyi Zimmer, Christopher Brumgard, Christopher Hanley, Jesse Markomanolis, George Miller, Ross Leverman, Dustin Atchley, Scott Larrea, Veronica Vergara Oak Ridge Natl Lab Oak Ridge TN 37830 USA

ISBN: (数字)9781450362290

ISBN: (纸本)9781450362290

The I/O subsystem for the Summit supercomputer, No. 1 on the Top500 list, and its ecosystem of analysis platforms is composed of two distinct layers, namely the in-system layer and the center-wide parallel file system layer (PFS), Spider 3. The in-system layer uses node-local SSDs and provides 26.7 TB/s for reads, 9.7 TB/s for writes, and 4.6 billion IOPS to Summit. The Spider 3 PFS layer uses IBM's Spectrum Scale (TM) and provides 2.5 TB/s and 2.6 million IOPS to Summit and other systems. While deploying them as two distinct layers was operationally efficient, it also presented usability challenges in terms of multiple mount points and lack of transparency in data movement. To address these challenges, we have developed novel end-to-end I/O solutions for the concerted use of the two storage layers. We present the I/O subsystem architecture, the end-to-end I/O solution space, their design considerations and our deployment experience.

关键词： High performance computing parallel file systems parallel IO performance bechmarking

来源：评论

学校读者我要写书评

暂无评论

Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC systems with Eley

引用

FUTURE GENERATION COMPUTER systems-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年 86卷 308-318页

作者： Yildiz, Orcun Zhou, Amelie Chi Ibrahim, Shadi Univ Rennes INRIA CNRS IRISA Rennes France Argonne Natl Lab Lemont IL 60439 USA Shenzhen Univ Shenzhen Peoples R China IMT Atlantique INRIA LS2N Nantes France

Burst Buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications. (C) 2018 Published by Elsevier B.V.

关键词： HPC MapReduce Big data parallel file systems Burst buffers Interference Prefetch

来源：评论

学校读者我要写书评

暂无评论

pWebDAV: a multi-tier storage system 26

pWebDAV: a multi-tier storage system

引用

26th Euromicro International Conference on parallel, Distributed, and Network-Based Processing (PDP)

作者： Filippidis, Christos Cotronis, Yiannis Univ Athens Dept Informat & Telecommun Athens Greece

ISBN: (纸本)9781538649756

Experiments using the Large Hadron Collider (LHC) currently generate tens of petabytes of reduced data per year, observational and simulation data in the climate domain is expected to reach eXabytes by 2021, and light source experiments are expected to generate hundreds of terabytes per day. At such extreme scale, the substantial amount of concurrency can cause critical contention issue of the I/O system. This study introduces pWebDAV as a heterogeneous, multi-tier storage system. pWebDAV proposes a dynamically coordinated I/O architecture offering overall data flow solutions (remote-local access). The fundamental idea is to implement, for each data transfer, I/O policies on the fly. pWebDAV controls all I/O nodes, participating in the data transfer, directly regardless of the tier. pWebDAV approach can fully utilize the provided I/O & network resources and is able to minimize disk and network contention. The focus in this study is the Metadata node scalability performance.

关键词： Multi-tier storage system parallel file systems Data management Distributed systems GridFTP

来源：评论

学校读者我要写书评

暂无评论

On the Performance of Spark on HPC systems: Towards a Complete Picture 4th

On the Performance of Spark on HPC Systems: Towards a Comple...

引用

4th Asian Conference on Supercomputing Frontiers (SCFA)

作者： Yildiz, Orcun Ibrahim, Shadi Univ Rennes CNRS IRISA INRIA Rennes France IMT Atlantique INRIA LS2N Nantes France

ISBN: (纸本)9783319699530;9783319699523

Big Data analytics frameworks (e.g., Apache Hadoop and Apache Spark) have been increasingly used by many companies and research labs to facilitate large-scale data analysis. However, with the growing needs of users and size of data, commodity-based infrastructure will strain under the heavy weight of Big Data. On the other hand, HPC systems offer a rich set of opportunities for Big Data processing. As first steps toward Big Data processing on HPC systems, several research efforts have been devoted to understanding the performance of Big Data applications on these systems. Yet the HPC specific performance considerations have not been fully investigated. In this work, we conduct an experimental campaign to provide a clearer understanding of the performance of Spark, the de facto in-memory data processing framework, on HPC systems. We ran Spark using representative Big Data workloads on Grid'5000 testbed to evaluate how the latency, contention and file system's configuration can influence the application performance. We discuss the implications of our findings and draw attention to new ways (e.g., burst buffers) to improve the performance of Spark on HPC systems.

关键词： HPC MapReduce Spark parallel file systems Contention

来源：评论

学校读者我要写书评

暂无评论

Collective I/O Performance on the Santos Dumont Supercomputer 26

Collective I/O Performance on the Santos Dumont Supercompute...

引用

26th Euromicro International Conference on parallel, Distributed, and Network-Based Processing (PDP)

作者： Carneiro, Andre Ramos Bez, Jean Luca Boito, Francieli Zanon Fagundes, Bruno Alves Osthoff, Carla Navaux, Philippe O. A. Natl Lab Sci Comp LNCC Petropolis Brazil Fed Univ Rio Grande do Sul UFRGS Inst Informat Porto Alegre RS Brazil INRIA Grenoble Informat Lab LIG Grenoble France

ISBN: (纸本)9781538649756

The historical gap between processing and data access speeds causes many applications to spend a large portion of their execution on I/O operations. From the point of view of a large-scale, expensive, supercomputer, it is important to ensure applications achieve the best I/O performance to promote an efficient usage of the machine. In this paper, we evaluate the I/O infrastructure of the Santos Dumont supercomputer, the largest one from Latin America. More specifically, we investigate the performance of collective I/O operations. By conducting an analysis of a scientific application that uses the machine, we identify large performance differences between the available MPI implementations. We then further study the observed phenomenon using the BT-IO and IOR benchmarks, in addition to a custom microbenchmark. We conclude that the customized MPI implementation by Bull (used by more than 20% of the jobs) presents the worst performance for small collective write operations. Our results are being used to help the Santos Dumont users to achieve the best performance for their applications. Additionally, by investigating the observed phenomenon, we provide information to help improve future MPI-IO collective write implementations.

关键词： Collective I/O High Performance Computing MPI MPI-IO parallel file systems parallel I/O

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：