检索结果-内蒙古大学图书馆

Gateway: New high performance computing facility for EFDA task force on integrated Tokamak modelling

FUSION ENGINEERING AND DESIGN 2010年第3-4期85卷 410-414页

作者： Guillerminet, B. Iannone, F. Imbeaux, F. Manduchi, G. Maslennikov, A. Pais, V. Strand, P. CR ENEA Frascati Assoc EURATOM ENEA Fus I-00044 Frascati Italy CEA Cadarache CEA DSM DRFC Assoc EURATOM CEA F-13108 St Paul Les Durance France Consorzio RFX Assoc EURATOM ENEA Fus I-35127 Padua Italy CASPUR Interuniv Consortium Applicat Super Comp R I-00185 Rome Italy Assoc EURATOM MEdC Laser Dept Bucharest 077125 Romania Chalmers Dept Radio & Space Sci SE-41296 Gothenburg Sweden

The EFDA task force on Integrated Tokamak Modelling (ITM-TF) is providing the EU fusion community with a complete and flexible suite of reliable software tools and codes able to simulate the next ITER and DEMO plasma discharges. EFDA has launched the Gateway project as the first computing facility to be jointly used by EU fusion associations. The Gateway has been designed to allow the ITM-TF members to work together on a common platform and share their codes, developments tools and technologies as well as to make able the inter-operation with tera-scale supercomputer facilities. Technically the Gateway is a rather small high computing facility in operation since 2008, with 1 Teraflops of theoretical peak and 100 Terabytes shared storage area for experimental and simulation data. This paper describes the information technologies involved in the Gateway facility, particularly as regards the low latency interconnect network for multi-core platform as well as fast I/O solutions implemented with storage over Infiniband and high performance parallel file systems. The solutions adopted on the Gateway facility had undergone to specific benchmarks showing excellent performances in typical fusion data handling issues. Finally the interoperability between Gateway and ENEA CRESCO supercomputer will be described showing the access solutions that allow the exploitation of tera-scale supercomputer facilities. (C) 2009 Elsevier B.V. All rights reserved.

关键词： ITM-task force High performance computing parallel file system

来源：评论

学校读者我要写书评

暂无评论

HPC@SCALE: A Hands-on Approach for Training Next-Gen HPC Software Architects 28

HPC@SCALE: A Hands-on Approach for Training Next-Gen HPC Sof...

引用

IEEE 28th International Conference on High Performance Computing, Data and Analytics (HiPC)

作者： Islam, Tanzima Z. Phelps, Chase Texas State Univ Comp Sci San Marcos TX 78666 USA

ISBN: (纸本)9781665410373

High Performance Computing (HPC) systems enable multi-scale simulations to gain meaningful insights into otherwise experimentally intractable phenomena such as climate change and destabilizing drug-protein interactions to cure cancer. High levels of parallelism and heterogeneous architectures in these systems offer unprecedented computational capability at the cost of complexity and dynamism. Recent efforts in workforce development have focused on preparing students with the background to write a program for these complex heterogeneous platforms successfully In However, computing is only one of the three tasks an HPC application performs;the other two are communication and I/O. Since network bandwidth is not scaling proportionately with computational capabilities, moving the large volume of data generated by these applications through the network slows down scientific progress. A high-level data-driven analysis shows that most existing curricula do not prepare students to consider design choices to scale parallel I/O, which is a crucial building component of an end-to-end system. At its core, the problem of scaling data-intensive applications is common in both high-performance, high-throughput, and Cloud computing environments, so any training in that regard will have a broad impact. To fill this gap, we have designed a new course called HPCA@SCALE to train students at Texas State University in building scalable end-to-end system software focusing on minimizing parallel I/O.

关键词： HPC parallel I/O curriculum development hands-on checkpointing system parallel file system

来源：评论

学校读者我要写书评

暂无评论

Caching Support for CHFS Node-local Persistent Memory file system 36

Caching Support for CHFS Node-local Persistent Memory File S...

引用

36th IEEE International parallel and Distributed Processing Symposium (IEEE IPDPS)

作者： Tatebe, Osamu Ohtsuji, Hiroki Univ Tsukuba Tsukuba Ibaraki Japan Fujitsu Ltd Kawasaki Kanagawa Japan

ISBN: (纸本)9781665497473

In this study, we designed a caching parallel file system for node-local persistent memory. The caching parallel file system improves access performance to the backend parallel file system in a way transparent to users. Although existing caching parallel file systems succeed in improving bandwidth, the performance of metadata is limited by that of the backend parallel file system. Based on the CHFS ad hoc parallel and distributed file system, our system maintains the high metadata performance of CHFS by relaxing the consistency with the backend system in easy-to-understand semantics. Upon implementation, our system's performance was evaluated using the 10500 benchmark, and was found to exhibit almost identical performance to that of the original CHFS. This shows that it is possible to realize a caching parallel file system without sacrificing metadata performance.

关键词： caching file system parallel file system persistent memory

来源：评论

学校读者我要写书评

暂无评论

Accelerating I/O in Distributed Data Processing systems with Apache Arrow CHFS 25

Accelerating I/O in Distributed Data Processing Systems with...

引用

25th IEEE International Conference on Cluster Computing (CLUSTER)

作者： Koyama, Sohei Hiraga, Kohei Tatebe, Osamu Univ Tsukuba Tsukuba Ibaraki Japan

ISBN: (纸本)9798350370621

In recent years, distributed data-processing frameworks have become popular for processing big data. However, in an HPC, where the computation and storage nodes are separated, the bandwidth between the computation and storage components is small, causing a reduction in data processing throughput. Therefore, in this paper, data were stored on the computation node to solve the data processing throughput degradation. We propose an I/O acceleration method that integrates Apache Arrow and CHFS. It leverages non-volatile memory, a state-of-the-art storage device, via CHFS and leverages CHFS from a distributed data processing framework via Apache Arrow's abstract file system API. The evaluation results showed that the system achieved up to 11.60 times higher bandwidth than when reading data from the parallel file system Lustre. This study also compared with Apache Arrow with BeeOND and UnifyFS, other ad hoc filesystems. The proposed Apache Arrow CHFS showed up to 1.67x/1.23x better write performance. The implementation is published at https://***/tsukuba-hpcs/arrow-chfs

关键词： parallel file system distributed data processing HPC

来源：评论

学校读者我要写书评

暂无评论

ClusterLog: Clustering Logs for Effective Log-based Anomaly Detection 12

ClusterLog: Clustering Logs for Effective Log-based Anomaly ...

引用

IEEE/ACM 12th Workshop on Fault Tolerance for HPC at eXtreme Scale (FTXS)

作者： Egersdoerfer, Chris Zhang, Di Dai, Dong Univ North Carolina Charlotte Comp Sci Dept Charlotte NC 28223 USA

ISBN: (纸本)9781665488471

With the increasing prevalence of scalable file systems in the context of High Performance Computing (HPC), the importance of accurate anomaly detection on runtime logs is increasing. But as it currently stands, many state-of-the-art methods for log-based anomaly detection, such as DeepLog, have encountered numerous challenges when applied to logs from many parallel file systems (PF-Ses), often due to their irregularity and ambiguity in time-based log sequences. To circumvent these problems, this study proposes ClusterLog, a log pre-processing method that clusters the temporal sequence of log keys based on their semantic similarity. By grouping semantically and sentimentally similar logs, this approach aims to represent log sequences with the smallest amount of unique log keys, intending to improve the ability for a downstream sequence-based model to effectively learn the log patterns. The preliminary results of ClusterLog indicate not only its effectiveness in reducing the granularity of log sequences without the loss of important sequence information but also its generalizability to different file systems' logs.

关键词： anomaly detection log analysis parallel file system

来源：评论

学校读者我要写书评

暂无评论

MLBS: Transparent Data Caching in Hierarchical Storage for Out-of-Core HPC Applications 26

MLBS: Transparent Data Caching in Hierarchical Storage for O...

引用

26th International Conference on High Performance Computing, Data and Analytics (HiPCW)

作者： Alturkestani, Tariq Tonellot, Thierry Ltaief, Hatem Abdelkhalak, Rached Etienne, Vincent Keyes, David King Abdullah Univ Sci & Technol Extreme Comp Res Ctr Thuwal Saudi Arabia Explorat & Petr Engn Adv Res Ctr Dhahran Saudi Arabia

ISBN: (纸本)9781728145358

Out-of-core simulation systems produce and/or consume a massive amount of data that cannot fit on a single compute node memory and that usually needs to be read and/or written back and forth during computation. I/O data movement may thus represent a bottleneck in large-scale simulations. To increase I/O bandwidth, high-end supercomputers are equipped with hierarchical storage subsystems such as node-local and remote-shared NVMe and SSD-based Burst Buffers. Advanced caching systems have recently been developed to efficiently utilize the multi-layered nature of the new storage hierarchy. Utilization of software components results in more efficient data accesses, at the cost of reduced computation kernel performance and limited numbers of simultaneous applications that can utilize the additional storage layers. We introduce MultiLayered Buffer Storage (MLBS), a data object container that provides novel methods for caching and prefetching data in out-of-core scientific applications to perform asynchronously expensive I/O operations on systems equipped with hierarchical storage. The main idea consists in decoupling I/O operations from computational phases using dedicated hardware resources to perform expensive context switches. MLBS monitors I/O traffic in each storage layer allowing fair utilization of shared resources while controlling the impact on kernels' performance. By continually prefetching up and down across all hardware layers of the memory/storage subsystems, MLBS transforms the original I/O-bound behavior of evaluated applications and shifts it closer to a memory-bound regime. Our evaluation on a Cray XC40 system for a representative I/O-bound application, seismic inversion, shows that MLBS outperforms state-of-the-art filesystems, i.e., Lustre, Data Elevator and DataWarp by 6.06X, 2.23X, and 1.90X, respectively.

关键词： Multilayered Buffer system Asynchronous DO Caching and Prefetching Lustre Burst Buffer DataWarp Data Elevator parallel file system

来源：评论

学校读者我要写书评

暂无评论

Does Varying BeeGFS Configuration Affect the I/O Performance of HPC Workloads? 25

Does Varying BeeGFS Configuration Affect the I/O Performance...

引用

25th IEEE International Conference on Cluster Computing (CLUSTER)

作者： Borkar, Arnav Tony, Joel Vamsi, Hari K. N. Barman, Tushar Bhisikar, Yash Sreenath, T. M. Paul, Arnab K. BITS Pilani KK Birla Goa Campus Pilani Rajasthan India

ISBN: (纸本)9798350370621

The increasing gap between processor and disk performance leads to high performance computing (HPC) applications facing I/O bottlenecks. This makes parallel file systems one of the most important components in an HPC cluster. This work analyzes the I/O performance of different workloads for various BeeGFS configurations. Our analysis shows that the default allocation strategies and striping configuration leads to an imbalanced distribution of workload data, thereby negatively affecting the I/O performance.

关键词： BeeGFS High Performance Computing parallel file system

来源：评论

学校读者我要写书评

暂无评论

file schemes for fault tolerant support in heterogeneous distributed storage systems

File schemes for fault tolerant support in heterogeneous dis...

引用

International Conference on parallel and Distributed Processing Techniques and Applications

作者： García, F Calderón, A Fernández, J Sánchez, LM Univ Carlos III Madrid Dept Comp Sci Comp Architecture Grp Madrid Spain

ISBN: (纸本)1932415262

One important piece of system software for clusters is the parallel file system. Two main problems can be found in current parallel file systems and parallel I/O libraries for clusters. One is that they do not use standard servers, thus it is very difficult to use these systems in heterogeneous environments. Other is that with multiple servers running long time is more probably a fault that blocks clients. There are not a clear solution that integrate, inside the parallel file system itself, the fault tolerance support needed in a flexible way. We propose the use of existing network file servers at the server end of a parallel file system to offer a parallel file system for heterogeneous clusters. By integrating redundant techniques into the parallel file system, is possible to define a fault tolerance support from partition level to file level. This paper describes the design of the fault tolerance support for the next version of the Expand parallel file system. (1) and a preliminar evaluation that has been made in Linux clusters of the first prototype.

关键词： parallel file system NFS data declustering clusters fault-tolerance

来源：评论

学校读者我要写书评

暂无评论

A parallel data storage interface to GridFTP

引用

On the Move Confederated International Conference on CoopIS/DOA/GADA, and ODBASE

作者： Sanchez, Alberto Perez, Maria S. Gueant, Pierre Montes, Jesus Herrero, Pilar Univ Politecn Madrid Fac Informat Madrid Spain

ISBN: (纸本)3540482741

Most of the grid projects are characterized by accessing huge volumes of data. For supporting this feature, different data services have arisen in the "grid" world. One of the most successful initiatives in that field is GridFTP, a high-performance transfer protocol, based on FTP but optimized for wide area networks. Although GridFTP provides reasonably good performance, GridFTP servers keep constituting a bottleneck for data-intensive applications. One of the most important modules of a GridFTP server is the Data Storage Interface (DSI), which specifies how to read and write to the storage system, allowing the server to transform the data. With the aim of improving the performance of the GridFTP server, we have designed a new DSI, based on MAPFS, a parallel file system. This paper describes this new DSI and its evaluation, showing the advantages of dealing data through this optimized GridFTP server.

关键词： data grid GridFTP Data Storage Interface (DSI) parallel file system MAPFS

来源：评论

学校读者我要写书评

暂无评论

Applying Selectively parallel I/O Compression to parallel Storage systems 1

引用

20th European Conference on parallel Computing (Euro-Par)

作者： Filgueira, Rosa Atkinson, Malcolm Tanimura, Yusuke Kojima, Isao Univ Edinburgh Sch Informat Edinburgh EH8 9AB Midlothian Scotland AIST Informat Technol Res Inst Tsukuba Ibaraki Japan

ISBN: (数字)9783319098739

ISBN: (纸本)9783319098739;9783319098722

This paper presents a new I/O technique called Selectively parallel I/O Compression (SPIOC) for providing high-speed storage and access to data in QoS enabled parallel storage systems. SPIOC reduces the time of I/O operations by applying transparent compression between the computing and the storage systems. SPIOC can predict whether to compress or not at runtime, allowing parallel or sequential compression techniques, guaranteeing QoS and allowing partial and full reading by decompressing the minimum part of the file. SPIOC maximises the measured efficiency of data movement by applying run-time customising compression before storing data in the Papio storage system.

关键词： parallel file system Data Intensive Computing Compression algorithms Adaptive systems

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：