Application skeletons: Construction and use in eScience

作     者：Katz, Daniel S. Merzky, Andre Zhang, Zhao Jha, Shantenu 

作者机构：Univ Chicago Computat Inst Chicago IL 60637 USA Argonne Natl Lab Chicago IL USA Rutgers State Univ RADICAL Lab New Brunswick NJ 08903 USA Univ Calif Berkeley AMPLab Berkeley CA 94720 USA 

出 版 物：《FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE》 (下代计算机系统)

年 卷 期：2016年第59卷

页      面：114-124页

核心收录：

学科分类：08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：U.S. Department of Energy under the ASCR [DE-FG02-12ER26115, DE-SC0008617, DE-SC0008651] National Science Foundation Direct For Computer & Info Scie & Enginr Office of Advanced Cyberinfrastructure (OAC) Funding Source: National Science Foundation U.S. Department of Energy (DOE) [DE-SC0008651, DE-SC0008617] Funding Source: U.S. Department of Energy (DOE) 

主　　题：Computational science Data science Application modeling System modeling Performance modeling Parallel and distributed systems 

摘      要：Computer scientists who work on tools and systems to support eScience (a variety of parallel and distributed) applications usually use actual applications to prove that their systems will benefit science and engineering (e.g., improve application performance). Accessing and building the applications and necessary data sets can be difficult because of policy or technical issues, and it can be difficult to modify the characteristics of the applications to understand corner cases in the system design. In this paper, we present the Application Skeleton, a simple yet powerful tool to build synthetic applications that represent real applications, with runtime and I/O close to those of the real applications. This allows computer scientists to focus on the system they are building;they can work with the simpler skeleton applications and be sure that their work will also be applicable to the real applications. In addition, skeleton applications support simple reproducible system experiments since they are represented by a compact set of parameters. Our Application Skeleton tool (available as open source at https://***/applicationskeleton/Skeleton) currently can create easy-to-access, easy-to-build, and easy-to-run bag-of-task, (iterative) map reduce, and (iterative) multistage workflow applications. The tasks can be serial, parallel, or a mix of both. The parameters to represent the tasks can either be discovered through a manual profiling of the applications or through an automated method. We select three representative applications (Montage, BLAST, CyberShake Postprocessing), then describe and generate skeleton applications for each. We show that the skeleton applications have identical (or close) performance to that of the real applications. We then show examples of using skeleton applications to verify system optimizations such as data caching, I/O tuning, and task scheduling, as well as the system resilience mechanism, in some cases modifying the skeleton applica