检索结果-内蒙古大学图书馆

High Performance computing, Networking, Storage and Analysis (SCC), SC Companion:

作者： Robert L. Grossman Matthew Greenway Allison P. Heath Ray Powell Rafael D. Suarez Walt Wells Kevin White Malcolm Atkinson Iraklis Klampanos Heidi L. Alvarez Christine Harvey Joe J. Mambretti University of Chicago Chicago IL USA University of Edinburgh Edinburgh UK Florida International University Miami FL USA Richard Stockton College of New Jersey Galloway NJ USA Northwestern University Evanston IL USA

ISBN: (纸本)9781467362184

In this paper we describe the design, and implementation of the Open Science data Cloud, or OSDC. The goal of the OSDC is to provide petabyte-scale data cloud infrastructure and related services for scientists working with large quantities of data. Currently, the OSDC consists of more than 2000 cores and 2 PB of storage distributed across four data centers connected by 10G networks. We discuss some of the lessons learned during the past three years of operation and describe the software stacks used in the OSDC. We also describe some of the research projects in biology, the earth sciences, and social sciences enabled by the OSDC.

关键词： data intensive computing Cloud computing Science clouds Open data

来源：评论

学校读者我要写书评

暂无评论

OpenFlow Enabled Hadoop over Local and Wide Area Clusters

OpenFlow Enabled Hadoop over Local and Wide Area Clusters

引用

High Performance computing, Networking, Storage and Analysis (SCC), SC Companion:

作者： Sandhya Narayan Stuart Bailey Anand Daga Matthew Greenway Robert Grossman Allison Heath Ray Powell InfoBlox Inc. Santa Clara CA USA Computation Institute University of Chicago USA Department of Computer Science University of Houston USA

ISBN: (纸本)9781467362184

Hadoop has emerged as an important platform for data intensive computing. The shuffle and sort phases of a MapReduce computation often saturate top of the rack switches, as well as switches that aggregate multiple racks. In addition, MapReduce computations often have "hot spots" in which the computation is lengthened due to inadequate bandwidth to some of the nodes. In principle, OpenFlow enables an application to adjust the network topology as required by the computation, providing additional network bandwidth to those resources requiring it. We describe Hadoop-OFE, which is an OpenFlow enabled version of Hadoop that dynamically modifies the network topology in order to improve the performance of Hadoop.

关键词： Hadoop OpenFlow OpenFlow over Ethernet data intensive computing MapReduce Network topology teeth bar racking arithmetic Switches network bandwidth Shuffle

来源：评论

学校读者我要写书评

暂无评论

Task assignment policies in distributed server systems: A survey

引用

JOURNAL OF NETWORK AND COMPUTER APPLICATIONS 2011年第4期34卷 1123-1130页

作者： Semchedine, Fouzi Bouallouche-Medjkoune, Louiza Aissani, Djamil Univ Bejaia LAMOS Lab Modeling & Optimizat Syst Bejaia 06000 Algeria Univ Bejaia Doctoral Sch Comp Sci Networking & Distributed Sy Bejaia 06000 Algeria

data intensive computing, in the Web environment, motivates the distributed designs of Web server systems (Web clusters) because of their scalability and cost-effectiveness instead of one Web server with high performance. The task assignment policy, in such systems, focuses on the manner of assigning the tasks that reach these systems (e.g. the case of intensive requests that reach the distributed Web server systems) in order to minimize the response time and thus, improve the performance. These tasks, generally, follow the "heavy-tailed" distribution which has the property that there is a tiny fraction (about 3%) of large tasks that makes half (50%) of the total load. Several policies were proposed in the literature to deal with the nature of this Web traffic. This paper presents a state-of-art of the existing task assignment policies. We classify these policies in two classes: policies which assume that the task size is known a priori, and policies which assume that the task size is not known a priori (like TAGS, TAPTF and TAPTF-WC). The first class of policies regroups policies which consider no knowledge of load information at the servers when assigning the incoming tasks, known as static policies (like Random, Round Robin, etc.) and, policies, known as dynamic policies (like Central Queue Policy, Least Loaded First "LLF", etc.) which use some load information (e.g. the processing capacity, the queue load, etc.) to process. (C) 2011 Elsevier Ltd. All rights reserved.

关键词： data intensive computing Task assignment Distributed server systems Heavy-tailed distribution Load balancing Load sharing Performance

来源：评论

学校读者我要写书评

暂无评论

Global-aware and multi-order context-based prefetching for high-performance processors

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE computing APPLICATIONS 2011年第4期25卷 355-370页

作者： Chen, Yong Zhu, Huaiyu Roth, Philip C. Jin, Hui Sun, Xian-He Texas Tech Univ Dept Comp Sci Lubbock TX 79409 USA Univ Illinois Dept Comp Sci Urbana IL USA Univ Illinois Dept Elect & Comp Engn Urbana IL USA Oak Ridge Natl Lab Comp Sci & Math Div Future Technol Grp Oak Ridge TN USA IIT Dept Comp Sci Chicago IL 60616 USA IIT Scalable Comp Software Lab Chicago IL 60616 USA

data prefetching is widely used in high-end computing systems to accelerate data accesses and to bridge the increasing performance gap between processor and memory. Context-based prefetching has become a primary focus of study in recent years due to its general applicability. However, current context-based prefetchers only adopt the context analysis of a single order, which suffers from low prefetching coverage and thus limits the overall prefetching effectiveness. Also, existing approaches usually consider the context of the address stream from a single instruction but not the context of the address stream from all instructions, which further limits the context-based prefetching effectiveness. In this study, we propose a new context-based prefetcher called the Global-aware and Multi-order Context-based (GMC) prefetcher. The GMC prefetcher uses multi-order, local and global context analysis to increase prefetching coverage while maintaining prefetching accuracy. In extensive simulation testing of the SPEC-CPU2006 benchmarks with an enhanced CMP$im simulator, the proposed GMC prefetcher was shown to outperform existing prefetchers and to reduce the data-access latency effectively. The average Instructions Per Cycle (IPC) improvement of SPEC CINT2006 and CFP2006 benchmarks with GMC prefetching was over 55% and 44% respectively.

关键词： prefetching context-based prefetching prefetching accuracy prefetching coverage processor architectures memory hierarchy data access delay prefetch degree prefetch priority cache pollution bandwidth contention SPEC-CPU2006 CMP$im simulator PIN high-end computing data intensive computing

来源：评论

学校读者我要写书评

暂无评论

A Science Driven Production Cyberinfrastructure-the Open Science Grid

引用

JOURNAL OF GRID computing 2011年第2期9卷 201-218页

作者： Altunay, Mine Avery, Paul Blackburn, Kent Bockelman, Brian Ernst, Michael Fraser, Dan Quick, Robert Gardner, Robert Goasguen, Sebastien Levshina, Tanya Livny, Miron McGee, John Olson, Doug Pordes, Ruth Potekhin, Maxim Rana, Abhishek Roy, Alain Sehgal, Chander Sfiligoi, Igor Wuerthwein, Frank Fermilab Natl Accelerator Lab Batavia IL 60510 USA Univ Florida Gainesville FL USA CALTECH Pasadena CA 91125 USA Brookhaven Natl Lab Upton NY 11973 USA Argonne Natl Lab Downers Grove IL USA Indiana Univ Bloomington IN USA Univ Chicago Chicago IL 60637 USA Clemson Univ Clemson SC USA Univ Wisconsin Madison WI USA RENCI Chapel Hill NC USA Lawrence Berkeley Natl Lab Berkeley CA USA Univ Calif San Diego San Diego CA 92103 USA Univ Nebraska Lincoln NE USA

This article describes the Open Science Grid, a large distributed computational infrastructure in the United States which supports many different high-throughput scientific applications, and partners (federates) with other infrastructures nationally and internationally to form multi-domain integrated distributed systems for science. The Open Science Grid consortium not only provides services and software to an increasingly diverse set of scientific communities, but also fosters a collaborative team of practitioners and researchers who use, support and advance the state of the art in large-scale distributed computing. The scale of the infrastructure can be expressed by the daily throughput of around seven hundred thousand jobs, just under a million hours of computing, a million file transfers, and half a petabyte of data movement. In this paper we introduce and reflect on some of the OSG capabilities, usage and activities.

关键词： Scientific computing Distributed computing Grids High throughput computing data intensive computing

来源：评论

学校读者我要写书评

暂无评论

Multiple query scheduling for distributed semantic caches

引用

JOURNAL OF PARALLEL AND DISTRIBUTED computing 2010年第5期70卷 598-611页

作者： Nam, Beomseok Shin, Minho Andrade, Henrique Sussman, Alan Oracle Redwood Shores CA 94065 USA Dartmouth Coll ISTS Hanover NH 03755 USA IBM Corp TJ Watson Res Ctr Hawthorne NY 10532 USA Univ Maryland Dept Comp Sci College Pk MD 20742 USA

In distributed query processing systems, load balancing plays an important role in maximizing system throughput. When queries can leverage cached intermediate results, improving the cache hit ratio becomes as important as load balancing in query scheduling, especially when dealing with computationally expensive queries. The scheduling policies must be designed to take into consideration the dynamic contents of the distributed caching infrastructure. In this paper, we propose and discuss several distributed query scheduling policies that directly consider the available cache contents by employing distributed multidimensional indexing structures and an exponential moving average approach to predicting cache contents. These approaches are shown to produce better query plans and faster query response times than traditional scheduling policies that do not predict dynamic contents in distributed caches. We experimentally demonstrate the utility of the scheduling policies using MQO, which is a distributed, Grid-enabled, multiple query processing middleware system we developed to optimize query processing for data analysis and visualization applications. (C) 2010 Elsevier Inc. All rights reserved.

关键词： Multiple query optimization Distributed query scheduling data intensive computing

来源：评论

学校读者我要写书评

暂无评论

TRANSFORMER: A NEW PARADIGM FOR BUILDING data-PARALLEL PROGRAMMING MODELS

引用

IEEE MICRO 2010年第4期30卷 55-64页

作者： Wang, Peng Meng, Dan Han, Jizhong Zhan, Jianfeng Tu, Bibo Shi, Xiaofeng Wan, Le Chinese Acad Sci Inst Comp Technol Beijing 100190 Peoples R China

Cloud computing drives the design and development of diverse programming models for massive data processing. the transformer programming framework aims to facilitate the building of diverse data-parallel programming models. Transformer has two layers: a common runtime system and a model-specific system. using transformer, the authors show how to implement three programming models: dryad-like data flow, MapReduce, and all-pairs.

关键词： actor model All-Pairs cloud computing Computational modeling data flow data intensive computing data models Libraries Load modeling MapReduce Programming programming model Receivers Runtime

来源：评论

学校读者我要写书评

暂无评论

Design and Implementation of Metadata System in PetaShare

Design and Implementation of Metadata System in PetaShare

引用

21st International Conference on Scientific and Statistical database Management

作者： Wang, Xinqi Kosar, Tevfik Louisiana State Univ Dept Comp Sci Baton Rouge LA 70803 USA Louisiana State Univ Ctr Computat & Technol Baton Rouge LA 70803 USA

ISBN: (纸本)9783642022784

As the size of scientific and commercial datasets grows, it becomes imperative that an expressive metadata framework to be developed to facilitate access to the semantics of the data. However, the drive to enhance expressiveness and maximize performance often turns out to be contradictory. Hence, the design of metadata framework needs to take into consideration many factors in order to achieve balance between expressive power and performance. In this paper, we discuss the metadata framework developed for PetaShare and explain the design and implementation decisions we made in the process.

关键词： Metadata data intensive computing Ontology Protege iRODS PetaShare

来源：评论

学校读者我要写书评

暂无评论

Multicore-Based High Performance Image Analysis for Batch Processing in Drug Discovery 18

Multicore-Based High Performance Image Analysis for Batch Pr...

引用

IMACS World Congress/Modelling and Simulation Society-of-Australia-and-New-Zealand (MSSANZ)/18th MODSIM09 Biennial Conference on Modelling and Simulation

作者： Wang, D. D. Bourke, D. Domanski, L. Vallotton, P. CSIRO Math & Informat Sci Biotech Imaging N Ryde NSW 1670 Australia

ISBN: (纸本)9780975840078

Automated image analysis allows drug companies to use High Content Screening (HCS) to measure subtle but important changes in fluorescently labeled cells, both rapidly and accurately. Researchers using HCS imaging systems generate thousands of images from which they measure millions of parameters. Some images can be very "dense", with hundreds of cells and complicated cell morphology. It may take several hours or even days to process the images generated from a single experiment which may be unacceptable for the workflows in laboratories. The large image datasets and fast turnaround requirement has made the efficient batch processing of the massive amount of images a challenging task. With the enormous progress in computing power, computers with multi-core CPUs are now becoming standard. The multi-core shift presents unprecedented opportunities for researchers to deal with large datasets with acceptable computing performance. Unfortunately, having a dual core CPU does not speedup one's application automatically. In fact, most applications use just a single core and see no speed improvements when run on a multi-core machine. It is still a challenge to develop parallel algorithms that address the issues of task concurrency and data parallelism, and that actually take advantage of those multiple processors. In this paper, some efforts have been made to develop a multi-core based high performance solution to speedup the batch processing for micro-well-plate-based HCS. The proposed solution employs an automatic parallelization engine which automatically dispatches the batch processing tasks. The engine automatically detects the number of processors available on the computer on which our High Content Analysis (HCA) software runs, and then creates equal number of work threads to process images in parallel. Therefore, the solution can automatically scale to additional cores and future multi-core processors. Loop parallelization mechanism has been implemented to assign diffe

关键词： High Performance computing data intensive computing Parallel Processing Image Analysis Batch processing High Content Screening

来源：评论

学校读者我要写书评

暂无评论

A distributed multi-storage I/O system for data intensive scientific computing

引用

PARALLEL computing 2003年第11-12期29卷 1623-1643页

作者： Shen, XH Choudhary, A Northwestern Univ Dept Elect & Comp Engn Ctr Parallel & Distributed Comp Evanston IL 60208 USA

More and more parallel applications are running in a distributed environment to take advantage of easily available and inexpensive commodity resources. For data intensive applications, employing multiple distributed storage resources has many advantages. In this paper, we present a Multi-Storage I/O System (MS-I/O) that cannot only effectively manage various distributed storage resources in the system, but also provide novel high performance storage access schemes. MS-I/O employs many state-of-the-art I/O optimizations such as collective I/O, asynchronous I/O etc. and a number of new techniques such as data location, data replication, subfile, superfile and data access history. In addition, many MS-I/O optimization schemes can work simultaneously within a single data access session, greatly improving the performance. Although I/O optimization techniques can help improve performance, it also complicates I/O system. In addition, most optimization techniques have their limitations. Therefore, selecting accurate optimization policies requires expert knowledge which is not suitable for end users who may have little knowledge of I/O techniques. So the task of I/O optimization decision should be left to the I/O system itself, that is, automatic from user's point of view. We present a User Access Pattern data structure which is associated with each dataset that can help MS-I/O easily make accurate I/O optimization decisions. (C) 2003 Elsevier B.V. All rights reserved.

关键词： multi-storage I/O system access pattern data intensive computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：