检索结果-内蒙古大学图书馆

1st IEEE International Conference on Big data computing Service and Applications (BigdataService)

作者： Kulasekaran, Sivakumar Esteva, Maria Trelogan, Jessica Liu, Si Texas Adv Comp Ctr Houston TX 77054 USA Inst Class Archaeol London England Univ Texas Austin Austin TX 78712 USA

ISBN: (纸本)9781479981281

data management entails a continuum of tasks to develop sustainable and reusable collections throughout their lifecycle. Large collections with complex data formats and structures may require what we define as "multitasking data management," involving a combination of manual and automated iterative tasks. When conducted in a desktop computing environment by curators, these tasks can be labor-intensive and disruptive of research. While the process can be made much more efficient within a data-intensive High Performance computing (DIC/HPC) infrastructure, it remains a challenge to implement generalizable services so that automated workflows can be easily performed by non-expert users. This paper introduces a framework for automating data management activities as data-intensive computing jobs within a multitasking workflow. Using as a case study a set of legacy data from an archaeological collection in need of reorganization, we identified the steps required to re-sort and move approximately 27,000 data files into a structured collection architecture. Because not all data management workflows are the same, and because there are a wide range of requirements for job submission within data-intensive HPC resources, we derived a set of generalizable modules that can be used as a guide for curators and HPC consultants. This framework may accommodate collections with different data types and data management requirements and can be conducted by curators trained in HPC usage but without ample computational expertise. Upon testing, we implemented the framework as a service on a DIC/HPC cluster.

关键词： Multitasking data management services high performance computing archaeology data data intensive computing

来源：评论

学校读者我要写书评

暂无评论

MapReduce Workload Modeling with Statistical Approach

引用

JOURNAL OF GRID computing 2012年第2期10卷 279-310页

作者： Yang, Hailong Luan, Zhongzhi Li, Wenjun Qian, Depei Beihang Univ Sino German Joint Software Inst State Key Lab Software Dev Environm Sch Comp Sci & Engn Beijing Peoples R China

Large-scale data-intensive cloud computing with the MapReduce framework is becoming pervasive for the core business of many academic, government, and industrial organizations. Hadoop, a state-of-the-art open source project, is by far the most successful realization of MapReduce framework. While MapReduce is easy- to-use, efficient and reliable for data-intensive computations, the excessive configuration parameters in Hadoop impose unexpected challenges on running various workloads with a Hadoop cluster effectively. Consequently, developers who have less experience with the Hadoop configuration system may devote a significant effort to write an application with poor performance, either because they have no idea how these configurations would influence the performance, or because they are not even aware that these configurations exist. There is a pressing need for comprehensive analysis and performance modeling to ease MapReduce application development and guide performance optimization under different Hadoop configurations. In this paper, we propose a statistical analysis approach to identify the relationships among workload characteristics, Hadoop configurations and workload performance. We apply principal component analysis and cluster analysis to 45 different metrics, which derive relationships between workload characteristics and corresponding performance under different Hadoop configurations. Regression models are also constructed that attempt to predict the performance of various workloads under different Hadoop configurations. Several non-intuitive relationships between workload characteristics and performance are revealed through our analysis and the experimental results demonstrate that our regression models accurately predict the performance of MapReduce workloads under different Hadoop configurations.

关键词： Cloud computing data intensive computing MapReduce Workload characterization Statistical analysis Performance prediction

来源：评论

学校读者我要写书评

暂无评论

Decreasing power consumption with energy efficient data aware strategies

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2013年第5期29卷 1152-1163页

作者： Vrbsky, Susan V. Galloway, Michael Carr, Robert Nori, Rahul Grubic, David Univ Alabama Dept Comp Sci Tuscaloosa AL 35487 USA

Regardless of whether data is stored in a cluster, grid, or cloud, data management is being recognized as a significant bottleneck. computing elements can be located far away from the data storage elements. The energy efficiency of the data centers storing this data is one of the biggest issues in data intensive computing. In order to address such issues, we are designing and analyzing a series of energy efficient data aware strategies involving data replication and CPU scheduling. In this paper, we present a new strategy for data replication, called Queued Least-Frequently-Used (QLFU), and study its performance to determine if it is an energy efficient strategy. We also study the benefits of using a data aware CPU scheduling strategy, called data backfilling, which uses job preemption in order to maximize CPU usage and allows for longer periods of suspension time to save energy. We measure the performance of QLFU and existing replica strategies on a small green cluster to study the running time and power consumption of the strategies with and without data backfilling. Results from this study have demonstrated that energy efficient data management can reduce energy consumption without negatively impacting response time. (c) 2013 Elsevier B.V. All rights reserved.

关键词： data backfilling data cluster data grid data intensive computing data replication Energy Power consumption QLFU

来源：评论

学校读者我要写书评

暂无评论

Task assignment policies in distributed server systems: A survey

引用

JOURNAL OF NETWORK AND COMPUTER APPLICATIONS 2011年第4期34卷 1123-1130页

作者： Semchedine, Fouzi Bouallouche-Medjkoune, Louiza Aissani, Djamil Univ Bejaia LAMOS Lab Modeling & Optimizat Syst Bejaia 06000 Algeria Univ Bejaia Doctoral Sch Comp Sci Networking & Distributed Sy Bejaia 06000 Algeria

data intensive computing, in the Web environment, motivates the distributed designs of Web server systems (Web clusters) because of their scalability and cost-effectiveness instead of one Web server with high performance. The task assignment policy, in such systems, focuses on the manner of assigning the tasks that reach these systems (e.g. the case of intensive requests that reach the distributed Web server systems) in order to minimize the response time and thus, improve the performance. These tasks, generally, follow the "heavy-tailed" distribution which has the property that there is a tiny fraction (about 3%) of large tasks that makes half (50%) of the total load. Several policies were proposed in the literature to deal with the nature of this Web traffic. This paper presents a state-of-art of the existing task assignment policies. We classify these policies in two classes: policies which assume that the task size is known a priori, and policies which assume that the task size is not known a priori (like TAGS, TAPTF and TAPTF-WC). The first class of policies regroups policies which consider no knowledge of load information at the servers when assigning the incoming tasks, known as static policies (like Random, Round Robin, etc.) and, policies, known as dynamic policies (like Central Queue Policy, Least Loaded First "LLF", etc.) which use some load information (e.g. the processing capacity, the queue load, etc.) to process. (C) 2011 Elsevier Ltd. All rights reserved.

关键词： data intensive computing Task assignment Distributed server systems Heavy-tailed distribution Load balancing Load sharing Performance

来源：评论

学校读者我要写书评

暂无评论

A high-performance application data environment for large-scale scientific computations

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2003年第12期14卷 1262-1274页

作者： Shen, XH Liao, WK Chouldhary, A Memik, G Kandemir, M Motorola Inc Core Technol Dept Libertyville IL 60048 USA Northwestern Univ Dept Elect & Comp Engn Evanston IL 60208 USA Penn State Univ Dept Comp Sci & Engn University Pk PA 16802 USA

Effective high-level data management is becoming an important issue with more and more scientific applications manipulating huge amounts of secondary-storage and tertiary-storage data using parallel processors. A major problem facing the current solutions to this data management problem is that these solutions either require a deep understanding of specific data storage architectures and file layouts to obtain the best performance (as in high-performance storage management systems and parallel file systems), or they sacrifice significant performance in exchange for ease-of-use and portability (as in traditional database management systems). In this paper, we discuss the design, implementation, and evaluation of a novel application development environment for scientific computations. This environment includes a number of components that make it easy for the programmers to code and run their applications without much programming effort and, at the same time, to harness the available computational and storage power on parallel architectures.

关键词： data intensive computing access pattern storage pattern MDMS

来源：评论

学校读者我要写书评

暂无评论

A data intensive heuristic approach to the two-stage streaming scheduling problem

引用

JOURNAL OF COMPUTER AND SYSTEM SCIENCES 2017年 89卷 64-79页

作者： Liang, Wei Hu, Chunhua Wu, Min Jin, Qun Hunan Univ Commerce Key Lab Hunan Prov Mobile Business Intelligence Changsha Hunan Peoples R China Hunan Univ Commerce Mobile E Busines Collaborat Innovat Ctr Hunan Pro Changsha Hunan Peoples R China China Univ Geosci Sch Automat Wuhan Hubei Peoples R China Waseda Univ Fac Human Sci Tokorozawa Saitama Japan

data intensive computing (DIC) provides a high performance computing approach to process large volume of data. In this study, a new formalization is introduced to present the two-stage DIC task execution in a stream manner. A novel heuristic algorithm is proposed for the scheduling problem due to the NP complexity. The theoretical approximation ratio bounds for the heuristic are analyzed and confirmed by the experimental evaluation. Overall, we observe that the proposed method conducts average 1.2 times makespan than the theoretic bound of the optimal solution. Besides, the proposed method outperforms the GA and FIFO scheduling schemes with overall improvements. (C) 2017 Elsevier Inc. All rights reserved.

关键词： data intensive computing Scheduling Makespan NP-hard

来源：评论

学校读者我要写书评

暂无评论

Modeling of large-scale social network services based on mechanisms of information diffusion: Sina Weibo as a case study

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2017年 74卷 291-301页

作者： Wang, Ru Rho, Seungmin Chen, Bo-Wei Cai, Wandong Northwestern Polytech Univ Sch Comp Sci & Engn Xian 710072 Peoples R China Sungkyul Univ Dept Media Software Anyang South Korea Monash Univ Malaysia Campus Sch Informat Technol Subang Jaya Selangor Malaysia

The purpose of this paper is to investigate the characteristics of the dissemination of information in the community. A variety of possible factors that affect the dissemination of information in Sina Weibo have been discussed. By analyzing the process of the information dissemination in the community of Sina Weibo, we found the information dissemination of Weibo community and the dynamic model are very similar. With the aid of data intensive computing theory, the various features have been mined and modeled. The dynamic model is improved and redefined to characterize the community. Then the SEINR model is proposed. The basic reproductive number, the existence of equilibrium point and the stability of the network are analyzed and proved in detail. By comparing with real data in Weibo community, we show that the SEINR model accurately reflects the dissemination of information community. Furthermore, we investigate the SEINR model in detail to show the influences of different parameters on information dissemination by simulations. (C) 2016 Elsevier B.V. All rights reserved.

关键词： Sina Weibo data intensive computing SEINR Propagation dynamics Basic reproduction number Equilibrium point

来源：评论

学校读者我要写书评

暂无评论

Design and implementation of task scheduling strategies for massive remote sensing data processing across multiple data centers

引用

SOFTWARE-PRACTICE & EXPERIENCE 2014年第7期44卷 873-886页

作者： Zhang, Wanfeng Wang, Lizhe Ma, Yan Liu, Dingsheng Chinese Acad Sci Inst Remote Sensing & Digital Earth Beijing 100864 Peoples R China China Univ Geosci Sch Comp Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China

data intensive applications of remote sensing data processing are more and more widespread resulting from the evolutions in computer and network technologies. Especially, bags-of-tasks (BoTs) applications with a mass of sharing input files and directed acyclic graph (DAG) applications with data dependencies in a widely distributed computing environment bring new challenges. In this article, a strategy of partitioning group based on hypergraph (PGH) is introduced to formulate the model of sharing files. Within the PGH algorithm, BoTs applications would be partitioned into several groups to minimize the time of data transferring. We also adopted another scheduling policy, which is called optimized task tree (OTT) strategy to handle the DAG workflow of massive remote sensing data processing with data dependencies. A scheduling queue of DAG tasks would be updated according to the priorities changing. With the help of GridSim simulation environment, we designed the Gridlets within scheduler to test the performance of PGH and OTT. Copyright (c) 2013 John Wiley & Sons, Ltd.

关键词： multi-datacenter infrastructure data intensive computing task scheduling big data computing

来源：评论

学校读者我要写书评

暂无评论

A simulation and data analysis system for large-scale, data-driven oil reservoir simulation studies

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2005年第11期17卷 1441-1467页

作者： Kurc, T Catalyurek, U Zhang, X Saltz, J Martino, R Wheeler, M Peszynska, M Sussman, A Hansen, C Sen, M Seifoullaev, R Stoffa, P Torres-Verdin, C Parashar, M Ohio State Univ Biomed Informat Dept Columbus OH 43210 USA Univ Texas Ctr Subsurface Modeling Inst Computat Engn & Sci Austin TX 78712 USA Oregon State Univ Dept Math Corvallis OR 97331 USA Univ Maryland Dept Comp Sci College Pk MD 20742 USA Univ Texas Inst Geophys Austin TX 78758 USA Univ Texas Dept Petr & Geosyst Engn Austin TX 78712 USA Rutgers State Univ Dept Elect & Comp Engn Piscataway NJ 08854 USA

The main goal of oil reservoir management is to provide more efficient, cost-effective and environmentally safer production of oil from reservoirs. Numerical simulations can aid in the design and implementation of optimal production strategies. However, traditional simulation-based approaches to optimizing reservoir management are rapidly overwhelmed by data volume when large numbers of realizations are sought using detailed geologic descriptions. In this paper, we describe a software architecture to facilitate large-scale simulation studies, involving ensembles of long-running simulations and analysis of vast volumes of output data. Copyright (c) 2005 John Wiley & Sons, Ltd.

关键词： oil reservoir simulation Grid computing data intensive computing data management

来源：评论

学校读者我要写书评

暂无评论

Towards building a multi-datacenter infrastructure for massive remote sensing image processing

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2013年第12期25卷 1798-1812页

作者： Zhang, Wanfeng Wang, Lizhe Liu, Dingsheng Song, Weijing Ma, Yan Liu, Peng Chen, Dan Chinese Acad Sci Ctr Earth Observat & Digital Earth Beijing Peoples R China China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Chinese Acad Sci Grad Univ Beijing Peoples R China

Earth observation applications are now facing the challenges of managing and processing massive data sets from multiple sources from large-scale distributed data centers (DCs). To solve this research problem, this paper presents an infrastructure of multiple data centers (MDC) for managing and processing massive remote sensing images. The proposed system is built on both groups of distributed DCs/clusters, which are equipped with DC or cluster resource manager. Access security and information service are introduced to support this architecture of MDC. We collaboratively organized the algorithm, and data belonged to the MDC in the manner of workflow. In practice, we succeeded in working out the concrete problems regarding procedures in processing applications collaboratively and transfer the massive remote sensing dataset fast and with stable cross-MDC. On the basis of the previously mentioned research work, we will investigate the platform integration of MDC. Copyright (c) 2012 John Wiley & Sons, Ltd.

关键词： data intensive computing remote sensing image processing multi-datacenter infrastructure cloud computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：