检索结果-内蒙古大学图书馆

作者： Liu, Yuan Shi, Xuanhua Jin, Hai Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Long-running stream applications usually share the same fundamental computational infrastructure. To improve the efficiency of data processing in stream processing systems, a data analysis operator could be partitioned into n parallel tasks. The partitioned tasks are usually deployed on m nodes coexisting with other application operators. Because the node performance can vary in unpredictable ways (i.e., (1) stream input rates may fluctuate and (2) computational resource availability varies as other applications are affected), the nodes have different processing steps, and the slow node determines the operator performance. Hence, the tasks should be redistributed at runtime for stream applications to meet their strict latency requirements. Our key idea is to redistribute the tasks to the best node dynamically adaptive to resource or load fluctuations. In this paper, we present a runtime-aware adaptive schedule mechanism that aims at minimizing the operator processing latency and minimizing the latency difference between different nodes' tasks. We propose a new abstraction called performance cost ratio (PCR) that evaluates the node performance. The higher the node's PCR is, the less cost the node will pay for processing one tuple, and the more tasks should be deployed on it. In a scheduling, we first sort tasks descendingly by their loads and sort nodes by their PCR. Then we reassign the amount of computation according to the node's PCR to keep the node's PCR and its input rate the same or in similar proportion in all PCRs. The PCR-based quantitative algorithm applies itself to make tasks loads quantized to the processing capacity of nodes, move the minimum amount of operator's tasks, and keep the tasks local at the same time. We have implemented a runtime-aware adaptive scheduler as an extension to Storm and evaluated this strategy. We achieve the optimization goal using less computational resources. Copyright © 2015 John Wiley & Sons, Ltd. Copyright © 2015 John Wil

关键词： Scheduling

来源：评论

学校读者我要写书评

暂无评论

CDMCR: multi-level fault-tolerant system for distributed applications in cloud

CDMCR: multi-level fault-tolerant system for distributed app...

引用

作者： Qiang, Weizhong Jiang, Changqing Ran, Longbo Zou, Deqing Jin, Hai Services Computing Technology and System Lab and Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Cloud provides users with a new model of utilizing the computing infrastructure with the ability to perform parallel and distributed computations using elastic virtual cluster. However, the multi-level and complex features make cloud computing system more prone to failure. In this paper, we present a multi-level fault-tolerant system for distributed applications in cloud named Distributed-application oriented Multi-level Checkpoint/Restart for Cloud (CDMCR). The CDMCR system backups the complete state of applications periodically with a snapshot-based distributed checkpointing protocol, including file system state. Thus, we cannot only recover processes but also rollback data. A multi-level recovery strategy is proposed, which includes process-level recovery, virtual machine recreation, and host rescheduling, enabling comprehensive and efficient fault tolerance for different components in cloud. We deploy CDMCR as PaaS, so that users can be liberated from node management and system configuration and get access to fault-tolerant service conveniently. We have implemented this system based on the Xen virtualization platform and the OpenNebula cloud platform. Experiments on the prototype demonstrate the correctness of the system. Analysis shows that CDMCR does not cause message loss or data loss, and the backup time remains nearly constant as the number of nodes increases on virtual cluster. Copyright © 2015 John Wiley & Sons, Ltd. Copyright © 2015 John Wiley & Sons, Ltd.

关键词： Copyrights

来源：评论

学校读者我要写书评

暂无评论

A survey of cloud resource management for complex engineering applications

引用

Frontiers of Computer Science 2016年第3期10卷 447-461页

作者： Haibao CHEN Song WU Hai JIN Wenguang CHEN Jidong ZHAI Yingwei LUO Xiaolin WANG Services Computing Technology and System Lab Cluster and Grid Computing LabSchool of Computer Science and Technology Huazhong University of Science and Technology Wuhan 430074 China Institute of High Performance Computing Tsinghua University Beijing 100084 China School of Electronics Engineering and Computer Science Peking University Beijing 100871 China School of Computer and Information Engineering Chuzhou University Chuzhou 239000 China

Traditionally, complex engineering applications （CEAs）, which consist of numerous components （software） and require a large amount of computing resources, usu- ally run in dedicated clusters or high performance computing （HPC） centers. Nowadays, Cloud computing system with the ability of providing massive computing resources and cus- tomizable execution environment is becoming an attractive option for CEAs. As a new type on Cloud applications, CEA also brings the challenges of dealing with Cloud resources. In this paper, we provide a comprehensive survey of Cloud resource management research for CEAs. The survey puts forward two important questions： 1） what are the main chal- lenges for CEAs to run in Clouds？ and 2） what are the prior research topics addressing these challenges？ We summarize and highlight the main challenges and prior research topics. Our work can be probably helpful to those scientists and en- gineers who are interested in running CEAs in Cloud envi- ronment.

关键词： Cloud computing complex engineering application resource management virtualization

来源：评论

学校读者我要写书评

暂无评论

Uniform Information Exchange in Multi-channel Wireless Ad Hoc Networks 17

Uniform Information Exchange in Multi-channel Wireless Ad Ho...

引用

International Conference on Autonomous Agents and Multiagent systems

作者： Dongxiao Yu Li Ning Yong Zhang Hai Jin Yuexuan Wang Francis C. M. Lau Shengzhong Feng Services Computing Technology and System Lab Big Data Technology and System Lab Clusters and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Center for High Performance Computing Shenzhen Institutes of Advanced Technology CAS Department of Computer Science The University of Hong Kong

ISBN: (纸本)9781510855076

Information exchange is a basic primitive for maintaining the smooth running of a network or a system with multiple communicating agents. Given k packets initially stored at k nodes respectively, the problem is to disseminate the k packets to the whole network with the objective of minimizing the time used. We study this problem in single-hop multi-channel networks of n nodes, and target on devising uniform distributed protocols that do not rely on any prior knowledge of network parameters, such as the network size n or the number of packet holders k. Uniform protocols have better scalability and are more suitable for implementation in reality Specifically, we propose a uniform distributed protocol that with high probability accomplishes the dissemination in O(k/F+F·log n) rounds, assuming F available channels. This protocol is asymptotically optimal when k is large (k ≥ F~2 log n), and provides the best possible linear speedup with multiple channels comparing with the results using a single channel. To the best of our knowledge, this is the first uniform protocol for information exchange in multi-channel networks.

关键词： Information exchange Multi-channel networks Distributed algorithm Uniform protocol information exchange distributed algorithms MULTICHANNEL Packet Uniform Uniforms Mobile ad hoc networks distributed protocol network parameters single-channel

来源：评论

学校读者我要写书评

暂无评论

UiLog： Improving Log-Based Fault Diagnosis by Log Analysis

引用

Journal of Computer Science & technology 2016年第5期31卷 1038-1052页

作者： De-Qing Zou Services Computing Technology and System Laboratory Huazhong University of Science and Technology Wuhan ~3007~ China Big Data Technology and System Laboratory Huazhong University of Science and Technology Wuhan 430074 China Cluster and Grid Computing Laboratory Huazhong University of Science and Technology Wuhan 430074 China School of Computer Science and Technology Huazhong University of Science and Technology Wuhan 430074 China

In modern computer systems, system event logs have always been the primary source for checking system status. As computer systems become more and more complex, the interaction between software and hardware increases frequently. The components will generate enormous log information, including running reports and fault information. The sheer quantity of data is a great challenge for analysis relying on the manual method. In this paper, we implement a management and analysis system of log information, which can assist system administrators to understand the real-time status of the entire system, classify logs into different fault types, and determine the root cause of the faults. In addition, we improve the existing fault correlation analysis method based on the results of system log classification. We apply the system in a cloud computing environment for evaluation. The results show that our system can classify fault logs automatically and effectively. With the proposed system, administrators can easily detect the root cause of faults.

关键词： fault diagnosis system event log log classification fault correlation analysis

来源：评论

学校读者我要写书评

暂无评论

HCOpt: An automatic optimizer for configuration parameters of Hadoop 2nd

HCOpt: An automatic optimizer for configuration parameters o...

引用

2nd International Conference on Human Centered computing, HCC 2016

作者： Zhang, Xiong Zeng, Linxi Shi, Xuanhua Wu, Song Xie, Xia Jin, Hai Services Computing Technology and System Laboratory Big Data Technology and System Laboratory Cluster and Grid Computing Laboratory School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

ISBN: (纸本)9783319318530

MapReduce is an efficient tool for data-intensive applications. Hadoop, an open-source implementation of MapReduce, has been widely adopted and experienced by some enterprises and scientific computing communities. However, when users intend to run a MapReduce program in Hadoop, they have to set a number of configuration parameters to make sure the program runs efficiently. Users often run into performance problems because they are unaware of how to set these parameters. To address these performance problems, we focus on the optimization opportunities presented by the high configurability of Hadoop, and propose an automation tool named HCOpt for performance optimization of Hadoop configuration parameters. HCOpt uses a Profile Engine to collect monitoring information from running MapReduce programs, a Prediction Engine to estimate the performance of a given Hadoop configuration and a genetic-based search algorithm to find an optimized configuration in the large search space. Our evaluation shows that HCOpt reduces the job completion time of Hadoop applications by up to 20 % when compared to applications run with configuration parameters that suggested by the rule-based optimization. © Springer International Publishing Switzerland 2016.

关键词： MapReduce

来源：评论

学校读者我要写书评

暂无评论

Sponge:An Oversubscription Strategy Supporting Performance Interference Management in Cloud

引用

China Communications 2015年第11期12卷 196-209页

作者： LIU Ying Services Computing Technology and System Lab Cluster and Grid Computing LabSchool of Computer Science and TechnologyHuazhong University of Science and Technology

Resource oversubscription optimizes the utilization of the computing resources. Many well-known virtual machine monitors(VMMs)such as Xen and KVM,adopt this approach to help maximize the yield of the cloud datacenters That is,with proper resource oversubscription strategies,more virtual machines(VMs) can be supported by limited resources. However performance interference among VMs hosting in the same physical machines(PMs) exists in cloud environment,and probably aggravated by resource oversubscription strategies,which aims to put more VMs into the same PM. In this paper,we present a resource oversubscription strategy called Sponge targeting cloud platforms Sponge mitigates the issue of performance interference among the oversubscribed co-hosting VMs. Sponge also provides a VM association strategy for each PM to handle with its besteffort. We performed our evaluation on a virtua datacenter simulated by Xen. Our evaluation results show that Sponge improves the resources utilization and manages to make each VM mee its performance requirement even hosting with other VMs in the same PM.

关键词： resources oversubscription performance interference cloud computing resource management

来源：评论

学校读者我要写书评

暂无评论

A Novel Metadata Extraction Method for Surveillance Video

A Novel Metadata Extraction Method for Surveillance Video

引用

International Conference on computing, Networking and Communications

作者： Ran Zheng Long Chen Hai Jin Lei Zhu Qin Zhang Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology

ISBN: (纸本)9781467385800

Surveillance videos are increasing massively with monitoring cameras widely deployed in cities. Metadata of moving objects in videos can reduce video storage and be utilized in many video analysis applications. The whole metadata extraction is too time-consuming to be fulfilled quickly. Therefore, it is urgent to parallelize and accelerate the extraction process. However, iterative execution of traditional metadata extraction methods makes the parallelization quite challenging. In this paper, we propose a novel Parallel Metadata Extraction (PME) method for surveillance video. A novel video segmentation algorithm is designed to segment whole video into independent video segments. The metadata of each video segment is extracted independently and simultaneously on computer nodes, which will be integrated later to guarantee the completeness of the metadata. Performance evaluations demonstrate that PME can accelerate extraction process greatly almost without metadata loss.

关键词： Surveillance Videos Video Segmentation Metadata Extraction Iterative Execution MapReduce

来源：评论

学校读者我要写书评

暂无评论

Piggyback game: Efficient event stream dissemination in Online Social Network systems

Piggyback game: Efficient event stream dissemination in Onli...

引用

International Conference on Network Protocols

作者： Fan Zhang Hanhua Chen Hai Jin Services Computing Technology and System Lab Big Data Technology and System Lab and Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan China

ISBN: (纸本)9781509032822

Event stream dissemination dominates the workloads in large-scale Online Social Network (OSN) systems. Based on the de facto per-user view data storage, event stream dissemination raises a large amount of inter-server traffics due to the complex interconnection among OSN users. The state-of-the-art schemes mainly explore the structure features of social graphs to reduce the inter-server messages for event stream dissemination. Different sub-graph structures are exploited for achieving the approximated optimal assignment. However, such schemes incur high costs of computation or communication. In this work, we follow a different design philosophy by using a game theoretic approach, which decomposes the high complex graph computation problem into individuals' rational strategy selection of each node. Specifically, we propose a novel social piggyback game to achieve a more efficient solution. We mathematically prove the existing of the Nash Equilibrium of the social piggyback game. Moreover, we propose an efficient best response dynamic algorithm to achieve the Nash Equilibrium, which quickly converges in a small number of iterations for large-scale OSNs. We further show that the communication cost of this design achieves a 1.5-approximation of the theoretical social optimal. We conduct comprehensive experiments to evaluate the performance of this design using large-scale real-world traces from popular OSN systems. Results show that the social piggyback game achieves a significant 302× improvement in system efficiency compared to existing schemes.

关键词： Games Schedules Heuristic algorithms Algorithm design and analysis Nash equilibrium Social network services Servers

来源：评论

学校读者我要写书评

暂无评论

Conch: A Cyclic MapReduce Model for Iterative Applications

Conch: A Cyclic MapReduce Model for Iterative Applications

引用

Euromicro Conference on Parallel, Distributed and Network-Based Processing

作者： Ran Zheng Genmao Yu Hai Jin Xuanhua Shi Qin Zhang Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

MapReduce programming model is a popular model to simplify but speed up data parallel applications. However, it is not efficient for iterative applications because of its repeated data transmission with HDFS (Hadoop Distributed File system). Conch, a cyclic MapReduce model, is designed for efficient processing of iterative applications. In order to minimize network overhead, shared data is cached locally and a "map-shuffle" phase is presented with a combined transmission mechanism. Meanwhile, a prediction scheduler for iterative applications is brought out to achieve better data locality in terms of runtime information. The experiments show that Conch can support iterative applications transparently and efficiently. Compared with Hadoop and HaLoop in single-job environment, Conch can achieve 13%-17% improvements on K-Means and fuzzy C-Means. Especially in multi-job environment, 63.6% and 28.6% improvements can be obtained compared with Hadoop and HaLoop.

关键词： Delays Iterative methods Computational modeling Prediction algorithms Heart beat Data transfer Data models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：