检索结果-内蒙古大学图书馆

Meeting deadlines for approximation processing in MapReduce environments

Frontiers of Information Technology & Electronic Engineering 2017年第11期18卷 1754-1772页

作者： Ming-hao HU Chang-jian WANG Yu-xing PENG National Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha 410073 China

To provide timely results for big data analytics, it is crucial to satisfy deadline requirements for MapReduce jobs in today＇s production environments. Much effort has been devoted to the problem of meeting deadlines, and typically there exist two kinds of solutions. The first is to allocate appropriate resources to complete the entire job before the specified time limit, where missed deadlines result because of tight deadline constraints or lack of resources; the second is to run a pre-constructed sample based on deadline constraints, which can satisfy the time requirement but fail to maximize the volumes of processed data. In this paper, we propose a deadline-oriented task scheduling approach, named ＇Dart＇, to address the above problem. Given a specified deadline and restricted resources, Dart uses an iterative estimation method, which is based on both historical data and job running status to precisely estimate the real-time job completion time. Based on the estimated time, Dart uses an approach-revise algorithm to make dynamic scheduling decisions for meeting deadlines while maximizing the amount of processed data and mitigating stragglers. Dart also efficiently handles task failures and data skew, protecting its performance from being harmed. We have validated our approach using workloads from OpenCloud and Facebook on a cluster of 64 virtual machines. The results show that Dart can not only effectively meet the deadline but also process near-maximum volumes of data even with tight deadlines and limited resources.

关键词： MapReduce Approximation jobs Deadline Task scheduling Straggler mitigation

来源：评论

学校读者我要写书评

暂无评论

Design of an Indoor Exploration and Multi-Objective Navigation System

Design of an Indoor Exploration and Multi-Objective Navigati...

引用

Chinese Control Conference (CCC)

作者： Guoxian Zheng Lei Zhang Huaxi Yulin Zhang Bo Ding State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang China MIS Universite de Picardie Jules Verne Amiens France National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China

In this paper, we propose an indoor robot autonomous navigation system. The robot firstly explores in an unknown environment, and then navigates autonomously by using the explored map. The robot is equipped a 2D laser scanner as the main sensor. The laser scanner is used for path planning and frontier-based exploration. A 2D global occupancy map is built for path planning, frontier-based exploration and multi-objective autonomous navigation. Laser scans are transmitted into Simultaneous Localization and Mapping (SLAM) process in the exploration phase. In indoor environment, the exploration efficiency is improved by merging a heuristic algorithm. By using multi-threading technology and a 3D perception approach proposed in this paper, the robot equipped with a low-cost RGBD sensor can detect all kinds of obstacles to achieve highly reliable navigation in complicated 3D environment. Meanwhile, we develop a multi-objective navigation application to make human-robot interaction more convenient and satisfy multi-task deployment. Our approaches are demonstrated by experimental results.

关键词： Navigation Three-dimensional displays Robot sensing systems Lasers Path planning Two dimensional displays

来源：评论

学校读者我要写书评

暂无评论

General audio tagging with ensembling convolutional neural networks and statistical features

arXiv

引用

arXiv 2018年

作者： Xu, Kele Zhu, Boqing Kong, Qiuqiang Mi, Haibo Ding, Bo Wang, Dezhi Wang, Huaimin National Key Laboratory of Parallel and Distributed Processing Changsha China College of Computer National University of Defense Technology Changsha China University of Surrey United Kingdom College of Meteorology and Oceanography National Univ. of Defense Tech Changsha China

Audio tagging aims to infer descriptive labels from audio clips. Audio tagging is challenging due to the limited size of data and noisy labels. In this paper, we describe our solution for the DCASE 2018 Task 2 general audio tagging challenge. The contributions of our solution include: We investigated a variety of convolutional neural network architectures to solve the audio tagging task. Statistical features are applied to capture statistical patterns of audio features to improve the classification performance. Ensemble learning is applied to ensemble the outputs from the deep classifiers to utilize complementary information. a sample reweight strategy is employed for ensemble training to address the noisy label problem. Our system achieves a mean average precision (mAP@3) of 0.958, outperforming the baseline system of 0.704. Our system ranked the 1st and 4th out of 558 submissions in the public and private leaderboard of DCASE 2018 Task 2 challenge. Copyright © 2018, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

CloudDPI: Cloud-based privacy-preserving deep packet inspection via reversible sketch 1

引用

9th International Symposium on Cyberspace Safety and Security, CSS 2017

作者： Li, Jie Su, Jinshu Wang, Xiaofeng Sun, Hao Chen, Shuhui School of Computer National University of Defense Technology Changsha Hunan410073 China National Key Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha Hunan410073 China

ISBN: (数字)9783319694719

ISBN: (纸本)9783319694702

Hardware-based middleboxes are ubiquitous in computer networks, which usually incur high deployment and management expenses. A recently arsing trend aims to address those problems by outsourcing the functions of traditional hardware-based middleboxes to high volume servers in a cloud. This technology is promising but still faces a few challenges. First, the widely adopted data encryption techniques contradict with payload inspection needs of some middleboxes such as DPI and IDS devices. Second, the inspection rules of middleboxes may be commercial properties, thus the middlebox providers want to keep their rules confidential under third-party cloud environments, and this creates hindrances for the cloud to perform outsourced middlebox functions. Third, performance of the outsourced middlebox is an inevitable issue that needs deliberate consideration. In this paper, we propose a cloud-based DPI middlebox implementation which performs payload inspection over encrypted traffic while preserving the privacy of both communication data and inspection rules. Our design employs a modified reversible sketch structure which is used for efficient error-free membership testing, and we utilize unkeyed one-way hash functions instead of complex cryptographic protocols to achieve the privacy preservation requirements. CloudDPI supports a wide range of real-world inspection rules, we conduct evaluations on ClamAV rule set and the experiment results demonstrate the effectiveness of our proposal. © 2017, Springer International Publishing AG.

关键词： Network function virtualization

来源：评论

学校读者我要写书评

暂无评论

SAWSDL-iMatcher: A Customizable and Effective Semantic Web Service Matchmaker

SSRN

引用

SSRN 2018年

作者： Wei, Dengping Wang, Ting Wang, Ji Bernstein, Abraham National University of Defense Technology Changsha410073 China University of Zürich Binzmühlestrasse 14 ZürichCH-8050 Switzerland National Laboratory for Parallel and Distributed Processing Changsha410073 China Dynamic and Distributed Information Systems Group

As the number of publicly available services grows, discovering proper services becomes an important issue and has attracted amount of attempts. This paper presents a new customizable and effective matchmaker, called SAWSDL-iMatcher. It supports a matchmaking mechanism, named iXQuery, which extends XQuery with various similarity joins for SAWSDL service discovery. Using SAWSDL-iMatcher, users can flexibly customize their preferred matching strategies according to different application requirements. SAWSDL-iMatcher currently supports several matching strategies, including syntactic and semantic matching strategies as well as several statistical-model-based matching strategies which can effectively aggregate similarity values from matching on various types of service description information such as service name, description text, and semantic annotation. Besides, we propose a semantic matching strategy to measure the similarity among SAWSDL semantic annotations. These matching strategies have been evaluated in SAWSDL-iMatcher on SAWSDL-TC2 and Jena Geography Dataset (JGD). The evaluation shows that different matching strategies are suitable for different tasks and contexts, which implies the necessity of a customizable matchmaker. In addition, it also provides evidence for the claim that the effectiveness of SAWSDL service matching can be significantly improved by statistical-model-based matching strategies. Our matchmaker is competitive with other matchmakers on benchmark tests at S3 contest 2009. © 2018, The Authors. All rights reserved.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

Growing construction and adaptive evolution of complex software systems

引用

Science China(Information Sciences) 2016年第5期59卷 5-7页

作者： Huaimin WANG Bo DING National Key Laboratory of Parallel and Distributed Processing College of ComputerNational University of Defense Technology

distributed software systems are becoming more and more complex *** is easy to find a huge amount of computing nodes in a nationwide or global information *** example,We Chat(Wei Xin),a well-known mobile application in China,has reached a record of 650 million monthly active users in the third quarter of *** the same time,researchers are starting to talk about software systems which have billions of lines of codes[1]or can last one hundred years.

关键词： Growing construction and adaptive evolution of complex software systems

来源：评论

学校读者我要写书评

暂无评论

VirtMan:design and implementation of a fast booting system for homogeneous virtual machines in iVCE

引用

Frontiers of Information Technology & Electronic Engineering 2016年第2期17卷 110-121页

作者： Zi-yang LI Yi-ming ZHANG Dong-sheng LI Peng-fei ZHANG Xi-cheng LU National Laboratory for Parallel and Distributed Processing School of ComputerNational University of Defense Technology

Internet-based virtual computing environment （iVCE） has been proposed to combine data centers and other kinds of computing resources on the Internet to provide efficient and economical services. Virtual machines （VMs） have been widely used in iVCE to isolate different users/jobs and ensure trustworthiness, but traditionally VMs require a long period of time for booting, which cannot meet the requirement of iVCE＇s large-scale and highly dynamic applications. To address this problem, in this paper we design and implement VirtMan, a fast booting system for a large number of virtual machines in iVCE. VirtMan uses the Linux Small Computer System Interface （SCSI） target to remotely mount to the source image in a scalable hierarchy, and leverages the homogeneity of a set of VMs to transfer only necessary image data at runtime. We have implemented VirtMan both as a standalone system and for OpenStack. In our 100-server testbed, VirtMan boots up 1000 VMs （with a 15 CB image of Windows Server 2008） on 100 physical servers in less than 120 s, which is three orders of magnitude lower than current public clouds.

关键词： Virtual machine Fast booting Homogeneity Internet-based virtual computing environment （iVCE）

来源：评论

学校读者我要写书评

暂无评论

Determinants of pull-based development in the context of continuous integration

引用

Science China(Information Sciences) 2016年第8期59卷 53-66页

作者： Yue YU Gang YIN Tao WANG Cheng YANG Huaimin WANG College of Computer National University of Defense Technology National Laboratory for Parallel and Distributed Processing

The pull-based development model, widely used in distributed software teams on open source communities, can efficiently gather the wisdom from crowds. Instead of sharing access to a central repository,contributors create a fork, update it locally, and request to have their changes merged back, i.e., submit a pull-request. On the one hand, this model lowers the barrier to entry for potential contributors since anyone can submit pull-requests to any repository, but on the other hand it also increases the burden on integrators, who are responsible for assessing the proposed patches and integrating the suitable changes into the central repository. The role of integrators in pull-based development is crucial. They must not only ensure that pull-requests should meet the project’s quality standards before being accepted, but also finish the evaluations in a timely manner. To keep up with the volume of incoming pull-requests, continuous integration(CI) is widely adopted to automatically build and test every pull-request at the time of submission. CI provides extra evidences relating to the quality of pull-requests, which would help integrators to make final decision(i.e., accept or reject). In this paper, we present a quantitative study that tries to discover which factors affect the process of pull-based development model, including acceptance and latency in the context of CI. Using regression modeling on data extracted from a sample of Git Hub projects deploying the Travis-CI service, we find that the evaluation process is a complex issue, requiring many independent variables to explain adequately. In particular, CI is a dominant factor for the process, which not only has a great influence on the evaluation process per se, but also changes the effects of some traditional predictors.

关键词： pull-request continuous integration Git Hub distributed software development empirical analysis

来源：评论

学校读者我要写书评

暂无评论

Collaborative deep learning across multiple data centers

arXiv

引用

arXiv 2018年

作者： Xu, Kele Mi, Haibo Feng, Dawei Wang, Huaimin Chen, Chuan Zheng, Zibin Lan, Xu National Key Laboratory of Parallel and Distributed Processing Changsha China College of Computer National University of Defense Technology Changsha China School of Data and Computer Science Sun Yat-Sen University Guangzhou China Queen Mary University of London London United Kingdom

Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data to a centralized data center due to not only bandwidth limitation but also the constraints of privacy regulations. Model averaging is a conventional choice for data parallelized training, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model averaging can be effective in the decentralized environment by using two strategies, namely, the cyclical learning rate and the increased number of epochs for local model training. With the two strategies, we show that model averaging can provide competitive performance in the decentralized mode compared to the data-centralized one. In a practical environment with multiple data centers, we conduct extensive experiments using state-of-the-art deep network architectures on different types of data. Results demonstrate the effectiveness and robustness of the proposed method. Copyright © 2018, The Authors. All rights reserved.

关键词： Network architecture

来源：评论

学校读者我要写书评

暂无评论

A systematic review of structured sparse learning

引用

Frontiers of Information Technology & Electronic Engineering 2017年第4期18卷 445-463页

作者： Lin-bo QIAO Bo-feng ZHANG Jin-shu SU Xi-cheng LU College of Computer National University of Defense Technology Changsha 410073 China National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha 410073 China

High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumption of sparsity, many computational problems can be handled efficiently in practice. Structured sparse learning encodes the structural information of the variables and has been quite successful in numerous research fields. With various types of structures discovered, sorts of structured regularizations have been proposed. These regularizations have greatly improved the efficacy of sparse learning algorithms through the use of specific structural information. In this article, we present a systematic review of structured sparse learning including ideas, formulations, algorithms, and applications. We present these algorithms in the unified framework of minimizing the sum of loss and penalty functions, summarize publicly accessible software implementations, and compare the computational complexity of typical optimization methods to solve structured sparse learning problems. In experiments, we present applications in unsupervised learning, for structured signal recovery and hierarchical image reconstruction, and in supervised learning in the context of a novel graph-guided logistic regression.

关键词： Sparse learning Structured sparse learning Structured regularization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：