检索结果-内蒙古大学图书馆

Meeting deadlines for approximation processing in MapReduce environments

Frontiers of Information technology & Electronic Engineering 2017年第11期18卷 1754-1772页

作者： Ming-hao HU Chang-jian WANG Yu-xing PENG National Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha 410073 China

To provide timely results for big data analytics, it is crucial to satisfy deadline requirements for MapReduce jobs in today＇s production environments. Much effort has been devoted to the problem of meeting deadlines, and typically there exist two kinds of solutions. The first is to allocate appropriate resources to complete the entire job before the specified time limit, where missed deadlines result because of tight deadline constraints or lack of resources; the second is to run a pre-constructed sample based on deadline constraints, which can satisfy the time requirement but fail to maximize the volumes of processed data. In this paper, we propose a deadline-oriented task scheduling approach, named ＇Dart＇, to address the above problem. Given a specified deadline and restricted resources, Dart uses an iterative estimation method, which is based on both historical data and job running status to precisely estimate the real-time job completion time. Based on the estimated time, Dart uses an approach-revise algorithm to make dynamic scheduling decisions for meeting deadlines while maximizing the amount of processed data and mitigating stragglers. Dart also efficiently handles task failures and data skew, protecting its performance from being harmed. We have validated our approach using workloads from OpenCloud and Facebook on a cluster of 64 virtual machines. The results show that Dart can not only effectively meet the deadline but also process near-maximum volumes of data even with tight deadlines and limited resources.

关键词： MapReduce Approximation jobs Deadline Task scheduling Straggler mitigation

来源：评论

学校读者我要写书评

暂无评论

A mobile recommendation system based on logistic regression and Gradient Boosting Decision Trees

A mobile recommendation system based on logistic regression ...

引用

2016 International Joint Conference on Neural Networks, IJCNN 2016

作者： Wang, Yaozheng Feng, Dawei Li, Dongsheng Chen, Xinyuan Zhao, Yunxiang Niu, Xin National Lab for Parallel and Distributed Processing School of Computer Science National University of Defense Technology Changsha410003 China

ISBN: (纸本)9781509006199

Real-life behaviors shown by the mobile users typically exhibit plenty noises, making it hard to construct an effective recommendation engine. In this paper, we present a fused model based on the LR algorithm and the GBDT algorithm to recommend vertical industry commodities in a mobile setting. A set of specifically designed methods are proposed to deal with the data preprocessing and feature extraction problem for the mobile recommendation scenario. The proposed method is evaluated on a large scale real-world dataset provided by the Alibaba mobile shopping department. Result on the F1 score has seen an improvement of 2%-36% compared with the baseline. © 2016 IEEE.

关键词： Recommender systems

来源：评论

学校读者我要写书评

暂无评论

A knowledge graph embedding method based on neural network 5

A knowledge graph embedding method based on neural network

引用

5th IEEE International Conference on Data science in Cyberspace, DSC 2020

作者： Li, Chenchen Li, Aiping Tu, Hongkui Wang, Ye Wang, Changhai National University of Defense Technology College of Computer ChangSha China National University of Defense Technology College of Computer Science and Technology on Parallel and Distributed Processing Laboratory ChangSha China

ISBN: (数字)9781728195582

ISBN: (纸本)9781728195582

As the basis of many knowledge graph completion tasks, the embedding representation of entities and relations in knowledge graph (KG) is an important task in the fields of Natural Language processing (NLP) and Artificial Intelligence (AI). While most of the existing knowledge graph embedding (KGE) models based on convolutional neural network (CNN) can obtain abundant feature embedding, they may ignore an important fact that the triples in the KG come from the text, as they simply learn about the feature embedding of entities and relations without considering contextual information. Therefore, in this paper, we propose an effective KGE model based on neural network. First of all, we convert the triple (h, r, t) of the KG into a sentence [h r t]. Then, the LSTM neural network is used to learn the long-term dependence of sentences from the input feature vectors. Then, on this basis, the two-layer convolutional neural network with several different filters is used to extract different local features. Finally, the obtained feature vectors are connected together, and the inner product is carried out with the weight vectors to obtain the score of the triple, so as to judge the validity of the given triple. We evaluate our model on two benchmark datasets FB15k-237 and WN18RR, the experimental results show that the model can effectively improve the accuracy of link prediction, achieving better results compared with other baseline models. © 2020 IEEE.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Tree-Structured parallel Regeneration for Multiple Data Losses in distributed Storage Systems Based on Erasure Codes

引用

China Communications 2013年第4期10卷 113-125页

作者：孙伟东王意洁裴晓强 National Key Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology

To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.

关键词： distributed storage system erasure code replication regeneration tree

来源：评论

学校读者我要写书评

暂无评论

Multi-class contour preserving classification

Multi-class contour preserving classification

引用

13th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2012

作者： Fuangkhon, Piyabute Tanprasert, Thitipong Distributed and Parallel Computing Research Laboratory Faculty of Science and Technology Assumption University Bangkok Thailand

ISBN: (纸本)9783642326387

The original contour preserving classification technique was proposed to improve the robustness and weight fault tolerance of a neural network applied with a two-class linearly separable problem. It was recently found to be improving the level of accuracy of two-class classification. This paper presents an augmentation of the original technique to improve the level of accuracy of multi-class classification by better preservation of the shape or distribution model of a multi-class problem. The test results on six real world multi-class datasets from UCI machine learning repository present that the proposed technique supports multi-class data and can improve the level of accuracy of multi-class classification more effectively. © 2012 Springer-Verlag.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

80GBaud time division multiplexing optical transceiver

80GBaud time division multiplexing optical transceiver

引用

Integrated Photonics Research, Silicon and Nanophotonics, IPRSN 2016

作者： Zhang, Luo Mingche, Lai Zhengbin, Pang Liquan, Xiao Weixia, Xu School of Computer Science National University of Defense Technology Changsha410073 China Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China

High baud rate optical transceiver based on time division multiplexing technology is proposed. A communication channel at 80GBoud with 4 bit streams at 20Gbps is realized by 4-stage cascaded high speed switches with s... 详细信息

ISBN: (纸本)9781943580149

关键词： Photonics

来源：评论

学校读者我要写书评

暂无评论

Mini-batch cutting plane method for regularized risk minimization

引用

Frontiers of Information technology & Electronic Engineering 2019年第11期20卷 1551-1563页

作者： Meng-long LU Lin-bo QIAO Da-wei FENG Dong-sheng LI Xi-cheng LU Science and Technology on Parallel and Distributed Laboratory National University of Defense TechnologyChangsha 410073China

Although concern has been recently expressed with regard to the solution to the non-convex problem, convex optimization is still important in machine learning, especially when the situation requires an interpretable model. Solution to the convex problem is a global minimum, and the final model can be explained mathematically. Typically, the convex problem is re-casted as a regularized risk minimization problem to prevent overfitting. The cutting plane method (CPM) is one of the best solvers for the convex problem, irrespective of whether the objective function is differentiable or not. However, CPM and its variants fail to adequately address large-scale data-intensive cases because these algorithms access the entire dataset in each iteration, which substantially increases the computational burden and memory cost. To alleviate this problem, we propose a novel algorithm named the mini-batch cutting plane method (MBCPM), which iterates with estimated cutting planes calculated on a small batch of sampled data and is capable of handling large-scale problems. Furthermore, the proposed MBCPM adopts a "sink" operation that detects and adjusts noisy estimations to guarantee convergence. Numerical experiments on extensive real-world datasets demonstrate the effectiveness of MBCPM, which is superior to the bundle methods for regularized risk minimization as well as popular stochastic gradient descent methods in terms of convergence speed.

关键词： Machine learning Optimization methods Gradient methods Cutting plane method

来源：评论

学校读者我要写书评

暂无评论

parallelization of the training for face detection with transactional memory

Parallelization of the training for face detection with tran...

引用

2011 International Conference on Automation and Robotics, ICAR 2011

作者： Zeng, Kun Key Laboratory and Technology for National Defence of Parallel and Distributed Processing School of Computer National Univiersity of Defense Technology ChangSha Hunan China

ISBN: (纸本)9783642256455

The development of multi-core processor makes the parallelization of traditional sequential algorithms increasingly important. Meanwhile, transactional memory serves a good parallel programming model. This paper takes the advantage of software transactional memory to parallelize the Multi-Exit Asymmetric Adaboost algorithm for face detection. The parallel version is evaluated on three different implementations of software transactional memory. The experiment results show that the transactional memory based parallelization outperforms the traditional lock based approach. A speedup of nearly seven is achieved on a eight-core machine on an eight-core system. © 2011 Springer-Verlag.

关键词： Face recognition

来源：评论

学校读者我要写书评

暂无评论

A survey on approaches and applications of knowledge representation learning 5

A survey on approaches and applications of knowledge represe...

引用

5th IEEE International Conference on Data science in Cyberspace, DSC 2020

作者： Li, Chenchen Li, Aiping Wang, Ye Tu, Hongkui Song, Yichen National University of Defense Technology College of Computer ChangSha China National University of Defense Technology College of Computer Science and Technology on Parallel and Distributed Processing Laboratory ChangSha China

ISBN: (纸本)9781728195582

Knowledge representation learning (KRL) is one of the important research topics in artificial intelligence and Natural language processing. It can efficiently calculate the semantics of entities and relations in a low-dimensional space, and effectively solve the problem of data sparsity, which significantly improve the performance of knowledge acquisition, fusion and reasoning and so on. Starting from the five perspectives of distance-based, semantic matching, bilinear-based, neural network model and additional information model, this paper first introduces the overall framework and specific model design, and then correspondingly introduces the experimental evaluation tasks, metrics and benchmark datasets of each model. On this basis, how to apply KRL to various downstream tasks is summarized. © 2020 IEEE.

关键词： Knowledge representation

来源：评论

学校读者我要写书评

暂无评论

Skyline query processing on interval uncertain data

Skyline query processing on interval uncertain data

引用

作者： Li, Xiaoyong Wang, Yijie Li, Xiaoling Wang, Guangdong National Lab. for Parallel and Distributed Processing School of Computer Science National University of Defense Technology Changsha China

ISBN: (纸本)9780769546698

Many recent applications involve processing and analyzing uncertain data. Recently, several research efforts have addressed answering skyline queries efficiently on massive uncertain datasets. However, the research lacks methods to compute these queries on uncertain data, where each dimension of the uncertain object is represented as an interval or an exact value. In this paper, we extensively study the problem of skyline query on these interval based uncertain objects, which has never been studied before. We first model the problem of querying the skylines on interval datasets. Typically, we address two efficient algorithms with I/O optimal for the conventional interval skyline queries and constrained interval skyline queries, respectively. Extensive experiments demonstrate the efficiency of all our proposed algorithms. © 2012 IEEE.

关键词： Query processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：