检索结果-内蒙古大学图书馆

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Xiandong Huang Qinglin Wang Shuyu Lu Ruochen Hao Songzhu Mei Jie Liu Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha China School of Computer Science National University of Defense Technology Changsha China University of Pittsburgh Pittsburgh USA

Convolutional Neural Networks (CNNs), one of the most representative algorithms of deep learning, are widely used in various artificial intelligence applications. Convolution operations often take most of the computational overhead of CNNs. The FFT-based algorithm can improve the efficiency of convolution by reducing its algorithm complexity, there are a lot of works about the high-performance implementation of FFT-based convolution on many-core CPUs. However, there is no optimization for the non-uniform memory access (NUMA) characteristics in many-core CPUs. In this paper, we present a NUMA-aware FFT-based convolution implementation on ARMv8 many-core CPUs with NUMA architectures. The implementation can reduce a number of remote memory access through the data reordering of FFT transformations and the three-level parallelization of the complex matrix multiplication. The experiment results on a ARMv8 many-core CPU with NUMA architectures demonstrate that our NUMA-aware implementation has much better performance than the state-of-the-art work in most cases.

关键词： Deep learning Recurrent neural networks Convolution Computer architecture Complexity theory Convolutional neural networks Kernel

来源：评论

学校读者我要写书评

暂无评论

EmbRace: Accelerating Sparse Communication for distributed Training of NLP Neural Networks

arXiv

引用

arXiv 2021年

作者： Li, Shengwei Lai, Zhiquan Li, Dongsheng Zhang, Yiming Ye, Xiangyu Duan, Yabo National Key Laboratory of Parallel Distributed Processing Computer College National University of Defense Technology China Xiamen University China

distributed data-parallel training has been widely adopted for deep neural network (DNN) models. Although current deep learning (DL) frameworks scale well for dense models like image classification models, we find that these DL frameworks have relatively low scalability for sparse models like natural language processing (NLP) models that have highly sparse embedding tables. Most existing works overlook the sparsity of model parameters thus suffering from significant but unnecessary communication overhead. In this paper, we propose EmbRace, an efficient communication framework to accelerate communications of distributed training for sparse models. EmbRace introduces Sparsity-aware Hybrid Communication, which integrates AlltoAll and model parallelism into data-parallel training, so as to reduce the communication overhead of highly sparse parameters. To effectively overlap sparse communication with both backward and forward computation, EmbRace further designs a 2D Communication Scheduling approach which optimizes the model computation procedure, relaxes the dependency of embeddings, and schedules the sparse communications of each embedding row with a priority queue. We have implemented a prototype of EmbRace based on PyTorch and Horovod, and conducted comprehensive evaluations with four representative NLP models. Experimental results show that EmbRace achieves up to 2.41X speedup compared to the state-of-the-art distributed training baselines. Copyright © 2021, The Authors. All rights reserved.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

Reply

引用

Trends in Neurosciences 1993年第5期16卷 P177-178页

作者： Mitsuo Kawato Hiroaki Gomi ATR Human Information Processing Research Laboratories Kyoto 619-02 Japan Laboratory of Parallel Distributed Processing Research Institute for Electronic Science Hokkaido University Sapporo Hokkaido 060 Japan

来源：评论

学校读者我要写书评

暂无评论

Skew-Aware Task Scheduling in Clouds

Skew-Aware Task Scheduling in Clouds

引用

2013 IEEE Seventh International Symposium on Service-Oriented System Engineering

作者： Dongsheng Li Yixing Chen Richard Hu Hai National Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology China Raffles Business Institute Singapore

ISBN: (纸本)9781467356596

Data skew is an important reason for the emergence of stragglers in MapReduce-like cloud systems. In this paper, we propose a Skew-Aware Task Scheduling (SATS) mechanism for iterative applications in MapReduce-like systems. The mechanism utilizes the similarity of data distribution in adjacent iterations of iterative applications to reduce the straggle problem caused by data skew. It collects the data distribution information during the execution of tasks for the current iteration, and uses the information to guide data partitioning in tasks for the next iteration. We implement the mechanism in the HaLoop system and deploy it in a cluster. Experiments show that the proposed mechanism could deal with the data skew and improve the load balancing effectively.

关键词： Load management distributed databases File systems Processor scheduling Computational modeling Data models Data structures

来源：评论

学校读者我要写书评

暂无评论

nSET: Novel simulation method for single-electron tunnel device with 1-Dimension multiple islands

nSET: Novel simulation method for single-electron tunnel dev...

引用

IEEE International Nanoelectronics Conference (INEC)

作者： Bingcai Sui Yaqing Chi Liang Fang National Laboratory of Parallel and Distributed Processing School of Computer National University of Defense Technology Hunan China National University of Defense Technology China

ISBN: (纸本)9781424435432

Single-electronic transistors (SETs) are considered as the attractive candidates for post-CMOS VLSI due to their ultra-small size and low power consumption. Because SETs with single island can not work at room temperature normally, more and more researchers begin to make research on the SETs with 1-dimension multi-islands. A new simulation method-nSET, is introduced in this paper. Compared with other methods, nSET can simulate the SET device with 1-dimension multiple islands with high speed and accuracy. Through the comparison, it can be get that nSET is accurate and fast compared with the classical Monte Carlo (MC) simulator, and is very useful for the ASIC design of SET devices.

关键词： Tunneling Programming profession Capacitance Circuit simulation Electrostatics Single electron transistors Application specific integrated circuits Discrete event simulation Very large scale integration Energy consumption

来源：评论

学校读者我要写书评

暂无评论

DLSM: Decoupled Live Storage Migration with distributed Device Mapper Storage

DLSM: Decoupled Live Storage Migration with Distributed Devi...

引用

2014 IEEE 8th International Symposium on Service Oriented System Engineering

作者： Zhaoning Zhang Ziyang Li Kui Wu Huiba Li Yuxing Peng Xicheng Lu National Key Laboratory for Parallel and Distributed Processing National University of Defense Technology China University of Victoria Victoria BC CA

As a key technique in cloud computing, live virtual machine (VM) migration makes cloud computing elastic and renders more efficient resource scheduling. In many cases, live storage migration is also desirable. Compared to live VM migration, however, live storage migration is relatively slow. The speed gap between live VM migration and live storage migration may lead to poor Quality of Service (QoS) for cloud users. In this paper, we present a new resource migration scheme, called Decoupled Live Storage Migration (DLSM), which decouples the live storage migration from the live VM migration. DLSM can migrate the VM immediately, while data blocks actually required by the VM are moved on demand at later times. It achieves the on-demand data migration at the storage level without the need of modifying the hypervisor. The key contribution of DLSM is that it reduces the network traffic and speeds up the live storage migration, by completely eliminating iterative copy of dirty data over the network. This good property is achieved without relying on any tracking mechanism for dirty data blocks. Experimental evaluation demonstrates the efficiency and effectiveness of DLSM.

关键词： Data transfer Switches Virtual machine monitors Convergence Linux Kernel Bandwidth

来源：评论

学校读者我要写书评

暂无评论

An Irregular cutting based distributed mapping algorithm for DVE systems

An Irregular cutting based distributed mapping algorithm for...

引用

Pacific-Asia Workshop on Computational Intelligence and Industrial Application, PACIIA

作者： Wei Zhang Hangjun Zhou Yuxing Peng Sikun Li Min Xi National Laboratory of Parallel and Distributed Processing School of Computer Science National University of Defense Technology Changsha China Department of Computer Science Xi'an Jiaotong University Xi'an China

distributed Virtual Environment (DVE) systems have become more and more important both in academic communities and the industries. To guarantee the load constrain, the physical world integrity and the virtual world integrity, the virtual environment should be divided into several parts and mapped to multiple servers, which are located in geographically different places. Many centralized algorithms based on regular cutting method are proposed to solve this problem. However, the dynamic property of clients and distributed property of the system make the existing algorithms unable to undertake it. In this paper, we propose a metric to measure the cutting quality and design a distributed mapping algorithm based on an irregular cutting. Simulation results show that our algorithm greatly decreases the costs of DVE systems for migration of clients and effectively improves the satisfaction degree of users.

关键词： Defense industry Virtual environment Computer industry Computer science Algorithm design and analysis Delay Partitioning algorithms Industrial training Computational intelligence Application software

来源：评论

学校读者我要写书评

暂无评论

Sparse Matrix Reordering Method Selection with parallel Computing and Deep Learning

Sparse Matrix Reordering Method Selection with Parallel Comp...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Rui Xia Jihu Guo Huajian Zhang Shun Yang Qinglin Wang Jie Liu College of Computer Science and Techonology National University of Defense Technology Changsha China Laboratory of Digitizing Software for Frontier Equipment National University of Defense Technology Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Sparse matrix reordering is an important step in Cholesky decomposition. By reordering the rows and columns of the matrix, the time of computation and storage cost can be greatly reduced. With the proposal of various reordering algorithms, the selection of suitable reordering methods for various matrices has become an important research topic. In this paper, we propose a method to predict the optimal reordering method by visualizing sparse matrices in chunks in a parallel manner and feeding them into a deep convolutional neural network. The results show that the theoretical performance can reach 95% of the optimal performance, the prediction accuracy of the method can reach up to 85%, the parallel framework achieves an average speedup ratio of 11.35 times over the serial framework, and the performance is greatly improved compared with the traversal selection method on large sparse matrices.

关键词： Deep learning Visualization Accuracy Costs Neural networks parallel processing Prediction algorithms

来源：评论

学校读者我要写书评

暂无评论

DMSS: A Dynamic Memory Scheduling System in Server Consolidation Environments

DMSS: A Dynamic Memory Scheduling System in Server Consolida...

引用

IEEE International Symposium on Object/Component/Service-Oriented Real-Time distributed Computing Workshops (ISORCW)

作者： Lanzheng Liu Rui Chu Yongchun Zhu Pengfei Zhang Liufeng Wang National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China Systems and Technology Group IBM China Beijing China

As the foundation of cloud computing, Server consolidation allows multiple computer infrastructures running as virtual machines in a single physical node. It improves the utilization of most kinds of resource but memory under x86 architecture. Because of inaccurate memory usage estimate and the lack of memory resource management, there is much service performance degradation in data centers, even though they have occupied a large amount of memory. Furthermore, memory becomes insufficient for a physical server when a lot of virtual machines depend on it. In order to improve this, we present a dynamic memory scheduling system called DMSS, which can manage memory resources in server consolidation environments and allocate memory among virtual machines on demand. We have designed and implemented the corresponding memory scheduling policy based on Xen virtualization platform to enhance memory efficiency and achieve service level agreement. The benchmark shows that DMSS can make an accurate and rapid response to memory changes and save more than 30% physical memory with less than 5% performance degradation. DMSS actually brings in economic benefits to cloud service providers because more virtual machines can be accommodated at lower costs.

关键词： Memory management Virtual machine monitors Monitoring Resource management Servers Dynamic scheduling Degradation

来源：评论

学校读者我要写书评

暂无评论

Reduced complexity tree-structured conditional random field for all-words word sense disambiguation

引用

Journal of Computational Information Systems 2012年第3期8卷 1177-1186页

作者： Zhou, Yun Wang, Ting Wang, Zhiyuan Xie, Songxian Computer School National University of Defense Technology Changsha 410073 China National Key Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha 410073 China

In this paper, we apply tree-structured conditional random field (TCRF) to all-words word sense disambiguation (WSD), where the graphical structure of TCRF is the dependency syntax tree produced by Minipar. The extremely large label space and high time complexity of TCRF hinders its application in semantic parsing, especially in all-words WSD, and we focus on reducing the time complexity of TCRF in this paper. The running time of TCRF is reduced significantly by edge feature selection and parallelization in training phase and beam search strategy in decoding phase. We evaluate the TCRF method on the English all-words task of Senseval-3, and the F1 value of TCRF method is 0.668, which outperforms the best result 0.652 in Senseval-3. For convenience of comparison, our training set only includes a part of Semcor, which reserves the space for improvement. Furthermore, the practical problem in the fields of information retrieval and natural language processing, often has a large label space and tree-structured information, which can also be solved by the TCRF method proposed in this paper. 1553-9105/Copyright © 2012 Binary Information Press.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：