检索结果-内蒙古大学图书馆

作者： Hou, Fu Mao, Xinjun College of Computer National University of Defense Technology Changsha410073 China National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha410073 China

With the rapid development and application of cloud computing, there exist plenty of clouds that are distributed on the open Internet, decentralized in the management, evolving with various services providing diverse functionalities and QoS. Moreover, because of the potential correlativity of cloud services and the dynamic of tenants' requirements, these services in multiple clouds are expected to be effectively managed in an autonomic and transparent way so that tenants can obtain continuous and efficient services. To this end, this paper proposes an approach based on self-organizing multi-agent system to achieving cross-clouds services management, including the service provision at the tenant-end and services aggregation at the cloud-end. In this approach, the clouds services are managed by a series of autonomous agents that are capable of autonomously accessing managed services. They can interact with each other to obtain macro-level services aggregation in terms of self-organization to adapt to the changes of both tenants' requirements and cloud services. The paper details the architecture, mechanisms, and algorithms to implement the aggregation and provision of services in cross-clouds. We also develop relevant cross-clouds services management platform called as CCloudMan, with which several experiments based on the public data sets have been conducted and the experimental results show the efficiency and usability of our proposed approach. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

关键词： Multi agent systems

来源：评论

学校读者我要写书评

暂无评论

Resource Allocation with Dynamic Substrate Network in Data Centre Networks

引用

China Communications 2013年第9期10卷 130-142页

作者：李小玲王怀民丁博李小勇刘惠 National Key Laboratory for Parallel and Distributed Processing School of Computer ScienceNational University of Defense Technology

The resource allocation problem in data centre networks refers to a map of the workloads provided by the cloud users/tenants to the Substrate Network(SN)which are provided by the cloud *** studies consider the dynamic arrival and departure of the workloads,while the dynamics of the substrate are *** this paper,we first propose the resource allocation with the dynamic SN,and denote it as ***,we propose an efficient mapping algorithm for *** performance of the proposed algorithm is evaluated by performing simulation *** results show that the proposed algorithm can effectively solve the GraphMap-DS.

关键词： resource allocation data centrenetwork workload substrate algorithm

来源：评论

学校读者我要写书评

暂无评论

Galaxyfly: A novel family of flexible-radix low-diameter topologies for large-scales interconnection networks 16

Galaxyfly: A novel family of flexible-radix low-diameter top...

引用

30th International Conference on Supercomputing, ICS 2016

作者： Lei, Fei Dong, Dezun Liao, Xiangke Su, Xing Li, Cunlu National Laboratory for Parallel and Distributed Processing Collaborative Innovation Center of High Performance Computing College of Computer National University of Defense Technology Changsha410073 China

ISBN: (纸本)9781450343619

Interconnection network plays an essential role in the architecture of large-scale high performance computing (HPC) systems. In the paper, we construct a novel family of lowdiameter topologies, Galaxyfly, using techniques of algebraic graphs over finite fields. Galaxyfly is guaranteed to retain a small constant diameter while achieving a flexible tradeoff between network scale and bisection bandwidth. Galaxyfly lowers the demands for high radix of network routers and is able to utilize routers with merely moderate radix to build exascale interconnection networks. We present effective congestion-aware routing algorithms for Galaxyfly by exploring its algebraic property. We conduct extensive simulations and analysis to evaluate the performance, cost and power consumption of Galaxyfly against state-of-the-art topologies. The results show that our design achieves better performance than most existing topologies under various routing algorithms and traffic patterns, and is cost-effective to deploy for exascale HPC systems. © 2016 ACM.

关键词： Topology

来源：评论

学校读者我要写书评

暂无评论

SAWSDL service discovery based on Fine-Grained data semantics

SAWSDL service discovery based on Fine-Grained data semantic...

引用

作者： Wei, Dengping Wang, Ting Wang, Ji School of Computer National University of Defense Technology 410073 Changsha China National Laboratory for Parallel and Distributed Processing 410073 Changsha China

With the aim to improve the effectiveness of SAWSDL service discovery, this paper proposes a novel discovery method for SAWSDL services, which is based on the matchmaking of so-called finegrained data semantics that is defined via sets of atomic elements with built-in data types. The fine-grained data semantics can be obtained by a transformation algorithm that decomposes parameters at message level into a set of atomic elements, considering the characteristics of SAWSDL service structure and semantic annotations. Then, a matchmaking algorithm is proposed for the matching of fine-grained data semantics, which avoids the complex and expensive structural matching at the message level. The fine-grained data semantics is transparent to the specific data structure of message-level parameters, therefore, it can help to match successfully similar Web services with different data structures of parameters. Moreover, a comprehensive measure is proposed by considering together several important components of SAWSDL service descriptions at the same time. Finally, this method is evaluated on SAWSDL service discovery test collection SAWSDL-TC2 and compared with other SAWSDL matchmakers. The experimental results show that our method can improve the effectiveness of SAWSDL service discovery with low average query response time. The results imply that fine-grained parameters fit to represent the data semantics of SAWSDL services, especially when data structures of parameters are not important for semantics. © 2011 The Institute of Electronics, Information and Communication Engineers.

关键词： Data structures

来源：评论

学校读者我要写书评

暂无评论

Communication Analysis for Multidimensional parallel Training of Large-scale DNN Models 25

Communication Analysis for Multidimensional Parallel Trainin...

引用

25th IEEE International Conferences on High Performance Computing and Communications, 9th International Conference on Data Science and Systems, 21st IEEE International Conference on Smart City and 9th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC/DSS/SmartCity/DependSys 2023

作者： Lai, Zhiquan Hao, Yanqi Li, Shengwei Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

Multidimensional parallel training has been widely applied to train large-scale deep learning models like GPT-3. The efficiency of parameter communication among training devices/processes is often the performance bottleneck of large model training. Analysis of parameter communication mode and traffic has important reference significance for the research of interconnection network design and computing task scheduling to improve the training performance. In this paper, we analyze the parametric communication modes in typical 3D parallel training (data parallelism, pipeline parallelism, and tensor parallelism), and model the traffic in different communication modes. Finally, taking GPT-3 as an example, we present the communication in its 3D parallel training. © 2023 IEEE.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Coordinate strip-mining and kernel fusion to lower power consumption on GPU

Coordinate strip-mining and kernel fusion to lower power con...

引用

作者： Wang, Guibin National Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha Hunan China

ISBN: (纸本)9783981080179

Although general purpose GPUs have relatively high computing capacity, they also introduce high power consumption compared with general purpose CPUs. Therefore low-power techniques targeted for GPUs will be one of the most hot topics in the future. On the other hand, in several application domains, users are unwilling to sacrifice performance to save power. In this paper, we propose an effective kernel fusion method to reduce the power consumption for GPUs without performance loss. Different from executing multiple kernels serially, the proposed method fuses several kernels into one larger kernel. Owing to the fact that most consecutive kernels in an application have data dependency and could not be fused directly, we split large kernel into multiple slices with strip-mining method, then fuse independent sliced kernels into one kernel. Based on the CUDA programming model, we propose three different kernel fusion implementations, with each one targeting for a special case. Based on the different strip-ming methods, we also propose two fusion mechanisms, which are called invariant-slice fusion and variant-slice fusion. The latter one could be better adapted to the requirements of the kernels to be fused. The experimental results validate that the proposed kernel fusion method could effectively reduce the power consumption for GPU. © 2011 EDAA.

关键词： Electric power utilization

来源：评论

学校读者我要写书评

暂无评论

Funnel: An Efficient Sparse Attention Accelerator with Multi-Dataflow Fusion 22

Funnel: An Efficient Sparse Attention Accelerator with Multi...

引用

22nd IEEE International Symposium on parallel and distributed processing with Applications, ISPA 2024

作者： Ma, Shenghong Xu, Jinwei Jiang, Jingfei Wang, Yaohua Li, Dongsheng National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing College of Computer Changsha China

ISBN: (纸本)9798331509712

The self-attention mechanism is the core component of Transformer, which provides a powerful ability to understand the sequence context. However, the self-attention mechanism also suffers from a large amount of redundant computation. Model sparsification can effectively reduce computational load, but the irregularity of non-zeros introduced by sparsification significantly decreases hardware efficiency. This paper proposes Funnel, an accelerator that dynamically predicts sparse attention patterns and efficiently processes unstructured sparse data. Firstly, we adopt a fast quantization method based on lookup table to minimize the cost of sparse patterns prediction. Secondly, we propose Funnel Computing Unit (FCU), a hardware architecture that efficiently handles sparse attention through multi-dataflow fusion. Sampled Dense-Dense Matrix Multiplication (SDDMM) and Sparse-Dense Matrix Multiplication (SpMM) are core components of sparse attention mechanism. FCU unifies the computation ways of matrix inner product and row-wise product to support SDDMM and SpMM at the same time, which greatly reduces the storage and movement overhead of intermediate results. Lastly, we devise a lightweight buffer and data tiling strategy tailored to the proposed accelerator, aimed at enhancing data reuse. Experiments demonstrate that our accelerator achieves 0.10-0.25 sparsity with small accuracy loss. When computing the self-attention layer, it attains hardware efficiency ranging from 60% to 85%. Compared to CPU and GPU, it achieves 5.60x and 8.20x speedup. Compared to the state-of-the-art attention accelerators A3, SpAtten, FTRANS, and Sanger, it achieves 7.37x, 4.52x, 9.58x, and 3.08x speedup. © 2024 IEEE.

关键词： FPGA Funnel computing unit Sparse attention Transformer

来源：评论

学校读者我要写书评

暂无评论

Implementation of encrypted data for outsourced database

Implementation of encrypted data for outsourced database

引用

2010 2nd International Conference on Computational Intelligence and Natural Computing, CINC 2010

作者： Wang, Zheng-Fei Tang, Ai-Guo Department of Computer Hunan Business College Changsha 410205 China National Key Laboratory for Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha 410073 China

ISBN: (纸本)9781424477036

Encryption technology has become an important mechanism of securing data stored in the outsourced database. However, it is a difficulty to query efficiently the encrypted data and many researchers take it into consideration. To solve the problem, an encrypted schema, based on the Postgresql DBMS, is proposed Through the security dictionary and the extended SQL, the approach implements the encrypted storage and efficiently query over the encrypted data in the outsourced databases. Results of experiments validate the efficiency and feasibility of our approach. ©2010 IEEE.

关键词： Cryptography

来源：评论

学校读者我要写书评

暂无评论

Efficient Large Models Fine-tuning on Commodity Servers via Memory-balanced Pipeline parallelism 25

Efficient Large Models Fine-tuning on Commodity Servers via ...

引用

作者： Liu, Yujie Lai, Zhiquan Liu, Weijie Wang, Wei Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

Large models have achieved impressive performance in many downstream tasks. Using pipeline parallelism to fine-tune large models on commodity GPU servers is an important way to make the excellent performance of large models available to the general public. Previous solutions fail to achieve an efficient memory-balanced pipeline parallelism. In this poster, we introduce a memory load-balanced pipeline parallel solution. This solution balances memory consumption across stages on commodity GPU servers via NVLink bridges. It establishes a new pathway to offload data from GPU to CPU by using the PCIe link of adjacent GPUs connected by the NVLink bridge. Furthermore, our method orchestrates offload operations to minimize the offload latency during large model fine-tuning. Experiments demonstrate that our solution can balance the memory footprint among pipeline stages without sacrificing training performance. © 2023 IEEE.

关键词： Program processors

来源：评论

学校读者我要写书评

暂无评论

Rethinking the distributed DNN Training Cluster Design from the Cost-effectiveness View 25

Rethinking the Distributed DNN Training Cluster Design from ...

引用

作者： Lai, Zhiquan Liu, Yujie Wang, Wei Hao, Yanqi Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

As deep learning grows rapidly, model training heavily relies on parallel methods and there exist numerous cluster configurations. However, current preferences for parallel training focus on data centers, overlooking the financial constraints faced by most researchers. To attain the best performance within the cost limitation, we introduce a throughput-cost metric to accurately characterize clusters' cost-effectiveness. Based on this metric, we design a cost-effective cluster featuring the 3090 with NVLink. The experiment results demonstrate that our cluster achieves remarkable cost-effectiveness in various distributed model training schemes. © 2023 IEEE.

关键词： Cost effectiveness

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：