检索结果-内蒙古大学图书馆

OHTMA:an optimized heuristic topology-aware mapping algorithm on the Tianhe-3 exascale supercomputer prototype

Frontiers of Information Technology & Electronic Engineering 2020年第6期21卷 939-949页

作者： Yi-shui LI Xin-hai CHEN Jie LIU Bo YANG Chun-ye GONG Xin-biao GAN Sheng-guo LI Han XU Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense TechnologyChangsha 410073China

With the rapid increase of the size of applications and the complexity of the supercomputer architecture,topology-aware process mapping becomes increasingly *** communication cost has become a dominant constraint of the performance of applications running on the *** avoid a bad mapping strategy which can lead to terrible communication performance,we propose an optimized heuristic topology-aware mapping algorithm(OHTMA).The algorithm attempts to minimize the hop-byte metric that we use to measure the mapping *** incorporates a new greedy heuristic method and pair-exchange-based *** reduces the number of long-distance communications and effectively enhances the locality of the *** results on the Tianhe-3 exascale supercomputer prototype indicate that OHTMA can significantly reduce the communication costs.

关键词： High-performance computing Topology mapping Heuristic algorithm

来源：评论

学校读者我要写书评

暂无评论

Effective Anomaly Detection Based on Reinforcement Learning in Network Traffic Data

Effective Anomaly Detection Based on Reinforcement Learning ...

引用

International Conference on parallel and distributed Systems (ICPADS)

作者： Zhongyang Wang Yijie Wang Hongzuo Xu Yongjun Wang Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology

Mixed-type data with both categorical and numerical features are ubiquitous in network security, but the existing methods are minimal to deal with them. Existing methods usually process mixed-type data through feature conversion, whereas their performance is downgraded by information loss and noise caused by the transformation. Meanwhile, existing methods usually superimpose domain knowledge and machine learning in which fixed thresholds are used. It cannot dynamically adjust the anomaly threshold to the actual scenario, resulting in inaccurate anomalies obtained, which results in poor performance. To address these issues, this paper proposes a novel Anomaly Detection method based on Reinforcement Learning, termed ADRL, which uses reinforcement learning to dynamically search for thresholds and accurately obtain anomaly candidate sets, fusing domain knowledge and machine learning fully and promoting each other. Specifically, ADRL uses prior domain knowledge to label known anomalies and uses entropy and deep autoencoder in the categorical and numerical feature spaces, respectively, to obtain anomaly scores combining with known anomaly information, which are integrated to get the overall anomaly scores via a dynamic integration strategy. To obtain accurate anomaly candidate sets, ADRL uses reinforcement learning to search for the best threshold. Detailedly, it initializes the anomaly threshold to get the initial anomaly candidate set and carries on the frequent rule mining to the anomaly candidate set to form the new knowledge. Then, ADRL uses the obtained knowledge to adjust the anomaly score and get the score modification rate. According to the modification rate, different threshold modification strategies are executed, and the best threshold, that is, the threshold under the maximum modification rate, is finally obtained, and the modified anomaly scores are obtained. The scores are used to re-carry out machine learning to improve the algorithm's accuracy for anomalo

关键词： Knowledge engineering Machine learning algorithms Conferences distributed databases Reinforcement learning Telecommunication traffic Network security

来源：评论

学校读者我要写书评

暂无评论

Entropy and Autoencoder-Based Outlier Detection in Mixed-Type Network Traffic Data

Entropy and Autoencoder-Based Outlier Detection in Mixed-Typ...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Zhongyang Wang Yijie Wang Zhenyu Huang Yongjun Wang Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology

Mixed-type data containing categorical and numerical features are pervasive in real life, but very limited outlier detection methods are available for these data. Some existing methods handle mixed-type data by feature converting, whereas their performance is downgraded by information loss and noise caused by the transformation. Meanwhile, the existing general algorithms cannot combine the characteristics of outliers in specific fields, leading to an unsatisfying performance in actual scenarios, such as the field of network security. This paper proposes a novel Entropy and Autoencoder-based Outlier Detection in mixed-type network traffic data, termed EAOD, which combines characteristics of outliers in specific fields and machine learning models to detect outliers. EAOD utilizes the expert rules made by domain knowledge summarized based on characteristics of existing outliers to label known outlier data. It performs holoentropy and deep autoencoder for the category and numerical feature spaces, respectively, in unlabeled data to obtain outlier scores integrated to get the final outlier scores via a dynamic integration strategy. Especially in the numerical feature space, to fully mine known outlier behavior patterns, deep autoencoders of outlier and normal types are constructed separately to capture unknown outliers jointly. Experiments show that EAOD significantly outperforms eight state-of-the-art outlier detectors on seven real network traffic datasets.

关键词： Telecommunication traffic Machine learning Detectors Network security Feature extraction Entropy Data models

来源：评论

学校读者我要写书评

暂无评论

Optimizing All-to-All Collective Communication on Tianhe Supercomputer

Optimizing All-to-All Collective Communication on Tianhe Sup...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Jintao Peng Jie Liu Yi Dai Min Xie Chunye Gong National University of Defense Technology ChangSha China Science and Technology on Parallel and Distributed Processing Laboratory NUDT Changsha China Changsha and Laboratory of Software Engineering for Complex System NUDT Changsha China

All-to-all communication has a wide range of ap-plications in parallel applications like FFT. On most supercom-puters, there are multiple cores in a node. Message aggregation is an efficient method for smaller messages. Using multi-leader to aggregate message show significant improvement in intra-node overhead. However, compared to one-leader aggregation, existing multi-leader design incurs more message count and less aggregated message size. This paper proposes an Overlapped Multi-worker Multi-port all-to-all (OVALL) to scale the message size and parallelism of the aggregation algorithm. The algorithm exploits the all-to-all multi-core parallelism, concurrent commu-nication, and overlapping capabilities. Experiment results show that, compared to systems built-in MPI, OVALL'implementation achieves up to 5.9x or 18x speedup compared to MPI on different HPC systems. For the Fast Fourier Transform (FFT) application, OVALL is up to 2.7x (8192 cores, system A) or 5.6x (4800 cores, system B) faster compared to built-in MPI on peak performance.

关键词： Multicore processing Fast Fourier transforms Scalability Aggregates Supercomputers Libraries Pipeline processing

来源：评论

学校读者我要写书评

暂无评论

Attention-based Encoder-Decoder Recurrent Neural Networks for HTTP Payload Anomaly Detection

Attention-based Encoder-Decoder Recurrent Neural Networks fo...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Shang Wu Yijie Wang Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology

Attack payloads are often short segments hidden in HTTP requests; thus they can be found by HTTP payload anomaly detection. Deep learning models learn data features during training without manual feature extraction, and better performance has received more attention. Recurrent Neural Network models process sequences directly, which are widely used in payload anomaly detection. However, due to the gradient vanishing problem, RNN has limits on processing the long sequences. Meanwhile, RNN uses its final hidden state for detection and pays more attention to the content of the end of the payload. Besides, deep learning generally lacks interpretability. The paper proposes an unsupervised deep learning model for HTTP payload Anomaly Detection, namely Attention-based Encoder-Decoder Recurrent Neural Networks Anomaly Detection model (AEDRAD). AEDRAD utilizes the encoder-decoder RNN and attention mechanism to detect anomalies by reconstructing the original sequences. AEDRAD filters the fields of HTTP protocol that cannot exist anomalies, focusing on the suspicious segments. Through the encoder-decoder network, the normal payload can be well-reconstructed while the anomaly payload fails. With the attention mechanism, AEDRAD generates practical features for further anomaly detection from a global perspective. Meanwhile, it marks abnormal fragments visually, which is conducive to a subsequent analysis by experts. The experimental results show that AEDRAD significantly outperforms three state-of-the-art unsupervised algorithms on two real datasets.

关键词： Deep learning Training Recurrent neural networks Protocols Focusing Manuals Feature extraction

来源：评论

学校读者我要写书评

暂无评论

parallelization of Fast Monte Carlo Dose Calculation for Radiotherapy Treatment Planning on the ARMv8 Architecture

Parallelization of Fast Monte Carlo Dose Calculation for Rad...

引用

International Conference on Information Science and Technology (ICIST)

作者： Chi Jin Qinglin Wang Yang Zhao Yong Dou Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha China

Monte Carlo (MC) simulation plays a key role in radiotherapy. Since the simulation time of the MC program cannot fully meet the clinical requirements, we use the ARM-based FT-2000+ multi-core processor for parallelization, which provides an effective solution for accelerating MC dose calculation. In this paper, we implement and verify FT-DPM, which is an OpenMP-based MC Dose Planning Method on FT-2000+. FT-DPM utilizes the parallelism of MC simulation and the advantage of ARM architecture to achieve the parallelization on ARM architecture. Meanwhile, we optimize the original DPM program in terms of memory allocation, data structure and data type. The experiments show that, compared with the original DPM code, FT-DPM obtains very accurate results and reaches the maximum speedups of 155.94 times for the electron case. The parallel program based on FT-2000+ only takes 44.1 seconds to simulate the particle transport of 100 million times, showing good clinical application potential. In addition, the speedup and efficiency of FT-DPM running on different core counts are also discussed.

关键词： Monte Carlo methods Multicore processing Simulation Scalability parallel processing Data structures Load management

来源：评论

学校读者我要写书评

暂无评论

Accelerate Graph Neural Network Training by Reusing Batch Data on GPUs

Accelerate Graph Neural Network Training by Reusing Batch Da...

引用

IEEE International Conference on Performance, Computing and Communications (IPCCC)

作者： Zhejiang Ran Zhiquan Lai Lizhi Zhang Dongsheng Li National Key Laboratory of Parallel and Distributed Processing School of Computer National University of Defense Technology Changsha China

ISBN: (纸本)9781665443326

With the increasing adoption of graph neural networks (GNNs) in the graph-based deep learning community, various graph programming frameworks and models have been developed to improve the productivity of GNNs. The current GNN frameworks choose GPU as an essential tool to accelerate GNN training. However, it is still challenging to train GNNs on large graphs with limited GPU memory. Unlike traditional neural networks, generating mini-batch data by sampling in GNNs requires some complicated tasks such as traversing the graph to select neighboring nodes and gathering their features. This process takes up most of the training and we find the main bottleneck comes from transferring nodes features from CPU to GPU through limited bandwidth. In this paper, We propose a method Reusing Batch Data for the problem of data transmission. This method utilizes the similarity between adjacent mini-batches to reduce repeated data transmission from CPU to GPU. Furthermore, to reduce the overhead introduced by this method, we design a fast algorithm based on GPU to detect repeated nodes’ data and achieve shorter additional computation time. Evaluations on three representative GNN models show that our method can reduce transmission time by up to 60% and speed the end-to-end GNN training by up to 1.79× over the state-ofthe-art baselines. Besides, Reusing Batch Data can effectively save GPU memory footprint by about 19% to 40% while still reducing the training time compared to the static cache strategy.

关键词： Training Productivity Deep learning Design methodology Conferences Graphics processing units Programming

来源：评论

学校读者我要写书评

暂无评论

Modeling Full Information with Graph Network for Joint Entity-Relation Extraction 21

Modeling Full Information with Graph Network for Joint Entit...

引用

Proceedings of the 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence

作者： Qian Wan Luona Wei Xinhai Chen Jie Liu Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology China College of Systems Engineering National University of Defense Technology China Laboratory of Software Engineering for Complex Systems National University of Defense Technology China

ISBN: (纸本)9781450385053

Fully capturing contextual information and analyzing the association between entity semantics and type is helpful for joint extraction task: 1) The context can reflect the part of speech and semantics of entity. 2) The entity type is closely related to the relation between entities. Previous research used to simply embed the contextual information into shallow layer of the model, ignoring the association between entity semantics and type. In this paper, we propose a graph network with full-information modeling to explicitly model different-level information in the text. The contextual information of entity is dynamically embedded in each span representation to improve the reasoning ability. To capture the fine-grained association between the semantics and type of entity, the graph network uses the feature of entity types to generate edge information between different nodes. Experimental results show that our model outperforms previous models on the CoNLL04 dataset and obtains competitive results on the SciERC dataset in both entity recognition and relation extraction. Extensive additional experiments further verify the effectiveness of the model.

关键词： Natural Language processing Named Entity Recognition Graph Network Deep Learning Relation Extraction

来源：评论

学校读者我要写书评

暂无评论

A Memory Saving Mechanism Based on Data Transferring for Pipeline parallelism

A Memory Saving Mechanism Based on Data Transferring for Pip...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Wei Jiang Rui Xu Sheng Ma Qiong Wang Xiang Hou Hongyi Lu Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

The correctness and robustness of the neural network model are usually proportional to its depth and width. Currently, the neural network models become deeper and wider to cope with complex applications, which leads to high memory capacity requirement and computer capacity requirements of the training process. The multi-accelerator parallelism is a promising choice for the two challenges, which deploys multiple accelerators in parallel for training neural networks. Among them, the pipeline parallel scheme has a great advantage in training speed, but its memory capacity requirements are relatively higher than other parallel schemes. Aiming at solving this challenge of pipeline parallel scheme, we propose a data transfer mechanism, which effectively reduces the peak memory usage of the training process by real-time data transferring. In the experiment, we implement our design and apply it to Pipedream, a mature pipeline parallel scheme. The memory requirement of training process is reduced by up to 48.5%, and the speed loss is kept within a reasonable range.

关键词： Training Computational modeling Pipelines Neural networks Memory management Bandwidth parallel processing

来源：评论

学校读者我要写书评

暂无评论

Co-designing the Topology/Algorithm to Accelerate distributed Training

Co-designing the Topology/Algorithm to Accelerate Distribute...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Xiang Hou Rui Xu Sheng Ma Qiong Wang Wei Jiang Hongyi Lu Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

With the development of Deep Learning (DL), Deep Neural Network (DNN) models have become more complex. At the same time, the development of the Internet makes it easy to obtain large data sets for DL training. Large-scale model parameters and training data enhance the level of AI by improving the accuracy of DNN models. But on the other hand, they also present more severe challenges to the hardware training platform because training a large model needs a lot of computing and memory resources that can easily exceed the capacity of a single processor. In this context, integrating more processors on a hierarchical system to conduct distributed training is a direction for the development of training platforms. In distributed training, collective communication operations (including all-to-all, all-reduce, and all-gather) take up a lot of training time, making the interconnection network between computing nodes one of the most critical factors affecting the system performance. The hierarchical torus topology, combined with the Ring All-Reduce collective communication algorithm, is one of the current mainstream distributed interconnection networks. However, we believe that its communication performance is not the best. In this work, we first designed a new intra-package communication topology, i.e. the switch-based fully connected topology, which shortens the time consumed by cross-node communication. Then, considering the characteristics of this topology, we carefully devised more efficient all-reduce and all-gather communication algorithms. Finally, combined with the torus topology, we implemented a novel distributed DL training platform. Compared with the hierarchical torus, our platform improves communication efficiency and provides 1.16-2.68 times speedup in distributed training of DNN models.

关键词： Training Deep learning Network topology Multiprocessor interconnection Computational modeling Training data Switches

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：