检索结果-内蒙古大学图书馆

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Wei Jiang Rui Xu Sheng Ma Qiong Wang Xiang Hou Hongyi Lu Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

The correctness and robustness of the neural network model are usually proportional to its depth and width. Currently, the neural network models become deeper and wider to cope with complex applications, which leads to high memory capacity requirement and computer capacity requirements of the training process. The multi-accelerator parallelism is a promising choice for the two challenges, which deploys multiple accelerators in parallel for training neural networks. Among them, the pipeline parallel scheme has a great advantage in training speed, but its memory capacity requirements are relatively higher than other parallel schemes. Aiming at solving this challenge of pipeline parallel scheme, we propose a data transfer mechanism, which effectively reduces the peak memory usage of the training process by real-time data transferring. In the experiment, we implement our design and apply it to Pipedream, a mature pipeline parallel scheme. The memory requirement of training process is reduced by up to 48.5%, and the speed loss is kept within a reasonable range.

关键词： Training Computational modeling Pipelines Neural networks Memory management Bandwidth parallel processing

来源：评论

学校读者我要写书评

暂无评论

Co-designing the Topology/Algorithm to Accelerate distributed Training

Co-designing the Topology/Algorithm to Accelerate Distribute...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Xiang Hou Rui Xu Sheng Ma Qiong Wang Wei Jiang Hongyi Lu Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

With the development of Deep Learning (DL), Deep Neural Network (DNN) models have become more complex. At the same time, the development of the Internet makes it easy to obtain large data sets for DL training. Large-scale model parameters and training data enhance the level of AI by improving the accuracy of DNN models. But on the other hand, they also present more severe challenges to the hardware training platform because training a large model needs a lot of computing and memory resources that can easily exceed the capacity of a single processor. In this context, integrating more processors on a hierarchical system to conduct distributed training is a direction for the development of training platforms. In distributed training, collective communication operations (including all-to-all, all-reduce, and all-gather) take up a lot of training time, making the interconnection network between computing nodes one of the most critical factors affecting the system performance. The hierarchical torus topology, combined with the Ring All-Reduce collective communication algorithm, is one of the current mainstream distributed interconnection networks. However, we believe that its communication performance is not the best. In this work, we first designed a new intra-package communication topology, i.e. the switch-based fully connected topology, which shortens the time consumed by cross-node communication. Then, considering the characteristics of this topology, we carefully devised more efficient all-reduce and all-gather communication algorithms. Finally, combined with the torus topology, we implemented a novel distributed DL training platform. Compared with the hierarchical torus, our platform improves communication efficiency and provides 1.16-2.68 times speedup in distributed training of DNN models.

关键词： Training Deep learning Network topology Multiprocessor interconnection Computational modeling Training data Switches

来源：评论

学校读者我要写书评

暂无评论

Surrogate Supervision-based Deep Weakly-supervised Anomaly Detection

Surrogate Supervision-based Deep Weakly-supervised Anomaly D...

引用

IEEE International Conference on Data Mining Workshops (ICDM Workshops)

作者： Zhiyue Wu Hongzuo Xu Yijie Wang Yongjun Wang Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

ISBN: (纸本)9781665424288

Many anomaly detection applications can provide partially observed anomalies, but only limited work is for this setting. Additionally, a number of anomaly detectors focus on learning a particular model of normal/abnormal class. However, the intra-class model might be too complicated to be accurately learned. It is still a non-trivial task to handle data with anomalies/inliers in skewed and heterogeneous distributions. To address these problems, this paper proposes an anomaly detection method to leverage Partially Labeled anomalies via Surrogate supervision-based Deviation learning (denominated PLSD). The original supervision (i.e., known anomalies and a set of explored inliers) is transferred to semantic-rich surrogate supervision signals (i.e., anomaly-inlier and inlier-inlier class) via vector concatenation. Then different relationships and interactions between anomalies and inliers are directly and efficiently learned thanks to the neural network’s connection property. Anomaly scoring is processed via the trained network and the high-efficacy inliers. Extensive experiments show that PLSD significantly prevails state-of-the-art semi/weakly-supervised anomaly detectors.

关键词： Learning systems Conferences Detectors Gaussian distribution Benchmark testing Task analysis Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

Word Embedding-based Context-sensitive Network Flow Payload Anomaly Detection

Word Embedding-based Context-sensitive Network Flow Payload ...

引用

International Conference on Applied Machine Learning (ICAML)

作者： Yizhou Li Yijie Wang Li Cheng Hongzuo Xu Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

ISBN: (纸本)9781665421263

Payload anomaly detection can discover malicious behaviors hidden in network packets. It is hard to handle payload due to its various possible characters and complex semantic context, and thus identifying abnormal payload is also a non-trivial task. Prior art only uses the n-gram language model to extract features, which directly leads to ultra-high-dimensional feature space and also fails to capture the context semantics fully. Accordingly, this paper proposes a word embedding-based context-sensitive network flow payload anomaly detection method (termed WECAD). First, WECAD obtains the initial feature representation of the payload through the word embedding-based method. Then, we propose a corpus pruning algorithm, which applies the cosine similarity clustering and frequency distribution to prune inconsequential characters. We only keep the essential characters to reduce the calculation space. Subsequently, we propose a context learning algorithm. It employs the co-occurrence matrix transformation technology and introduces the backward step size to consider the order relationship of essential characters. Comprehensive experiments on real-world intrusion detection datasets validate the effectiveness of our method.

关键词： Art Semantics Clustering algorithms Intrusion detection Machine learning Feature extraction Task analysis

来源：评论

学校读者我要写书评

暂无评论

Fden: Mining Effective Information of Features in Detecting Network Anomalies

Fden: Mining Effective Information of Features in Detecting ...

引用

IEEE International Conference on Acoustics, Speech and Signal processing

作者： Bin Li Yijie Wang Mingyu Liu Kele Xu Zhongyang Wang Li Cheng Yizhou Li Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology Changsha China

Network anomaly detection is important for detecting and reacting to the presence of network attacks. In this paper, we propose a novel method to effectively leverage the features in detecting network anomalies, named FDEn, consisting of flow-based Feature Derivation (FD) and prior knowledge incorporated Ensemble models (En pk). To mine the effective information in features, 149 features are derived to enrich the feature set of the original data with covering more characteristics of network traffic. To leverage these features effectively, an ensemble model En pk, including CatBoost and XGBoost, based on the bagging strategy is proposed to first detect anomalies by combining numerical features and categorical features. And then, En pk adjusts the predicted label of specific data by incorporating the prior knowledge of network security. We conduct empirically experiments on the data set provided by the Network Anomaly Detection Challenge (NADC), in which we obtain average improvement up to 61.6%, 31.7%, 50.2%, and 45.0%, in terms of the cost score, precision, recall and F1-score, respectively.

关键词： Knowledge engineering Conferences Telecommunication traffic Signal processing Feature extraction Numerical models Security

来源：评论

学校读者我要写书评

暂无评论

A Complexity-Reduced Block-Selective Algebraic Multigrid Method for Implicitly Coupled Velocity-Pressure System on High Performance computers

SSRN

引用

SSRN 2023年

作者： Liang, Yuechao Guo, Xiao-Wei Li, Chao Yuan, Fan Song, Min Zhang, Qingyang Chen, Xinhai Liu, Jie Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China Laboratory of Digitizing Software for Frontier Equipment National University of Defense Technology Changsha410073 China Institute for Quantum Information State Key Laboratory of High Performance Computing College of Computer Science and Technology National University of Defense Technology Changsha410073 China School of Mathematics and Computational Sciences Xiangtan University Xiangtan411105 China College of Computer Science and Technology National University of Defense Technology Changsha410073 China

The implicitly coupled pressure-based algorithm is widely acknowledged for its superior convergence and robustness in solving incompressible flow problems. However, the increased expansion scale of equations and difficulty in solving the resulting equation system has limited the widespread use of the coupling algorithm. This paper presents an optimized block-SAMG method, referred to as Red-SAMG, to significantly reduce computational complexity and memory consumption. Our approach employs a parallel modified independent set algorithm, allowing each process to perform matrix coarsening individually. An aggressive coarsening strategy further reduces complexity to address larger-scale problems. We conducted numerical experiments on a serial two-dimensional backward-facing step case and a parallel three-dimensional tunnel between rod bundles in the reactor core problem. The results demonstrate that the Red-SAMG method with aggressive coarsening reduces solution time by 12\% on the backward-facing step case. For large-scale three-dimensional problems under the same parallel conditions, the Red-SAMG method shortens the solution time by 14\% compared to the existing SAMG method. The Red-SAMG method with aggressive coarsening further reduces solution time by 49\%. Our optimized Red-SAMG method effectively reduces the complexity of the linear system arising from the implicitly coupled algorithm. The aggressive coarsening strategy also represents a significant advancement in improving the computational efficiency of the SAMG method. © 2023, The Authors. All rights reserved.

关键词： Coarsening

来源：评论

学校读者我要写书评

暂无评论

Deep learning for social network information cascade analysis: A survey 5

Deep learning for social network information cascade analysi...

引用

5th IEEE International Conference on Data science in Cyberspace, DSC 2020

作者： Gao, Liqun Zhou, Bin Jia, Yan Tu, Hongkui Wang, Ye Chen, Chenguang Wang, Haiyang Zhuang, Hongwu National University of Defense Technology College of Computer ChangSha China National University of Defense Technology Science and Technology on Parallel and Distributed Processing Laboratory ChangSha China

ISBN: (纸本)9781728195582

The phenomenon of information dissemination in social networks is widespread, and Social Network Information Cascade Analysis (SNICA) aims to acquire valuable knowledge in the process of information dissemination in social networks. As the number, volume, and resolution of social network data increase rapidly, traditional social network data analysis methods, especially the analysis method of social network graph (SNG) data becoming overwhelmed in SNICA. Recently, deep learning models have changed this situation, and it has achieved success in SNICA with its powerful implicit feature extraction capabilities. In this paper, we provide a comprehensive survey of recent progress in applying deep learning techniques for SNICA. We first introduce related concepts and summarize the advantages of deep learning technology in SNICA. Then, we propose a general framework for deep learning technology, which applies to SNICA. Then, different SNICA application scenarios in the framework are classified and discussed according to user behavior analysis, information cascade analysis, rumor detection, and social network event analysis. Finally, we discuss the limitations of current work and suggest future directions. © 2020 IEEE.

关键词： Data mining

来源：评论

学校读者我要写书评

暂无评论

A knowledge graph embedding method based on neural network 5

A knowledge graph embedding method based on neural network

引用

5th IEEE International Conference on Data science in Cyberspace, DSC 2020

作者： Li, Chenchen Li, Aiping Tu, Hongkui Wang, Ye Wang, Changhai National University of Defense Technology College of Computer ChangSha China National University of Defense Technology College of Computer Science and Technology on Parallel and Distributed Processing Laboratory ChangSha China

ISBN: (数字)9781728195582

ISBN: (纸本)9781728195582

As the basis of many knowledge graph completion tasks, the embedding representation of entities and relations in knowledge graph (KG) is an important task in the fields of Natural Language processing (NLP) and Artificial Intelligence (AI). While most of the existing knowledge graph embedding (KGE) models based on convolutional neural network (CNN) can obtain abundant feature embedding, they may ignore an important fact that the triples in the KG come from the text, as they simply learn about the feature embedding of entities and relations without considering contextual information. Therefore, in this paper, we propose an effective KGE model based on neural network. First of all, we convert the triple (h, r, t) of the KG into a sentence [h r t]. Then, the LSTM neural network is used to learn the long-term dependence of sentences from the input feature vectors. Then, on this basis, the two-layer convolutional neural network with several different filters is used to extract different local features. Finally, the obtained feature vectors are connected together, and the inner product is carried out with the weight vectors to obtain the score of the triple, so as to judge the validity of the given triple. We evaluate our model on two benchmark datasets FB15k-237 and WN18RR, the experimental results show that the model can effectively improve the accuracy of link prediction, achieving better results compared with other baseline models. © 2020 IEEE.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

A survey on approaches and applications of knowledge representation learning 5

A survey on approaches and applications of knowledge represe...

引用

5th IEEE International Conference on Data science in Cyberspace, DSC 2020

作者： Li, Chenchen Li, Aiping Wang, Ye Tu, Hongkui Song, Yichen National University of Defense Technology College of Computer ChangSha China National University of Defense Technology College of Computer Science and Technology on Parallel and Distributed Processing Laboratory ChangSha China

ISBN: (纸本)9781728195582

Knowledge representation learning (KRL) is one of the important research topics in artificial intelligence and Natural language processing. It can efficiently calculate the semantics of entities and relations in a low-dimensional space, and effectively solve the problem of data sparsity, which significantly improve the performance of knowledge acquisition, fusion and reasoning and so on. Starting from the five perspectives of distance-based, semantic matching, bilinear-based, neural network model and additional information model, this paper first introduces the overall framework and specific model design, and then correspondingly introduces the experimental evaluation tasks, metrics and benchmark datasets of each model. On this basis, how to apply KRL to various downstream tasks is summarized. © 2020 IEEE.

关键词： Knowledge representation

来源：评论

学校读者我要写书评

暂无评论

DRAM failure prediction in AIOps: Empirical evaluation, challenges and opportunities

arXiv

引用

arXiv 2021年

作者： Wu, Zhiyue Xu, Hongzuo Pang, Guansong Yu, Fengyuan Wang, Yijie Jian, Songlei Wang, Yongjun Science and Technology on Parallel and Distributed Processing Laboratory College of Computer National University of Defense Technology China University of Adelaide Australia

DRAM failure prediction is a vital task in AIOps, which is crucial to maintain the reliability and sustainable service of large-scale data centers. However, limited work has been done on DRAM failure prediction mainly due to the lack of public available datasets. This paper presents a comprehensive empirical evaluation of diverse machine learning techniques for DRAM failure prediction using a large-scale multi-source dataset, including more than three millions of records of kernel, address, and mcelog data, provided by Alibaba Cloud through PAKDD 2021 competition. Particularly, we first formulate the problem as a multi-class classification task and exhaustively evaluate seven popular/state-of-the-art classifiers on both the individual and multiple data sources. We then formulate the problem as an unsupervised anomaly detection task and evaluate three state-of-the-art anomaly detectors. Further, based on the empirical results and our experience of attending this competition, we discuss major challenges and present future research opportunities in this task. © 2021, CC BY.

关键词： Web services

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：