检索结果-内蒙古大学图书馆

IEEE International Conference on High Performance computing and Communications (HPCC)

作者： Zhiquan Lai Yanqi Hao Shengwei Li Dongsheng Li National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

Multidimensional parallel training has been widely applied to train large-scale deep learning models like GPT-3. The efficiency of parameter communication among training devices/processes is often the performance bottleneck of large model training. Analysis of parameter communication mode and traffic has important reference significance for the research of interconnection network design and computing task scheduling to improve the training performance. In this paper, we analyze the parametric communication modes in typical 3D parallel training (data parallelism, pipeline parallelism, and tensor parallelism), and model the traffic in different communication modes. Finally, taking GPT-3 as an example, we present the communication in its 3D parallel training.

关键词：

来源：评论

学校读者我要写书评

暂无评论

FedIERF: Federated Incremental Extremely Random Forest for Wearable Health Monitoring

引用

Journal of computer Science & Technology 2023年第5期38卷 970-984页

作者：胡春雨忽丽莎袁林陆佃杰吕蕾陈益强 Key Laboratory of Computing Power Network and Information Security Ministry of EducationShandong Computer Science Center(National Supercomputer Center in Jinan)Qilu University of Technology(Shandong Academy of Sciences)Jinan 250353China Shandong Provincial Key Laboratory of Computer Networks Shandong Fundamental Research Center for Computer ScienceJinan 250000China Shandong Provincial Key Laboratory for Novel Distributed Computer Software Technology Jinan 25000China Institute of Information Technology Hebei University of Economics and BusinessShijiazhuang 050061China School of Information Science and Engineering Shandong Normal UniversityJinan 250358China Institute of Computing Technology Chinese Academy of SciencesBeijing 100190China

Wearable health monitoring is a crucial technical tool that offers early warning for chronic diseases due to its superior portability and low power ***,most wearable health data is distributed across dfferent organizations,such as hospitals,research institutes,and companies,and can only be accessed by the owners of the data in compliance with data privacy *** first challenge addressed in this paper is communicating in a privacy-preserving manner among different *** second technical challenge is handling the dynamic expansion of the federation without model *** address the first challenge,we propose a horizontal federated learning method called Federated Extremely Random Forest(FedERF).Its contribution-based splitting score computing mechanism significantly mitigates the impact of privacy protection constraints on model *** on FedERF,we present a federated incremental learning method called Federated Incremental Extremely Random Forest(FedIERF)to address the second technical *** introduces a hardness-driven weighting mechanism and an importance-based updating scheme to update the existing federated model *** experiments show that FedERF achieves comparable performance with non-federated methods,and FedIERF effectively addresses the dynamic expansion of the *** opens up opportunities for cooperation between different organizations in wearable health monitoring.

关键词： federated learning incremental learning random forest wearable health monitoring

来源：评论

学校读者我要写书评

暂无评论

Efficient Large Models Fine-tuning on Commodity Servers via Memory-balanced Pipeline parallelism

Efficient Large Models Fine-tuning on Commodity Servers via ...

引用

IEEE International Conference on High Performance computing and Communications (HPCC)

作者： Yujie Liu Zhiquan Lai Weijie Liu Wei Wang Dongsheng Li National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

Large models have achieved impressive performance in many downstream tasks. Using pipeline parallelism to fine-tune large models on commodity GPU servers is an important way to make the excellent performance of large models available to the general public. Previous solutions fail to achieve an efficient memory-balanced pipeline parallelism. In this poster, we introduce a memory load-balanced pipeline parallel solution. This solution balances memory consumption across stages on commodity GPU servers via NVLink bridges. It establishes a new pathway to offload data from GPU to CPU by using the PCIe link of adjacent GPUs connected by the NVLink bridge. Furthermore, our method orchestrates offload operations to minimize the offload latency during large model fine-tuning. Experiments demonstrate that our solution can balance the memory footprint among pipeline stages without sacrificing training performance.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DMSA: Decentralized and Multi-keyword Selective Data Sharing and Acquisition

DMSA: Decentralized and Multi-keyword Selective Data Sharing...

引用

International Symposium on parallel and distributed Processing with Applications, ISPA

作者： Moheng Lin Peichang Shi Xiang Fu Feng Jiang Guodong Yi National Key Laboratory of Parallel and Distributed Computing College of Computer Science National University of Defense Technology Changsha China Xiangjiang Lab Changsha China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Blockchain technology has been extensively uti-lized in decentralized data-sharing applications, with the immutability of blockchain providing a witness for the circulation of data. However, current blockchain data-sharing solutions still fail to address the simultaneous screening needs of both the sender and receiver with multi-keywords. Without the capability to support bilateral simultaneous filtering, the disclosure of reasons for matching failures could inadvertently expose sensitive user data. Therefore, the challenge lies in enabling ciphertexts with multiple keywords and receivers with multiple interests to achieve mutual and simultaneous matching. Based on the technical foundations of SE (Searchable Encryption), MABE (Multi-Attribute Based Encryption), and polynomial fitting, this paper proposes a scheme called DMSA (Decentralized and Multi-keyword selective Sharing and selective Acquisition). This scheme can satisfy soundness, enabling ciphertexts carrying multiple keywords and receivers representing multiple interests to match each other simultaneously. We conducted a security analysis that confirms the security of DMSA against chosen-plaintext attacks. Our experimental results demonstrate a significant efficiency improvement, with a 67% increase over single-keyword data-sharing schemes and a 16% enhancement compared to the existing multi-keyword data-sharing solution.

关键词： distributed processing Filtering Data security keyword search Fitting Receivers Polynomials Data models Blockchains Encryption

来源：评论

学校读者我要写书评

暂无评论

HAF: a hybrid annotation framework based on expert knowledge and learning technique

引用

Science China(Information Sciences) 2022年第1期65卷 276-278页

作者： Zhixing LI Yue YU Tao WANG Gang YIN Xinjun MAO Huaimin WANG Key Laboratory of Parallel and Distributed Computing National University of Defense Technology College of Computer National University of Defense Technology

Dear editor,The increasing awareness of the potential value hidden in data has resulted in many data mining studies being conducted. In the domain of software engineering, for example, developers' behavioral data and code review data have been leveraged in social coding sites to automatically recommend relevant projects [1] and candidate reviewers [2, 3].

关键词：

来源：评论

学校读者我要写书评

暂无评论

OLM2: Automatic Optimal Strategy Generating for Large-Scale Model Training with Limited-Memory

OLM2: Automatic Optimal Strategy Generating for Large-Scale ...

引用

IEEE International Conference on Joint Cloud computing (JCC)

作者： Zhilin Yang Yu Tang Linbo Qiao Xi Yang Zhen Huang National Key Laboratory of Parallel and Distributed Computing College of Computer Science National University of Defense Technology Changsha 410073 China

The scale of model parameters and the amount of training data is exponentially increasing. It requires more GPU memory with the exponential increasement of model parameters. Recomputation and swapping are two main memory optimization methods that have been extensively studied, and there are also optimization strategies that combine the two methods. However, most of them are based on heuristic search strategies, which do not explore the complete solution space and can’t guarantee the optimality of the solution results. An optimal search strategy with tensor-level recomputation and swapping is expected in large-scale model training. In this paper, we propose an optimal strategy searching algorithm combining tensor-based recomputation and swapping. Specifically, the memory swapping strategy is reformulated as an optimization problem, which converts the memory constraints into mixed integer programming, to find the optimal memory optimization strategy. By leveraging the advantages of both recomputation and swapping, this approach minimizes computation consumption without exceeding the available memory limitation. Experimental results show that our method exhibits about 60% reduction in memory requirements during the training process. Furthermore, our method can reduce the overall training time beyond the existing algorithms. Compared to Checkmate, our approach achieves about 0.3–0.9% reduction in computation cost per iteration.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DrugProtKGE: Weakly Supervised Knowledge Graph Embedding for Highly-Effective Drug-Protein Interaction Representation

DrugProtKGE: Weakly Supervised Knowledge Graph Embedding for...

引用

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Qiu, Yanlong Wang, Siqi Yang, Xi Qiu, Xinyuan Wu, Chengkun Cui, Yingbo Yang, Canqun National University of Defense Technology Institute for Quantum Information State Key Laboratory of High-Performance Computing College of Computer Science Hunan Changsha410073 China National Supercomputer Center in Tianjin Tianjin300457 China National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing College of Computer Science Hunan Changsha410073 China National University of Defense Technology Department of Biology and Chemistry College of Science Hunan Changsha410073 China

ISBN: (纸本)9798350337488

With the exponential growth of biomedical knowledge in unstructured text repositories such as PubMed, it is imminent to establish a knowledge graph-style, efficient searchable and targeted database that can support the need of information retrieval from researchers and clinicians. To mine knowledge from graph databases, most previous methods view a triple in a graph (see Fig. 1) as the basic processing unit and embed the triplet element (i.e. drugs/chemicals, proteins/genes and their interaction) as separated embedding matrices, which cannot capture the semantic correlation among triple elements. To remedy the loss of semantic correlation caused by disjoint embeddings, we propose a novel approach to learn triple embeddings by combining entities and interactions into a unified representation. Furthermore, traditional methods usually learn triple embeddings from scratch, which cannot take advantage of the rich domain knowledge embedded in pre-trained models, and is also another significant reason for the fact that they cannot distinguish the differences implied by the same entity in the multi-interaction triples. In this paper, we propose a novel fine-tuning based approach to learn better triple embeddings by creating weakly supervised signals from pre-trained knowledge graph embeddings. The method automatically samples triples from knowledge graphs and estimates their pairwise similarity from pre-trained embedding models. The triples are then fed pairwise into a Siamese-like neural architecture, where the triple representation is fine-tuned in the manner bootstrapped by triple similarity scores. Finally, we demonstrate that triple embeddings learned with our method can be readily applied to several downstream applications (e.g. triple classification and triple clustering). We evaluated the proposed method on two open-source drug-protein knowledge graphs constructed from PubMed abstracts, as provided by BioCreative. Our method achieves consistent improvement in both t

关键词： Drug-Protein Interaction Knowledge Graph Embedding Triple Embedding Weakly Supervised Learning

来源：评论

学校读者我要写书评

暂无评论

Graph Structure Learning via Transfer Entropy for Multivariate Time Series Anomaly Detection

Graph Structure Learning via Transfer Entropy for Multivaria...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Mingyu Liu Yijie Wang Xiaohui Zhou Yongjun Wang National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Computer Science and Technology National University of Defense Technology Changsha China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Multivariate time series anomaly detection (MTAD) poses a challenge due to temporal and feature dependencies. The critical aspects of enhancing the detection performance lie in accurately capturing the dependencies between variables within the sliding window and effectively leveraging them. Existing studies rely on domain knowledge to pre-set the window size, and overlook the strength of dependencies while calculating direction based on variable similarity. This paper proposes GSLTE, a graph structure learning method for MTAD. GSLTE employs Fast Fourier Transform to conduct iterative segmentation of the whole series, selecting the dominant Fourier frequency as the window size for each subsequence within the minimum interval. GSLTE quantifies the direction and strength of the dependencies based on variable-lag transfer entropy which is achieved through Dynamic Time Warping method to learn asymmetric links between variables. Extensive experiments show that GNN-based MTAD methods applying GSLTE can further improve anomaly detection performance while outperforming state-of-the-art competitors.

关键词： Learning systems Time-frequency analysis Fast Fourier transforms Time series analysis Signal processing Feature extraction Entropy Iterative methods Speech processing Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

A Survey on Talking Head Generation: The Methods, Status and Challenges

SSRN

引用

SSRN 2023年

作者： Cai, Yali Qiao, Peng Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha410073 China

The talking head generation aims to synthesize a speech video of the source identity from a driving video or audio or text data irrelevant to the source identity. It can not only be applied to games and virtual reality applications, but also provide data for fake data detection. In recent years, the research of talking head generation is widely popular, and the authenticity of the generated results has also been greatly improved. However, the synthetic results still have great room for progress. We summarize the existing researches in this paper, hoping to offer assistance for later researchers. Furthermore, we divide these methods into three categories according to the input data type, namely video, audio and text driven talking head generation methods, and analyze them in detail. In addition, we also summarize the data sets commonly used in this kind of research and explore the evaluation criteria for measuring the performance of the method. Finally, the shortcomings of the existing methods in this field and the future direction are presented in last section. © 2023, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Deep Time Series Anomaly Detection with Local Temporal Pattern Learning

Deep Time Series Anomaly Detection with Local Temporal Patte...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yizhou Li Yijie Wang Hongzuo Xu Xiaohui Zhou National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China Intelligent Game and Decision Lab (IGDL) Beijing China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Self-supervised time series anomaly detection (TSAD) demonstrates remarkable performance improvement by extracting high-level data semantics through proxy tasks. Nonetheless, most existing self-supervised TSAD techniques rely on manual- or neural-based transformations when designing proxy tasks, overlooking the intrinsic temporal patterns of time series. This paper proposes a local temporal pattern learning-based time series anomaly detection (LTPAD). LTPAD first generates sub-sequences. Pairwise sub-sequences naturally manifest proximity relationships along the time axis, and such correlations can be used to construct supervision and train neural networks to facilitate the learning of temporal patterns. Time intervals between two sub-sequences serve as labels for sub-sequence pairs. By classifying these labeled data pairs, our model captures the local temporal patterns of time series, thereby modeling the temporal pattern-aware "normality". Abnormal scores of testing data are acquired by evaluating their conformity to these learned patterns shared in training data. Extensive experiments show that LTPAD significantly outperforms state-of-the-art competitors.

关键词： Time series analysis Semantics Neural networks Training data Manuals Signal processing Data models Speech processing Anomaly detection Testing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：