检索结果-内蒙古大学图书馆

International Symposium on parallel and distributed Processing with Applications, ISPA

作者： Yanqi Hao Zhiquan Lai Wei Wang Shengwei Li Weijie Liu Keshi Ge Dongsheng Li National Key Laboratory of Parallel and Distributed Computing(PDL) College of Computer National University of Defense Technology Changsha China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Large deep neural network (DNN) models have demonstrated exceptional performance across diverse downstream tasks. Sharded data parallelism (SDP) has been widely used to reduce the memory footprint of model states. In a DNN training cluster, a device usually has multiple inter-device links that connect to other devices, like NVLink and InfiniBand. However, existing SDP approaches employ a single link at any given time, encountering challenges in efficient training due to significant communication overheads. We observe that the inter-device links can work independently without affecting each other. To reduce the fatal communication overhead of distributed training of large DNNs, this paper introduces HSDP, an efficient SDP training approach that enables the simultaneous utilization of multiple inter-device links. HSDP partitions models in a novel fine-grained manner and orchestrates the communication processes of partitioned parameters while considering inter-device links. This design enables concurrent communication execution and reduces communication overhead. To further optimize the training performance of HSDP, we propose a HSDP planner. The HSDP planner first abstracts the model partition and execution of HSDP into a communication parallel strategy, and builds a cost model to estimate the performance of each strategy. We then formulate the strategy searching as an optimization problem and solve it with an off-the-shelf solver. Evaluations on representative DNN workloads demonstrate that HSDP achieves up to 1.30× speedup compared to the state-of-the-art SDP training approaches.

关键词： Training Performance evaluation Costs distributed databases Artificial neural networks parallel processing Throughput Search problems Data models Optimization

来源：评论

学校读者我要写书评

暂无评论

DrugProtKGE: Weakly Supervised Knowledge Graph Embedding for Highly-Effective Drug-Protein Interaction Representation

DrugProtKGE: Weakly Supervised Knowledge Graph Embedding for...

引用

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Qiu, Yanlong Wang, Siqi Yang, Xi Qiu, Xinyuan Wu, Chengkun Cui, Yingbo Yang, Canqun National University of Defense Technology Institute for Quantum Information State Key Laboratory of High-Performance Computing College of Computer Science Hunan Changsha410073 China National Supercomputer Center in Tianjin Tianjin300457 China National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing College of Computer Science Hunan Changsha410073 China National University of Defense Technology Department of Biology and Chemistry College of Science Hunan Changsha410073 China

ISBN: (纸本)9798350337488

With the exponential growth of biomedical knowledge in unstructured text repositories such as PubMed, it is imminent to establish a knowledge graph-style, efficient searchable and targeted database that can support the need of information retrieval from researchers and clinicians. To mine knowledge from graph databases, most previous methods view a triple in a graph (see Fig. 1) as the basic processing unit and embed the triplet element (i.e. drugs/chemicals, proteins/genes and their interaction) as separated embedding matrices, which cannot capture the semantic correlation among triple elements. To remedy the loss of semantic correlation caused by disjoint embeddings, we propose a novel approach to learn triple embeddings by combining entities and interactions into a unified representation. Furthermore, traditional methods usually learn triple embeddings from scratch, which cannot take advantage of the rich domain knowledge embedded in pre-trained models, and is also another significant reason for the fact that they cannot distinguish the differences implied by the same entity in the multi-interaction triples. In this paper, we propose a novel fine-tuning based approach to learn better triple embeddings by creating weakly supervised signals from pre-trained knowledge graph embeddings. The method automatically samples triples from knowledge graphs and estimates their pairwise similarity from pre-trained embedding models. The triples are then fed pairwise into a Siamese-like neural architecture, where the triple representation is fine-tuned in the manner bootstrapped by triple similarity scores. Finally, we demonstrate that triple embeddings learned with our method can be readily applied to several downstream applications (e.g. triple classification and triple clustering). We evaluated the proposed method on two open-source drug-protein knowledge graphs constructed from PubMed abstracts, as provided by BioCreative. Our method achieves consistent improvement in both t

关键词： Drug-Protein Interaction Knowledge Graph Embedding Triple Embedding Weakly Supervised Learning

来源：评论

学校读者我要写书评

暂无评论

A Counterfactual Ultrasound Anti-Interference Self-Supervised Network for B-mode Ultrasound Tongue Extraction

A Counterfactual Ultrasound Anti-Interference Self-Supervise...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yan Jia Yuqing Cheng Kele Xu Yong Dou Peng Qiao Zhouyu He National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Systems Engineering National University of Defense Technology Changsha China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

B-mode ultrasound tongue imaging is a non-invasive and real-time method for visualizing vocal tract deformation. However, accurately extracting the tongue’s surface contour remains a significant challenge due to the low signal-to-noise ratio (SNR) and prevalent speckle noise in ultrasound images. Traditional supervised learning models often require large labeled datasets, which are labor-intensive to produce and susceptible to noise interference. To address these limitations, we present a novel Counterfactual Ultrasound Anti-Interference Self-Supervised Network (CUAI-SSN), which integrates self-supervised learning (SSL) with counterfactual data augmentation, progressively disentangles confounding factors, ensuring that the model generalizes well across varied ultrasound conditions. Our approach leverages causal reasoning to decouple noise from relevant features, enabling the model to learn robust representations that focus on essential tongue structures. By generating counterfactual image-label pairs, our method introduces alternative, noise-independent scenarios that enhance model training. Furthermore, we introduce attention mechanisms to enhance the network’s ability to capture fine-grained details even in noisy conditions. Extensive experiments on real ultrasound tongue images demonstrate that CUAI-SSN outperforms existing methods, setting a new benchmark for automated contour extraction in ultrasound tongue imaging. Our code is publicly available at https://***/inexhaustible419/CounterfactualultrasoundAI.

关键词： Training Ultrasonic imaging Tongue Self-supervised learning Data augmentation Data models Cognition Data mining Noise measurement Signal to noise ratio

来源：评论

学校读者我要写书评

暂无评论

Pro-Prophet: A Systematic Load Balancing Method for Efficient parallel Training of Large-scale MoE Models

arXiv

引用

arXiv 2024年

作者： Wang, Wei Lai, Zhiquan Li, Shengwei Liu, Weijie Ge, Keshi Shen, Ao Su, Huayou Li, Dongsheng The National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Hunan Changsha China

he size of deep learning models has been increasing to enhance model quality. The linear increase in training computation budget with model size means that training an extremely large-scale model is exceedingly time-consuming. Recently, the Mixture of Expert (MoE) has drawn significant attention as it can scale models to extra-large sizes with a stable computation budget. However, inefficient distributed training of large-scale MoE models hinders their broader application. Specifically, a considerable dynamic load imbalance occurs among devices during training, significantly reducing throughput. Several load-balancing works have been proposed to address the challenge. System-level solutions draw more attention for their hardware affinity and non-disruption of model convergence compared to algorithm-level ones. However, they are troubled by high communication costs and poor communication-computation overlapping. To address these challenges, we propose a systematic load-balancing method, Pro-Prophet, which consists of a planner and a scheduler for efficient parallel training of large-scale MoE models. To adapt to the dynamic load imbalance, we profile training statistics and use them to design Pro-Prophet. For lower communication volume, Pro-Prophet planner determines a series of lightweight load-balancing strategies and efficiently searches for a communication-efficient one for training based on the statistics. For sufficient overlapping of communication and computation, Pro-Prophet scheduler schedules the data-dependent operations based on the statistics and operation features, further improving the training throughput. We conduct extensive experiments in four clusters and five MoE models. The results indicate that Pro-Prophet achieves up to 2.66x speedup compared to two popular MoE frameworks including Deepspeed-MoE and FasterMoE. Furthermore, Pro-Prophet has demonstrated a load-balancing improvement of up to 11.01x compared to a representative load-balancing work,

关键词： Budget control

来源：评论

学校读者我要写书评

暂无评论

DMSA: Decentralized and Multi-keyword Selective Data Sharing and Acquisition

DMSA: Decentralized and Multi-keyword Selective Data Sharing...

引用

International Symposium on parallel and distributed Processing with Applications, ISPA

作者： Moheng Lin Peichang Shi Xiang Fu Feng Jiang Guodong Yi National Key Laboratory of Parallel and Distributed Computing College of Computer Science National University of Defense Technology Changsha China Xiangjiang Lab Changsha China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Blockchain technology has been extensively uti-lized in decentralized data-sharing applications, with the immutability of blockchain providing a witness for the circulation of data. However, current blockchain data-sharing solutions still fail to address the simultaneous screening needs of both the sender and receiver with multi-keywords. Without the capability to support bilateral simultaneous filtering, the disclosure of reasons for matching failures could inadvertently expose sensitive user data. Therefore, the challenge lies in enabling ciphertexts with multiple keywords and receivers with multiple interests to achieve mutual and simultaneous matching. Based on the technical foundations of SE (Searchable Encryption), MABE (Multi-Attribute Based Encryption), and polynomial fitting, this paper proposes a scheme called DMSA (Decentralized and Multi-keyword selective Sharing and selective Acquisition). This scheme can satisfy soundness, enabling ciphertexts carrying multiple keywords and receivers representing multiple interests to match each other simultaneously. We conducted a security analysis that confirms the security of DMSA against chosen-plaintext attacks. Our experimental results demonstrate a significant efficiency improvement, with a 67% increase over single-keyword data-sharing schemes and a 16% enhancement compared to the existing multi-keyword data-sharing solution.

关键词： distributed processing Filtering Data security Keyword search Fitting Receivers Polynomials Data models Blockchains Encryption

来源：评论

学校读者我要写书评

暂无评论

A survey of script learning

引用

Frontiers of Information technology & Electronic Engineering 2021年第3期22卷 341-373页

作者： Yi HAN Linbo QIAO Jianming ZHENG Hefeng WU Dongsheng LI Xiangke LIAO Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense TechnologyChangsha 410000China Science and Technology on Information Systems Engineering Laboratory National University of Defense TechnologyChangsha 410000China School of Data and Computer Science Sun Yat-sen UniversityGuangzhou 510006China

Script is the structured knowledge representation of prototypical real-life event *** the commonsense knowledge inside the script can be helpful for machines in understanding natural language and drawing commonsensible *** learning is an interesting and promising research direction,in which a trained script learning system can process narrative texts to capture script knowledge and draw ***,there are currently no survey articles on script learning,so we are providing this comprehensive survey to deeply investigate the standard framework and the major research topics on script *** research field contains three main topics:event representations,script learning models,and evaluation *** each topic,we systematically summarize and categorize the existing script learning systems,and carefully analyze and compare the advantages and disadvantages of the representative *** also discuss the current state of the research and possible future directions.

关键词： Script learning Natural language processing Commonsense knowledge modeling Event reasoning

来源：评论

学校读者我要写书评

暂无评论

A Survey on Talking Head Generation: The Methods, Status and Challenges

SSRN

引用

SSRN 2023年

作者： Cai, Yali Qiao, Peng Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha410073 China

The talking head generation aims to synthesize a speech video of the source identity from a driving video or audio or text data irrelevant to the source identity. It can not only be applied to games and virtual reality applications, but also provide data for fake data detection. In recent years, the research of talking head generation is widely popular, and the authenticity of the generated results has also been greatly improved. However, the synthetic results still have great room for progress. We summarize the existing researches in this paper, hoping to offer assistance for later researchers. Furthermore, we divide these methods into three categories according to the input data type, namely video, audio and text driven talking head generation methods, and analyze them in detail. In addition, we also summarize the data sets commonly used in this kind of research and explore the evaluation criteria for measuring the performance of the method. Finally, the shortcomings of the existing methods in this field and the future direction are presented in last section. © 2023, The Authors. All rights reserved.

关键词： Deep learning

来源：评论

学校读者我要写书评

暂无评论

Deep Time Series Anomaly Detection with Local Temporal Pattern Learning

Deep Time Series Anomaly Detection with Local Temporal Patte...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yizhou Li Yijie Wang Hongzuo Xu Xiaohui Zhou National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China Intelligent Game and Decision Lab (IGDL) Beijing China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Self-supervised time series anomaly detection (TSAD) demonstrates remarkable performance improvement by extracting high-level data semantics through proxy tasks. Nonetheless, most existing self-supervised TSAD techniques rely on manual- or neural-based transformations when designing proxy tasks, overlooking the intrinsic temporal patterns of time series. This paper proposes a local temporal pattern learning-based time series anomaly detection (LTPAD). LTPAD first generates sub-sequences. Pairwise sub-sequences naturally manifest proximity relationships along the time axis, and such correlations can be used to construct supervision and train neural networks to facilitate the learning of temporal patterns. Time intervals between two sub-sequences serve as labels for sub-sequence pairs. By classifying these labeled data pairs, our model captures the local temporal patterns of time series, thereby modeling the temporal pattern-aware "normality". Abnormal scores of testing data are acquired by evaluating their conformity to these learned patterns shared in training data. Extensive experiments show that LTPAD significantly outperforms state-of-the-art competitors.

关键词： Time series analysis Semantics Neural networks Training data Manuals Signal processing Data models Speech processing Anomaly detection Testing

来源：评论

学校读者我要写书评

暂无评论

Bi-Objective Scheduling Algorithm for Hybrid Workflow in JointCloud

Bi-Objective Scheduling Algorithm for Hybrid Workflow in Joi...

引用

IEEE International Conference on Joint Cloud Computing (JCC)

作者： Rui Li Huaimin Wang Peichang Shi National Key Laboratory of Parallel and Distributed Processing College of Computer Science National University of Defense Technology Changsha China State Key Laboratory of Complex & Critica Software Environment College of Computer Science National University of Defense Technology Changsha China

ISBN: (数字)9798350387339

ISBN: (纸本)9798350387346

Big data workflows are widely used in IoT, recommended systems, and real-time vision applications, and they continue to grow in complexity. These hybrid workflows consist of both resource-intensive batch jobs and latency-sensitive stream jobs. Examples include the data analytics workflow, which incorporates batch data transformations and low-latency querying, and the machine learning workflow, which processes stream data feature extraction before performing batch training and low-latency inference. However, existing research on workflow scheduling primarily focuses on either stream or batch workflows, neglecting the efficient scheduling of hybrid workflows that respect their diverse resource requirements and the costly data transfers between *** this article, we propose a hybrid workflow model that defines the optimal placement of hybrid workflows (OHWP) as a bi-objective optimization problem. Our proposed model takes into account parameters related to inter-communication between stream and batch jobs, as well as the heterogeneous resources in JointCloud environment. Additionally, we present OHWP-PS (OHWP on a Pruned Space), a scheduling algorithm for hybrid workflows that minimizes both cost and latency by improving the initial population and dynamically updating the search space. The results demonstrate that the proposed OHWP-PS algorithm is effective and competitive across all experiments.

关键词： Training Scheduling algorithms Machine vision Heuristic algorithms Machine learning Feature extraction Remote working

来源：评论

学校读者我要写书评

暂无评论

A Float-Point Arithmetic Logic Unit for High Performance RISC-V Processor

A Float-Point Arithmetic Logic Unit for High Performance RIS...

引用

Frontiers technology of Information and computer (ICFTIC), IEEE International Conference on

作者： Chunyuan Wei Chuan Yuan Yu Qi Bangjian Xu Xin Niu College of Computer Science and Electronic Engineering Hunan University Changsha China College of Electrical and Information Engineering Hunan University Changsha China Science and Technology on Parallel and Distributed Laboratory College of Computer National University of Defense Technology Changsha China

ISBN: (数字)9798331541750

ISBN: (纸本)9798331541767

With the rapid advancement of artificial intelligence, chips have become increasingly important. The emerging RISC-V instruction set gradually provides powerful computing support for this field. In this context, along with the computing requirements of deep learning, this paper presents the design of a high-performance floating-point arithmetic logic unit (FALU) that facilitates calculations with double-precision, single-precision, half-precision, and Bfloat16 precision data. This design is based on a single-channel algorithm with merged rounding. It improves and implements a composite adder that combines high and low bits. It also proposes a tree-like floating-point comparator based on the Kogge-Stone parallel prefix network. To ensure that the FALU components meet performance requirements, we undergo functional verification in the Vivado simulation environment. Operating at 1.47GHz under the 28nm CMOS process, the components achieve the predetermined performance indicators.

关键词： Deep learning Context Accuracy Instruction sets CMOS process Logic Artificial intelligence Adders Floating-point arithmetic

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：