检索结果-内蒙古大学图书馆

Merge-Weighted Dynamic Time Warping for Speech Recognition

Journal of Computer Science & Technology 2014年第6期29卷 1072-1082页

作者：张湘莉兰骆志刚李明 State Key Laboratory of Pathogen and Biosecurity Beijing Institute of Microbiology and Epidemiology Science and Technology on Parallel and Distributed Processing Laboratory School of Computer Science National University of Defense Technology David R.Cheriton School of Computer Science University of WaterlooWaterlooOntario N2L 3G1Canada ACM IEEE

Obtaining training material for rarely used English words and common given names from countries where English is not spoken is difficult due to excessive time, storage and cost factors. By considering personal privacy, language- independent （LI） with lightweight speaker-dependent （SD） automatic speech recognition （ASR） is a convenient option to solve tile problem. The dynamic time warping （DTW） algorithm is the state-of-the-art algorithm for small-footprint SD ASR for real-time applications with limited storage and small vocabularies. These applications include voice dialing on mobile devices, menu-driven recognition, and voice control on vehicles and robotics. However, traditional DTW has several lhnitations, such as high computational complexity, constraint induced coarse approximation, and inaccuracy problems. In this paper, we introduce the merge-weighted dynamic time warping （MWDTW） algorithm. This method defines a template confidence index for measuring the similarity between merged training data and testing data, while following the core DTW process. MWDTW is simple, efficient, and easy to implement. With extensive experiments on three representative SD speech recognition datasets, we demonstrate that our method outperforms DTW, DTW on merged speech data, the hidden Markov model （HMM） significantly, and is also six times faster than DTW overall.

关键词： merge-weighted dynamic time warping natural language processing speech recognition and synthesis tem-plate confidence index

来源：评论

学校读者我要写书评

暂无评论

An Efficient Broadcast Authentication Protocol in Wireless Sensor Networks

引用

Chinese Journal of Electronics 2023年第2期18卷 368-372页

作者： Xin Zhao Xiaodong Wang Wanrong Yu Xingming Zhou National Key Laboratory of Parallel and Distributed Processing National University of Defense Technology Changsha China

Broadcast authentication is a critical security service in wireless sensor networks. A protocol named $\mu\text{TESLA}$ [1] has been proposed to provide efficient authentication service for such networks. However, when applied to applications such as time synchronization and fire alarm in which broadcast messages are sent infrequently, $\mu\text{TESLA}$ encounters problems of wasted key resources and slow message verification. This paper presents a new protocol named GBA (Generalized broadcast authentication), for efficient broadcast authentication in these applications. GBA utilizes the one-way key chain mechanism of $\mu\text{TESLA}$ , but modifies the keys and time intervals association, and changes the key disclosure mechanism according to the message transmission model in these applications. The proposed technique can take full use of key resources, and shorten the message verification time to an acceptable level. The analysis and experiments show that GBA is more efficient and practical than $\mu\text{TESLA}$ in applications with various message transmission models.

关键词： Wireless sensor networks Analytical models Protocols Authentication Synchronization Security Message authentication

来源：评论

学校读者我要写书评

暂无评论

CD-Sched: An Automated Scheduling Framework for Accelerating Neural Network Training on Shared Memory CPU-DSP Platforms 23

CD-Sched: An Automated Scheduling Framework for Accelerating...

引用

Proceedings of the 2023 International Conference on Power, Communication, computing and Networking Technologies

作者： Yuanyuan Xiao Zhiquan Lai Dongsheng Li National Key Laboratory for Parallel and Distributed Processing National University of Defense Technology China National Key Laboratory of Parallel and Distributed Processing National University of Defense Technology China

ISBN: (纸本)9781450399951

DSP holds significant potential for important applications in Deep Neural Networks. However, there is currently a lack of research focused on shared-memory CPU-DSP heterogeneous chips. This paper proposes CD-Sched, an automated scheduling framework that aims to address this gap. By predicting the latency of operators on both CPU and DSP, CD-Sched automatically schedules the computation of operators to the appropriate computing device. This scheduling optimization accelerates the computation of individual operators and ultimately improves the overall training time of neural networks. In end-to-end training tasks, CD-Sched can significantly reduce the overall training time, with an average reduction of approximately 10.77%.

关键词： computation offloading

来源：评论

学校读者我要写书评

暂无评论

HSDP: Accelerating Large-scale Model Training via Efficient Sharded Data parallelism

HSDP: Accelerating Large-scale Model Training via Efficient ...

引用

International Symposium on parallel and distributed Processing with Applications, ISPA

作者： Yanqi Hao Zhiquan Lai Wei Wang Shengwei Li Weijie Liu Keshi Ge Dongsheng Li National Key Laboratory of Parallel and Distributed Computing(PDL) College of Computer National University of Defense Technology Changsha China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Large deep neural network (DNN) models have demonstrated exceptional performance across diverse downstream tasks. Sharded data parallelism (SDP) has been widely used to reduce the memory footprint of model states. In a DNN training cluster, a device usually has multiple inter-device links that connect to other devices, like NVLink and InfiniBand. However, existing SDP approaches employ a single link at any given time, encountering challenges in efficient training due to significant communication overheads. We observe that the inter-device links can work independently without affecting each other. To reduce the fatal communication overhead of distributed training of large DNNs, this paper introduces HSDP, an efficient SDP training approach that enables the simultaneous utilization of multiple inter-device links. HSDP partitions models in a novel fine-grained manner and orchestrates the communication processes of partitioned parameters while considering inter-device links. This design enables concurrent communication execution and reduces communication overhead. To further optimize the training performance of HSDP, we propose a HSDP planner. The HSDP planner first abstracts the model partition and execution of HSDP into a communication parallel strategy, and builds a cost model to estimate the performance of each strategy. We then formulate the strategy searching as an optimization problem and solve it with an off-the-shelf solver. Evaluations on representative DNN workloads demonstrate that HSDP achieves up to 1.30× speedup compared to the state-of-the-art SDP training approaches.

关键词： Training Performance evaluation Costs distributed databases Artificial neural networks parallel processing Throughput Search problems Data models Optimization

来源：评论

学校读者我要写书评

暂无评论

DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

arXiv

引用

arXiv 2023年

作者： Lu, Menglong Huang, Zhen Zhao, Yunxiang Tian, Zhiliang Liu, Yang Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology China Beijing Institute of Biotechnology China

Self-training emerges as an important research line on domain adaptation. By taking the model’s prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanishment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%. © 2023, CC BY-NC-SA.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Media Access Control Protocol Based on Hybrid Antennas in Ad Hoc Networks

Media Access Control Protocol Based on Hybrid Antennas in Ad...

引用

Symposia and Workshops on Ubiquitous, Autonomic and Trusted computing, UIC-ATC

作者： Yanrong Ding Xiaodong Wang Hu Shen Pengyu Guo National Key Laboratory for Parallel and Distributed Processing Changsha China

Nested Circular Directional MAC, a modified medium access control protocol of DMAC protocol, is proposed in this paper to support both directional antennas and omni-directional antennas simultaneously in one Ad Hoc network. Omni-directional antennas follow the RTS/CTS mechanism which is multi-hop and directional antennas follow the DRTS/DCTS mechanism, ensuring the asymmetric communication effectively. In order to resolve new deafness problem and hidden terminal problem, the directional antennas also use circular DRTS and nested antenna circumrotation. Simulation results show that NCDMAC supports hybrid antennas effectively and is better than MMAC and Circular-MAC in terms of increasing throughput, but it also increases the communication delay.

关键词： Delay Deafness Directive antennas Throughput Directional antennas Media Access Protocol

来源：评论

学校读者我要写书评

暂无评论

3D parallelism for Transformers via Integer Programming

3D Parallelism for Transformers via Integer Programming

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Hao Zheng Peng Liang Yu Tang Yanqi Shi Linbo Qiao Dongsheng Li National Key Laboratory of Parallel and Distributed Computing National University Of Defense Technology Changsha P.R.China

Transformer models, such as BERT, GPT, and ViT, have been applied to a wide range of areas in recent years, due to their efficacy. In order to improve the training efficiency of Transformer models, different distributed training approaches have been proposed, like Megatron-LM [8]. However, when multi-dimensional parallelism strategies are considered, due to the complexity, existing works can not harmonize the different strategies well enough to obtain a globally optimal solution. In this paper, we propose a parallelism strategy searching algorithm PTIP, which generates operator-level parallelism strategies consisting of three schemes: data parallelism, tensor parallelism, and pipeline parallelism. PTIP abstracts these three parallelism schemes simultaneously into an auxiliary graph, reformulates the searching problem into a mixed-integer programming (MIP) problem, and uses a MIP solver to obtain a high-quality multi-dimensional strategy. Experiments conducted on Transformers demonstrate that PTIP obtains 13.9% − 24.7% performance improvement compared to Megatron-LM [8].

关键词：

来源：评论

学校读者我要写书评

暂无评论

Pro-Prophet: A Systematic Load Balancing Method for Efficient parallel Training of Large-scale MoE Models

arXiv

引用

arXiv 2024年

作者： Wang, Wei Lai, Zhiquan Li, Shengwei Liu, Weijie Ge, Keshi Shen, Ao Su, Huayou Li, Dongsheng The National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Hunan Changsha China

he size of deep learning models has been increasing to enhance model quality. The linear increase in training computation budget with model size means that training an extremely large-scale model is exceedingly time-consuming. Recently, the Mixture of Expert (MoE) has drawn significant attention as it can scale models to extra-large sizes with a stable computation budget. However, inefficient distributed training of large-scale MoE models hinders their broader application. Specifically, a considerable dynamic load imbalance occurs among devices during training, significantly reducing throughput. Several load-balancing works have been proposed to address the challenge. System-level solutions draw more attention for their hardware affinity and non-disruption of model convergence compared to algorithm-level ones. However, they are troubled by high communication costs and poor communication-computation overlapping. To address these challenges, we propose a systematic load-balancing method, Pro-Prophet, which consists of a planner and a scheduler for efficient parallel training of large-scale MoE models. To adapt to the dynamic load imbalance, we profile training statistics and use them to design Pro-Prophet. For lower communication volume, Pro-Prophet planner determines a series of lightweight load-balancing strategies and efficiently searches for a communication-efficient one for training based on the statistics. For sufficient overlapping of communication and computation, Pro-Prophet scheduler schedules the data-dependent operations based on the statistics and operation features, further improving the training throughput. We conduct extensive experiments in four clusters and five MoE models. The results indicate that Pro-Prophet achieves up to 2.66x speedup compared to two popular MoE frameworks including Deepspeed-MoE and FasterMoE. Furthermore, Pro-Prophet has demonstrated a load-balancing improvement of up to 11.01x compared to a representative load-balancing work,

关键词： Budget control

来源：评论

学校读者我要写书评

暂无评论

A novel workflow management model based on mobile agents for Internet electronic commerce

A novel workflow management model based on mobile agents for...

引用

Technology of Object-Oriented Languages and Systems (TOOLS)

作者： Wu Gang Wu Quanyuan Wang Huaimin Parallel & Distributed Computing National Laboratory Changsha Institute of Technology Changsha China

Electronic commerce on the Internet is receiving a lot of attention. Workflow management is regarded as an effective mechanism for managing the business processes behind electronic commerce, but the present workflow management model has many drawbacks in this field. The paper gives a novel workflow management model based on mobile agents for Internet electronic commerce, and its advantages are also presented.

关键词： Mobile agents Internet Electronic commerce Companies Application software Technology management Business continuity distributed computing Laboratories Business communication

来源：评论

学校读者我要写书评

暂无评论

A load balancing-supported constant degree DHT ID assignment method

引用

Tien Tzu Hsueh Pao/Acta Electronica Sinica 2010年第11期38卷 2649-2654页

作者： Wang, Xiao-Hai Peng, Yu-Xing Li, Dong-Sheng Key Laboratory of Science and Technology for Parallel and Distributed Processing National University of Defense Technology Changsha Hunan 410073 China

A Load Balancing-Supported ID assignment method is the foundation to implement and maintain DHT overlays, realized constant degree DHTs usually use simple pure centralized or distributed ID management strategies, which cannot resolve the contradiction between cost of maintaining topologies' information and topologies' balance. Analyzing the universal tree structures in the topologies, an ID Assignment method RFIDAM based on the internal structure Routing Forest is proposed, which regularly aggregates local balancing information to guide new nodes' joining for overall balance. The experimental results show, with low maintenance and routing message overhead, the system's loading balance is efficiently ensured with the length of IDs differ by at most 2.

关键词： Topology

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：