检索结果-内蒙古大学图书馆

LOFS: A Lightweight Online File Storage Strategy for Effective Data Deduplication at Network Edge

IEEE TRANSACTIONS ON parallel AND distributed SYSTEMS 2022年第10期33卷 2263-2276页

作者： Cheng, Geyao Guo, Deke Luo, Lailong Xia, Junxu Gu, Siyuan Natl Univ Def Technol Sci & Technol Informat Syst Engn Lab Changsha 410073 Hunan Peoples R China Natl Univ Def Technol Natl Lab Parallel & Distributed Proc Changsha 410073 Hunan Peoples R China

Edge computing responds to users' requests with low latency by storing the relevant files at the network edge. Various data deduplication technologies are currently employed at edge to eliminate redundant data chunks for space saving. However, the lookup for the global huge-volume fingerprint indexes imposed by detecting redundancies can significantly degrade the data processing performance. Besides, we envision a novel file storage strategy that realizes the following rationales simultaneously: 1) space efficiency, 2) access efficiency, and 3) load balance, while the existing methods fail to achieve them at one shot. To this end, we report LOFS, a Lightweight Online File Storage strategy, which aims at eliminating redundancies through maximizing the probability of successful data deduplication, while realizing the three design rationales simultaneously. LOFS leverages a lightweight three-layer hash mapping scheme to solve this problem with constant-time complexity. To be specific, LOFS employs the Bloom filter to generate a sketch for each file, and thereafter feeds the sketches to the Locality Sensitivity hash (LSH) such that similar files are likely to be projected nearby in LSH tablespace. At last, LOFS assigns the files to real-world edge servers with the joint consideration of the LSH load distribution and the edge server capacity. Trace-driven experiments show that LOFS closely tracks the global deduplication ratio and generates a relatively low load std compared with the comparison methods.

关键词： Servers Indexes Costs Redundancy Throughput Image edge detection Resource management Data deduplication locality sensitivity hash edge computing storage systems

来源：评论

学校读者我要写书评

暂无评论

High performance dilated convolutions on multi-core DSPs

引用

CCF TRANSACTIONS ON HIGH PERFORMANCE COMPUTING 2024年第1期6卷 78-93页

作者： Wang, Yang Wang, Qinglin Pei, Xiangdong Mei, Songzhu Li, Rongchun Liu, Jie Natl Univ Def Technol Natl Key Lab Parallel & Distributed Comp Changsha 410073 Hunan Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Hunan Peoples R China Beijing Inst Astronaut Syst Engn Beijing 100076 Peoples R China

Dilated convolutions are widely used to accomplish wide receptive fields while keeping the resolution of feature maps in deep learning applications, such as semantic segmentation and object detection. However, the data locality in dilated convolutions deteriorates rapidly with the increase of dilation rate, which brings a great challenge to the high-performance direct implementation of convolutions. Multi-core digital signal processors (DSPs) with software-controlled on-chip memories allow programmers to move data between on-chip and off-chip memories by hand so that it may be very friendly to the direct implementation of dilated convolutions. In this paper, we introduce a high-performance parallel direct implementation of dilated convolutions on multi-core DSPs in a CPU-DSP heterogeneous prototype processor, which can effectively capture the data locality in dilated convolutions. The experimental results demonstrate that the direct implementation achieves much better performance than GEMM-based ones on multi-core DSPs for all the tested layers, and gets much higher efficiency than the high-performance libraries on three other architectures in cases with large feature maps. In addition, the direct implementation also exhibits good scalability.

关键词： CNNs Dilated convolutions DSPs parallel algorithm

来源：评论

学校读者我要写书评

暂无评论

EmbRace: Accelerating Sparse Communication for distributed Training of Deep Neural Networks 51

EmbRace: Accelerating Sparse Communication for Distributed T...

引用

51st International Conference on parallel processing (ICPP)

作者： Li, Shengwei Lai, Zhiquan Li, Dongsheng Zhang, Yiming Ye, Xiangyu Duan, Yabo Univ Def Technol Natl Key Lab Parallel & Distributed Proc Comp Coll Changsha Peoples R China Xiamen Univ Xiamen Peoples R China

ISBN: (纸本)9781450397339

distributed data-parallel training has been widely adopted for deep neural network (DNN) models. Although current deep learning (DL) frameworks scale well for dense models like image classification models, we find that these DL frameworks have relatively low scalability for sparse models like natural language processing (NLP) models that have highly sparse embedding tables. Most existing works overlook the sparsity of model parameters thus suffering from significant but unnecessary communication overhead. In this paper, we propose EmbRace, an efficient communication framework to accelerate communications of distributed training for sparse models. EmbRace introduces Sparsity-aware Hybrid Communication, which integrates AlltoAll and model parallelism into data-parallel training, so as to reduce the communication overhead of highly sparse parameters. To effectively overlap sparse communication with both backward and forward computation, EmbRace further designs a 2D Communication Scheduling approach which optimizes the model computation procedure, relaxes the dependency of embeddings, and schedules the sparse communications of each embedding row with a priority queue. We have implemented a prototype of EmbRace based on PyTorch and Horovod, and conducted comprehensive evaluations with four representative NLP models. Experimental results show that EmbRace achieves up to 2.41x speedup compared to the state-of-the-art distributed training baselines.

关键词： distributed training deep learning sparsity of NLP models communication scheduling

来源：评论

学校读者我要写书评

暂无评论

Deep-to-Bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation 1

引用

15th International Conference on Knowledge Science, Engineering, and Management (KSEM)

作者： Wang, Ankun Liu, Feng Huang, Zhen Hu, Minghao Li, Dongsheng Chen, Yifan Xie, Xinjia Natl Univ Def Technol Natl Key Lab Parallel & Distributed Proc Changsha Peoples R China Informat Res Ctr Mil Sci Beijing Peoples R China

ISBN: (数字)9783031109867

ISBN: (纸本)9783031109867;9783031109850

There are millions of parameters and huge computational power consumption behind the outstanding performance of pre-trained language models in natural language processing tasks. Knowledge distillation is considered as a compression strategy to address this problem. However, previous works have the following shortcomings: (i) distill partial transformer layers of the teacher model, which not only do not make full use of the teacher-side information. But also break the coherence of the information, (ii) neglect the difficulty differences of knowledge from deep to shallow, which corresponds to different level information of teacher model. In this paper, we introduce a deep-to-bottom weights decay review mechanism to knowledge distillation, which could fuse teacher-side information while taking each layer's difficulty level into consideration. To validate our claims, we distill a 12-layer BERT into a 6-layer model and evaluate it on the GLUE dataset. Experimental results show that our review approach is not only able to outperform other existing techniques, but also outperform the original model on partial datasets.

关键词： Knowledge distillation Pre-training language model Model compression

来源：评论

学校读者我要写书评

暂无评论

DPSS: Dynamic Parameter Selection for Outlier Detection on Data Streams 28

DPSS: Dynamic Parameter Selection for Outlier Detection on D...

引用

IEEE 28th International Conference on parallel and distributed Systems (IEEE ICPADS)

作者： Zhang, Ruyi Wang, Yijie Zhou, Haifang Li, Bin Xu, Hongzuo Natl Univ Def Technol Coll Comp Sci & Technol Parallel & Distributed Proc Lab Changsha Peoples R China

ISBN: (纸本)9781665473156

Outlier detection on data streams identifies unusual states to sense and alarm potential risks and faults of the target systems in both the cyber and physical world. As different parameter settings of machine learning algorithms can result in dramatically different performance, automatic parameter selection is also of great importance in deploying outlier detection algorithms in data streams. However, current canonical parameter selection methods suffer from two key challenges: (i) Data streams generally evolve over time, but these existing methods use a fixed training set, which fails to handle this evolving environment and often results in suboptimal parameter recommendations;(ii) The stream is infinite, and thus any parameter selection method taking the entire stream as input is infeasible. In light of these limitations, this paper introduces a Dynamic Parameter Selection method for outlier detection on data Streams (DPSS for short). DPSS uses Gaussian process regression to model the relationship between parameters and detecting performance and uses Bayesian optimization to explore the optimal parameter setting. For each new subsequence, DPSS updates the recommended parameter setting to suit the evolving characteristics. Besides, DPSS only uses historical calculations to guide the parameter setting sampling and adjust the Gaussian process regression results. DPSS can be employed as an auxiliary plug-in tool to improve the detection performance of outlier detection methods. Extensive experiments show that our method can significantly improve the F-score of outlier detectors in data streams compared to its counterparts and obtains more superior parameter selection performance than other state-ofthe-art parameter selection approaches. DPSS also achieves better time and memory efficiency compared to competitors.

关键词： Parameter Selection Data Stream Outlier Detection Bayesian optimization

来源：评论

学校读者我要写书评

暂无评论

Comprehensive Deadlock Prevention for GPU Collective Communication 25

Comprehensive Deadlock Prevention for GPU Collective Communi...

引用

20th European Conference on Computer Systems-EuroSys

作者： Pan, Lichen Liu, Juncheng Fu, Yongquan Yuan, Jinhui Zhang, Rongkai Li, Pengze Xiao, Zhen Peking Univ Sch Comp Sci Beijing Peoples R China OneFlow Res Stockholm Sweden Natl Univ Def Technol Coll Comp Sci & Technol Natl Key Lab Parallel & Distributed Comp Changsha Peoples R China

ISBN: (纸本)9798400711961

distributed deep neural network training necessitates efficient GPU collective communications, which are inherently susceptible to deadlocks. GPU collective deadlocks arise easily in distributed deep learning applications when multiple collectives circularly wait for each other. GPU collective deadlocks pose a significant challenge to the correct functioning and efficiency of distributed deep learning, and no general effective solutions are currently available. Only in specific scenarios, ad-hoc methods, making an application invoke collectives in a consistent order across GPUs, can be used to prevent circular collective dependency and deadlocks. This paper presents DFCCL, a novel GPU collective communication library that provides a comprehensive approach for GPU collective deadlock prevention while maintaining high performance. DFCCL achieves preemption for GPU collectives at the bottom library level, effectively preventing deadlocks even if applications cause circular collective dependency. DFCCL ensures high performance with its execution and scheduling methods for collectives. Experiments show that DFCCL effectively prevents GPU collective deadlocks in various situations. Moreover, extensive evaluations demonstrate that DFCCL delivers performance comparable to or superior to NCCL, the state-of-the-art collective communication library highly optimized for NVIDIA GPUs.

关键词： Collective Communication GPU Deadlock Prevention Preemption

来源：评论

学校读者我要写书评

暂无评论

ParTransgrid: A scalable parallel preprocessing tool for unstructured-grid cell-centered computational fluid dynamics applications

引用

SOFTWARE-PRACTICE & EXPERIENCE 2023年第1期53卷 6-26页

作者： Zhang, Jian Liu, Jie Zhou, Naichun Tang, Jing He, Xie Chen, Jianqiang Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha Peoples R China China Aerodynam Res & Dev Ctr Computat Aerodynam Inst Mianyang Sichuan Peoples R China

The development of a basic scalable preprocessing tool is the key routine to accelerate the entire computational fluid dynamics (CFD) workflow toward the exascale computing era. In this work, a parallel preprocessing tool, called ParTransgrid, is developed to translate the general grid format like CFD General Notation System into an efficient distributed mesh data format for large-scale parallel computing. Through ParTransgrid, a flexible face-based parallel unstructured mesh data structure designed in Hierarchical Data Format can be obtained to support various cell-centered unstructured CFD solvers. The whole parallel preprocessing operations include parallel grid I/O, parallel mesh partition, and parallel mesh migration, which are linked together to resolve the run-time and memory consumption bottlenecks for increasingly large grid size problems. An inverted index search strategy combined with a multi-master-slave communication paradigm is proposed to improve the pairwise face matching efficiency and reduce the communication overhead when constructing the distributed sparse graph in the phase of parallel mesh partition. And we present a simplified owner update rule to fast the procedure of raw partition boundaries migration and the building of shared faces/nodes communication mapping list between new sub-meshes with an order of magnitude of speed-up. Experiment results reveal that ParTransgrid can be easily scaled to billion-level grid CFD applications, the preparation time for parallel computing with hundreds of thousands of cores is reduced to a few minutes.

关键词： CFD CGNS parallel computing preprocessing unstructured grid

来源：评论

学校读者我要写书评

暂无评论

The Doctrine of MEAN: Realizing Deduplication Storage at Unreliable Edge

引用

IEEE TRANSACTIONS ON parallel AND distributed SYSTEMS 2023年第10期34卷 2811-2826页

作者： Xia, Junxu Cheng, Geyao Luo, Lailong Guo, Deke Lv, Pin Sun, Bowen Natl Univ Def Technol Sci & Technol Informat Syst Engn Lab Changsha 410073 Hunan Peoples R China Natl Univ Def Technol Sci & Technol Informat Syst Engn Lab Natl Lab Parallel & Distributed Proc Changsha 410073 Hunan Peoples R China Guangxi Univ Sch Comp Elect & Informat Nanning 530004 Guangxi Peoples R China

Placing popular data at the network edge helps reduce the retrieval latency, but it also brings challenges to the limited edge storage space. Currently, using available yet not necessarily reliable edge resources is common sense for edge space expansion, while deploying deduplication storage strategies is a general method for better space utilization. However, a contradiction arises when jointly implementing data deduplication with unreliable edge resources. On the one hand, the deduplication policy stipulates that any data chunk can be stored exactly once;on the other hand, the use of unreliable resources imposes that data should be backed up for the seek of file availability. To resolve such contradiction, we propose MEAN, a deduplication-enabled storage system using unreliable resources at the network edge. The core idea of MEAN is to place similar files together for better deduplication and maintain replicas of popular files for higher reliability. We first formulate this problem and prove its NP-hardness, then provide efficient heuristics based on similarity-aware hierarchical clustering. Three different reliability scenarios are comprehensively considered to develop our algorithms. We also implement a prototype system and evaluate the performance of MEAN with a real-world dataset. The results show that MEAN can fortify the file hit ratio under unreliable environments by 77% while reducing the file retrieval delay up to 71%, compared with the state-of-the-art approach.

关键词： Deduplication fault tolerance storage system edge computing

来源：评论

学校读者我要写书评

暂无评论

ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations

ST-PINN: A Self-Training Physics-Informed Neural Network for...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Yan, Junjun Chen, Xinhai Wang, Zhichao Zhoui, Enqiang Liu, Jie Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Lab Changsha 410073 Peoples R China Natl Univ Def Technol Lab Digitizing Software Frontier Equipment Changsha 410073 Peoples R China

ISBN: (纸本)9781665488679

Partial differential equations (PDEs) are an essential computational kernel in physics and engineering. With the advance of deep learning, physics-informed neural networks (PINNs), as a mesh-free method, have shown great potential for fast PDE solving in various applications. To address the issue of low accuracy and convergence problems of existing PINNs, we propose a self-training physics-informed neural network, ST-PINN. Specifically, ST-PINN introduces a pseudo label based self-learning algorithm during training. It employs governing equation as the pseudo-labeled evaluation index and selects the highest confidence examples from the sample points to attach the pseudo labels. To our best knowledge, we are the first to incorporate a self-training mechanism into physics-informed learning. We conduct experiments on five PDE problems in different fields and scenarios. The results demonstrate that the proposed method allows the network to learn more physical information and benefit convergence. The ST-PINN outperforms existing physics-informed neural network methods and improves the accuracy by a factor of 1.33x-2.54x.

关键词： partial differential equations physics-informed neural networks pseudo label self-training

来源：评论

学校读者我要写书评

暂无评论

Local-to-Global Deep Clustering on Approximate Uniform Manifold

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2023年第5期35卷 5035-5046页

作者： Wang, Tuo Zhang, Xiang Lan, Long Luo, Zhigang Natl Univ Def Technol Inst Quantum Informat Changsha 410073 Peoples R China Natl Univ Def Technol State Key Lab High Performance Comp Changsha 410073 Peoples R China Natl Univ Def Technol Coll Comp Changsha 410073 Peoples R China Natl Univ Def Technol Sci & Technol Parallel & Distributed Proc Changsha 410073 Peoples R China

Deep clustering usually treats the clustering assignments as supervisory signals to learn a more compact representation with deep neural networks, under the guidance of clustering-oriented losses. Nevertheless, we observe that, without reliable supervision, such losses for global clustering would destroy the locally geometric structure underlying data. In this paper, we propose a local-to-global deep clustering method based on approximate uniform manifold (LGC-AUM) to address this issue in a two-stage fashion. In the local stage, an intra-manifold preservation loss is proposed to preserve intra-manifold structures locally on basis of approximate uniform manifold, and an inter-manifold discrimination loss is for global inter-manifold structure. Thus, this stage serves to learn more discriminative structure-preserving features by reducing the correlations between different manifolds, which paves the way for the final clustering. Build off the learned features, the second stage explores a clustering loss based on approximate uniform manifold to establish stable network training for effective clustering with two auxiliary distributions. Experiments on five benchmark datasets verify the efficacy of our LGC-AUM as compared to several well-behaved clustering counterparts.

关键词： Manifolds Clustering methods Task analysis Representation learning Training Data visualization Deep learning Manifold structure feature discrimination target distribution deep clustering

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：