With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strategy, called hybrid parallelism distribu...
详细信息
ISBN:
(纸本)9798400701405
With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strategy, called hybrid parallelism distributed training, is imported to tackle the problem of deploying large-scale models. However, how to evaluate the hybrid strategy and the utilization of each device remains a challenge since existing works either profile on a real large-scale cluster with high time and money costs or only analyze a specific type of parallelism without considering the hybrid parallelism. In this work, we proposed DistSim, an event-based performance model to accurately analyze each device's computation and communication activities with low profiling costs. DistDim breaks down the model into events according to the given distributed strategy, which can be profiled on two nodes. Then DistSim leverages the hierarchy of different parallel strategies to generate the computation and communication event-flow from layer level to model level and finally the activity timeline of each device participating in training. Experiment shows that DistSim can reach <4% errors when predicting distributing training batch time and <5% errors when predicting a single device's activity time in various hybrid strategy settings. We also provide a use-case of DistSim, automatically evaluate and search the best distributed training strategy, and find a hybrid strategy with at most 7.37x throughput improvement.
With the ever-growing network traffic and the vast amount of abnormal traffic being created, anomaly detection methods have attracted close attention in the cybersecurity domain. Generative adversarial networks (GANs)...
详细信息
The increasing use of fine spatial and temporal data in hydrological modeling has resulted in a prohibitively computational demand and run time. The serial computing method adopted by most modeling routines has largel...
详细信息
With the rise of advanced applications based on Artificial Intelligence (AI) and Internet-of-Things (IoT), mobile devices have become more intelligent, introducing a novel concept, Mobile Edge Intelligence. But the li...
详细信息
Conventional electronic Artificial Neural networks (ANNs) accelerators focus on architecture design and numerical computation optimization to improve the training speed. Optical technology with low energy consumption ...
详细信息
ISBN:
(纸本)9783030967727;9783030967710
Conventional electronic Artificial Neural networks (ANNs) accelerators focus on architecture design and numerical computation optimization to improve the training speed. Optical technology with low energy consumption and high transmission speed are expected to play an important role in the next generation of computing architectures. To provide a better understanding of optical technology used in ANN acceleration, we present a comprehensive review for the optical implementations of ANNs accelerator in this paper. We propose a classification of existing solutions which are categorized into optical computing acceleration and optical communication acceleration according to optical effects and optical architectures. Moreover, we discuss the challenges for these photonic neural network acceleration approaches to highlight the most promising future research opportunities in this field.
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game T...
ISBN:
(纸本)9789819708000
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game Theory Based Task Offloading Scheme for Maximizing Social Welfare in Edge computing;research on Dos Attack Simulation and Detection in Low-Orbit Satellite Network;malware Detection Method Based on Visualization;TOC: Joint Task Offloading and Computation Reuse in Vehicular Edge computing;distributed Task Offloading for IoAV Using DDP-DQN;a Task Offloading and Resource Allocation Optimization Method in End-Edge-Cloud Orchestrated computing;multi-agent Cooperative Intrusion Detection Based on Generative Data Augmentation;an Uncertainty-Aware Auction Mechanism for Federated Learning;Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC;query Optimization Mechanism for Blockchain-Based Efficient Data Traceability;research on the Evolution Path of Network Hotspot Events Based on the Event Evolutionary Graph;Task Offloading in UAV-Assisted Vehicular Edge computingnetworks;path Planning of Coastal Ships Based on Improved Hybrid A-Star;data Augmentation Method Based on Partial Noise Diffusion Strategy for One-Class Defect Detection Task;K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data;MPQUIC Transmission Control Strategy for SDN-Based Satellite Network;an Energy Prediction Method for Energy Harvesting Wireless Sensor with Dynamically Adjusting Weight Factor;Bayesian Optimization for Auto-tuning Convolution Neural Network on GPU;persistent Sketch: A Memory-Efficient and Robust Algorithm for Finding Top-k Persistent Flows;faCa: Fast Aware and Competition-Avoided Balancing for Data Center Network;Optimizing GNN Inference Processing on Very Long Vector Processor;GDTM: Gaussian Differential Trust Mechanism for Optimal Recommender System;privacy-Enhanced Dyna
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game T...
ISBN:
(纸本)9789819708079
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game Theory Based Task Offloading Scheme for Maximizing Social Welfare in Edge computing;research on Dos Attack Simulation and Detection in Low-Orbit Satellite Network;malware Detection Method Based on Visualization;TOC: Joint Task Offloading and Computation Reuse in Vehicular Edge computing;distributed Task Offloading for IoAV Using DDP-DQN;a Task Offloading and Resource Allocation Optimization Method in End-Edge-Cloud Orchestrated computing;multi-agent Cooperative Intrusion Detection Based on Generative Data Augmentation;an Uncertainty-Aware Auction Mechanism for Federated Learning;Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC;query Optimization Mechanism for Blockchain-Based Efficient Data Traceability;research on the Evolution Path of Network Hotspot Events Based on the Event Evolutionary Graph;Task Offloading in UAV-Assisted Vehicular Edge computingnetworks;path Planning of Coastal Ships Based on Improved Hybrid A-Star;data Augmentation Method Based on Partial Noise Diffusion Strategy for One-Class Defect Detection Task;K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data;MPQUIC Transmission Control Strategy for SDN-Based Satellite Network;an Energy Prediction Method for Energy Harvesting Wireless Sensor with Dynamically Adjusting Weight Factor;Bayesian Optimization for Auto-tuning Convolution Neural Network on GPU;persistent Sketch: A Memory-Efficient and Robust Algorithm for Finding Top-k Persistent Flows;faCa: Fast Aware and Competition-Avoided Balancing for Data Center Network;Optimizing GNN Inference Processing on Very Long Vector Processor;GDTM: Gaussian Differential Trust Mechanism for Optimal Recommender System;privacy-Enhanced Dyna
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game T...
ISBN:
(纸本)9789819708338
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game Theory Based Task Offloading Scheme for Maximizing Social Welfare in Edge computing;research on Dos Attack Simulation and Detection in Low-Orbit Satellite Network;malware Detection Method Based on Visualization;TOC: Joint Task Offloading and Computation Reuse in Vehicular Edge computing;distributed Task Offloading for IoAV Using DDP-DQN;a Task Offloading and Resource Allocation Optimization Method in End-Edge-Cloud Orchestrated computing;multi-agent Cooperative Intrusion Detection Based on Generative Data Augmentation;an Uncertainty-Aware Auction Mechanism for Federated Learning;Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC;query Optimization Mechanism for Blockchain-Based Efficient Data Traceability;research on the Evolution Path of Network Hotspot Events Based on the Event Evolutionary Graph;Task Offloading in UAV-Assisted Vehicular Edge computingnetworks;path Planning of Coastal Ships Based on Improved Hybrid A-Star;data Augmentation Method Based on Partial Noise Diffusion Strategy for One-Class Defect Detection Task;K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data;MPQUIC Transmission Control Strategy for SDN-Based Satellite Network;an Energy Prediction Method for Energy Harvesting Wireless Sensor with Dynamically Adjusting Weight Factor;Bayesian Optimization for Auto-tuning Convolution Neural Network on GPU;persistent Sketch: A Memory-Efficient and Robust Algorithm for Finding Top-k Persistent Flows;faCa: Fast Aware and Competition-Avoided Balancing for Data Center Network;Optimizing GNN Inference Processing on Very Long Vector Processor;GDTM: Gaussian Differential Trust Mechanism for Optimal Recommender System;privacy-Enhanced Dyna
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game T...
ISBN:
(纸本)9789819708109
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game Theory Based Task Offloading Scheme for Maximizing Social Welfare in Edge computing;research on Dos Attack Simulation and Detection in Low-Orbit Satellite Network;malware Detection Method Based on Visualization;TOC: Joint Task Offloading and Computation Reuse in Vehicular Edge computing;distributed Task Offloading for IoAV Using DDP-DQN;a Task Offloading and Resource Allocation Optimization Method in End-Edge-Cloud Orchestrated computing;multi-agent Cooperative Intrusion Detection Based on Generative Data Augmentation;an Uncertainty-Aware Auction Mechanism for Federated Learning;Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC;query Optimization Mechanism for Blockchain-Based Efficient Data Traceability;research on the Evolution Path of Network Hotspot Events Based on the Event Evolutionary Graph;Task Offloading in UAV-Assisted Vehicular Edge computingnetworks;path Planning of Coastal Ships Based on Improved Hybrid A-Star;data Augmentation Method Based on Partial Noise Diffusion Strategy for One-Class Defect Detection Task;K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data;MPQUIC Transmission Control Strategy for SDN-Based Satellite Network;an Energy Prediction Method for Energy Harvesting Wireless Sensor with Dynamically Adjusting Weight Factor;Bayesian Optimization for Auto-tuning Convolution Neural Network on GPU;persistent Sketch: A Memory-Efficient and Robust Algorithm for Finding Top-k Persistent Flows;faCa: Fast Aware and Competition-Avoided Balancing for Data Center Network;Optimizing GNN Inference Processing on Very Long Vector Processor;GDTM: Gaussian Differential Trust Mechanism for Optimal Recommender System;privacy-Enhanced Dyna
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game T...
ISBN:
(纸本)9789819707973
The proceedings contain 165 papers. The special focus in this conference is on Algorithms and Architectures for parallel Processing. The topics include: MDCF: Multiple Dynamic Cuckoo Filters for LSM-Tree;a Game Theory Based Task Offloading Scheme for Maximizing Social Welfare in Edge computing;research on Dos Attack Simulation and Detection in Low-Orbit Satellite Network;malware Detection Method Based on Visualization;TOC: Joint Task Offloading and Computation Reuse in Vehicular Edge computing;distributed Task Offloading for IoAV Using DDP-DQN;a Task Offloading and Resource Allocation Optimization Method in End-Edge-Cloud Orchestrated computing;multi-agent Cooperative Intrusion Detection Based on Generative Data Augmentation;an Uncertainty-Aware Auction Mechanism for Federated Learning;Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC;query Optimization Mechanism for Blockchain-Based Efficient Data Traceability;research on the Evolution Path of Network Hotspot Events Based on the Event Evolutionary Graph;Task Offloading in UAV-Assisted Vehicular Edge computingnetworks;path Planning of Coastal Ships Based on Improved Hybrid A-Star;data Augmentation Method Based on Partial Noise Diffusion Strategy for One-Class Defect Detection Task;K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data;MPQUIC Transmission Control Strategy for SDN-Based Satellite Network;an Energy Prediction Method for Energy Harvesting Wireless Sensor with Dynamically Adjusting Weight Factor;Bayesian Optimization for Auto-tuning Convolution Neural Network on GPU;persistent Sketch: A Memory-Efficient and Robust Algorithm for Finding Top-k Persistent Flows;faCa: Fast Aware and Competition-Avoided Balancing for Data Center Network;Optimizing GNN Inference Processing on Very Long Vector Processor;GDTM: Gaussian Differential Trust Mechanism for Optimal Recommender System;privacy-Enhanced Dyna
暂无评论