Decentralized Multi-agent Learning (DML) enables collaborative model training while preserving data privacy. However, inherent heterogeneity in agents' resources (computation, communication, and task size) may lea...
详细信息
ISBN:
(纸本)9798350386066;9798350386059
Decentralized Multi-agent Learning (DML) enables collaborative model training while preserving data privacy. However, inherent heterogeneity in agents' resources (computation, communication, and task size) may lead to substantial variations in training time. This heterogeneity creates a bottleneck, lengthening the overall training time due to straggler effects and potentially wasting spare resources of faster agents. To minimize training time in heterogeneous environments, we present a Communication-Efficient Training Workload Balancing for Decentralized Multi-Agent Learning (ComDML), which balances the workload among agents through a decentralized approach. Leveraging local-loss split training, ComDML enables parallel updates, where slower agents offload part of their workload to faster agents. To minimize the overall training time, ComDML optimizes the workload balancing by jointly considering the communication and computation capacities of agents, which hinges upon integer programming. A dynamic decentralized pairing scheduler is developed to efficiently pair agents and determine optimal offloading amounts. We prove that in ComDML, both slower and faster agents' models converge, for convex and non-convex functions. Furthermore, extensive experimental results on popular datasets (CIFAR-10, CIFAR-100, and CINIC-10) and their non-I.I.D. variants, with large models such as ResNet-56 and ResNet-110, demonstrate that ComDML can significantly reduce the overall training time while maintaining model accuracy, compared to state-of-the-art methods. ComDML demonstrates robustness in heterogeneous environments, and privacy measures can be seamlessly integrated for enhanced data protection.
Multi-core processors, cloud computing, and distributed applications now define the landscape of modern software development and research. Efficiently teaching multithreading concepts is vital to equip students for re...
详细信息
ISBN:
(纸本)9798350351583;9798350351576
Multi-core processors, cloud computing, and distributed applications now define the landscape of modern software development and research. Efficiently teaching multithreading concepts is vital to equip students for real-world industry challenges. Navigating the complexities of multithreading proves challenging for students due to the high level of abstraction and difficulty associated with these concepts. The complexity of these technological fields raises the necessity for a comprehensive educational approach that facilitates students to understand the multithreading functionalities and usages with ease. This experience paper details our teaching approach, utilizing narrative-driven methods in parallel and distributed programming laboratories. We emphasize the relevance, novelty, and benefits of integrating interactive storytelling into the tangled learning process of parallel and distributed programming concepts. Concurrency, parallelization, and workload distribution are explained through live narrations, accompanied by animated presentations following a procedural storyline that spans laboratories. Complex ideas are transformed into meaningful character interactions, like stone-age people training octopuses to make ice-cream concurrently. This unique approach aims not only to enhance comprehension but also to make learning a vibrant and memorable experience for students, bridging the gap between theory and practical application. Our report extracts valuable lessons from student activities, emphasizing the motivational impact of engaging narratives. Students reflect on the relevance of concepts, shifting focus from implementation to understanding. Beyond just facilitating and easier access to knowledge, our approach aims to raise a deeper appreciation for the intricate world of parallel and distributed programming domain by encouraging students to see beyond the technical depth.
Fuzzy Cognitive Maps (FCMs) are powerful tools for the modeling of complex systems, one of which is time series modeling. Although advocated for their interpretability, existing methods measure concept relevance based...
详细信息
Structural centrality measures are often used to approximate or predict dynamical influence in a network. The recently proposed Expected Force of Infection (ExF) measures the entropy of all potential transmission path...
详细信息
ISBN:
(纸本)9798350363074;9798350363081
Structural centrality measures are often used to approximate or predict dynamical influence in a network. The recently proposed Expected Force of Infection (ExF) measures the entropy of all potential transmission paths starting at a node, effectively characterizing a node's role in epidemic diffusion processes. However, this promising metric has seen limited adoption mainly due to an inefficient formulation and the lack of an open-source implementation. In this paper, we present a novel cluster-centric, parallel algorithm enhancing ExF's efficiency and scalability. Compared to the simple parallel version of the original formulation of the ExF our efficient, open-source GPU implementation enables key nodes detection at previously intractable scales, with speed-ups of up to 300x on networks with up to 44 million edges. Leveraging on our algorithm, we compare the ExF with other well-known centrality metrics, upon six real and synthetic contact networks. The ExF emerges as the best of the considered metrics in a few, important tasks: it predicts the likelihood of a global epidemic and its diffusion speed, based on the centrality of the seed node;and it predicts how many other infections will occur as a consequence, in some sense, of a specific node having caught the disease.
The integration of Urgent computing is essential in order to adhere to stringent time and quality constraints of emerging distributed applications, hence facilitating efficient decision-making processes in numerous fi...
详细信息
ISBN:
(纸本)9783031506833;9783031506840
The integration of Urgent computing is essential in order to adhere to stringent time and quality constraints of emerging distributed applications, hence facilitating efficient decision-making processes in numerous fields. Adaptation of such applications to produce outcomes within the desired confidence range and defined time interval can be of great benefit, especially in distributed and heterogeneous execution contexts. This study provides a justification for the necessity of dynamic adaptation in applications that are time-sensitive. Furthermore, we present our viewpoint on time-sensitive applications and undertake a thorough analysis of the underlying principles and challenges that need to be resolved in order to accomplish this goal. This research aims to provide a comparative analysis of our suggested vision for adaptation in contrast to the existing literature. We provide a comprehensive explanation of the architectural framework that we plan to construct, and conclude with discussing some on-going challenges.
Deep neural networks (DNNs) have gained popularity in various fields, including computer vision and speech recognition. However, the growing size of data and complexity of models have made training DNNs increasingly t...
详细信息
The likelihood of unanticipated node failures in large-scale parallel computers increases with growing numbers of nodes. Furthermore, global reduction operations become major bottlenecks due to their limited parallel ...
详细信息
ISBN:
(纸本)9798350364613;9798350364606
The likelihood of unanticipated node failures in large-scale parallel computers increases with growing numbers of nodes. Furthermore, global reduction operations become major bottlenecks due to their limited parallel scalability. The Preconditioned Conjugate Gradient (PCG) method faces these challenges.
Training recurrent neural network controllers in closed-loop control systems with combined Levenberg-Marquardt and Forward Accumulation Through Time algorithm advances the research in a grid-connected converter for so...
详细信息
ISBN:
(纸本)9789819746767;9789819746774
Training recurrent neural network controllers in closed-loop control systems with combined Levenberg-Marquardt and Forward Accumulation Through Time algorithm advances the research in a grid-connected converter for solar integration to a power system. However, an effective training algorithm is needed for a large number of trajectories with a high sampling frequency. Thus, we propose a new effective training mechanism based on parallelcomputing and weight dropout techniques for recurrent neural network controllers in this paper. Experimental results on both the Amazon Web Services (AWS) cloud and the Graphical Processing Unit (GPU) show that our proposed training mechanism runs at a more promising acceleration rate than the existing algorithms.
The proceedings contain 21 papers. The topics discussed include: enhancing the performance of photonic crystal and gates with machine learning optimization;bitcoin price prediction based on financial data, technical i...
ISBN:
(纸本)9798350381580
The proceedings contain 21 papers. The topics discussed include: enhancing the performance of photonic crystal and gates with machine learning optimization;bitcoin price prediction based on financial data, technical indicators, and news headlines sentiment analysis using CNN and GRU deep learning algorithms;promoting cybersecurity knowledge via gamification: an innovative intervention design;modeling and verification of the causal broadcast algorithm using colored Petri Nets;GPU-based parallel technique for solving n-similarity problem in textual data mining;a novel approach for specification and verification of symmetric distributed algorithms using spin;optimizing geophysical workloads in high-performance computing: leveraging machine learning and transformer models for enhanced parallelism and processor allocation;and rational Jacobi kernel functions: a novel massively parallelizable orthogonal kernel for support vector machines.
The proceedings contain 55 papers. The topics discussed include: human resource analysis based on used libraries in eclipse projects on GitHub;modified convolutional network for the identification of COVID-19 with a m...
ISBN:
(纸本)9781665404037
The proceedings contain 55 papers. The topics discussed include: human resource analysis based on used libraries in eclipse projects on GitHub;modified convolutional network for the identification of COVID-19 with a mobile system;which dependency was updated? exploring who changes dependencies in npm packages;developing event routing service to support context-aware service integration;interactive chatbots for software engineering: a case study of code reviewer recommendation;effectiveness of explaining a program to others in finding its bugs;study of microservice execution framework using spoken dialogue agents;association metrics between two continuous variables for software project data;structure-preserving deep autoencoder-based dimensionality reduction for data visualization;Arabic sign language recognition: towards a dual way communication system between deaf and non-deaf people;and an investigation on multiscale normalized deep scattering spectrum with deep residual network for acoustic scene classification.
暂无评论