The National Science Data Fabric (NSDF) is our solution to the problem of addressing the data-sharing needs of the growing data science community. NSDF is designed to make sharing data across geographically distribute...
详细信息
ISBN:
(纸本)9798400701559
The National Science Data Fabric (NSDF) is our solution to the problem of addressing the data-sharing needs of the growing data science community. NSDF is designed to make sharing data across geographically distributed sites easier for users who lack technical expertise and infrastructure. By developing an easy-to-install software stack, we promote the FAIR data-sharing principles in NSDF while leveraging existing high-speed data transfer infrastructures such as Globus and XRootD. This work shows how we leverage latency and throughput information between geo-distributed NSDF sites with NSDF entry points to optimize the automatic coordination of data placement and transfer across the data fabric, which can further improve the efficiency of data sharing.
One way for supporting incremental checkpointing is the exploitation of classical memory protection services - in particular the mprotect (...) system call offered by Posix compliant operating systems - for intercepti...
详细信息
distributedcomputing platforms involve multiple processing systems connected through a network and support the parallel execution of applications. They enable huge computational power and data processing with a quick...
详细信息
ISBN:
(数字)9783031505836
ISBN:
(纸本)9783031505829;9783031505836
distributedcomputing platforms involve multiple processing systems connected through a network and support the parallel execution of applications. They enable huge computational power and data processing with a quick response time. Examples of use cases requiring distributedcomputing are stream processing, batch processing, and client-server models. Most of these use cases involve tasks executed in a sequence on different computers to arrive at the results. Numerous distributedcomputing algorithms have been suggested in the literature, focusing on efficiently utilizing compute nodes to handle tasks within a workflow on on-premises setups. Industries that previously relied on on-premises setups for big data processing are shifting to cloud environments offered by providers such as Azure, Amazon, and Google. This transition is driven by the convenience of Platform-as-a-Service offerings scuh as Batch Services, Hadoop, and Spark. These PaaS services, coupled with auto-provisioning and auto-scaling, reduce costs through a Pay-As-You-Go model. However, a significant challenge with cloud services is configuring them with only a single type of machine for performing all the tasks in the distributed workflow, although each task has diverse compute node requirements. To address this issue in this paper, we propose an Intelligent task scheduling framework that uses a classifier-based dynamic task scheduling approach to determine the best available node for each task. The proposed framework improves the overall performance of the distributedcomputing workflow by optimizing task allocation and utilization of resources. Although Azure Batch Service is used in this paper to illustrate the proposed framework, our approach can also be implemented on other PaaS distributedcomputing platforms.
Training recurrent neural network controllers in closed-loop control systems with combined Levenberg-Marquardt and Forward Accumulation Through Time algorithm advances the research in a grid-connected converter for so...
详细信息
ISBN:
(纸本)9789819746767;9789819746774
Training recurrent neural network controllers in closed-loop control systems with combined Levenberg-Marquardt and Forward Accumulation Through Time algorithm advances the research in a grid-connected converter for solar integration to a power system. However, an effective training algorithm is needed for a large number of trajectories with a high sampling frequency. Thus, we propose a new effective training mechanism based on parallelcomputing and weight dropout techniques for recurrent neural network controllers in this paper. Experimental results on both the Amazon Web Services (AWS) cloud and the Graphical Processing Unit (GPU) show that our proposed training mechanism runs at a more promising acceleration rate than the existing algorithms.
Federated learning, an emerging distributed learning framework, has gained prominence recently. However, two challenges arise when applying federated learning to medical data. The first challenge stems from the inabil...
详细信息
As the electric power distribution system evolves towards a distributed architecture, the task of providing ancillary services to bulk-power systems through grid-edge assets becomes increasingly crucial. In this paper...
详细信息
GNU parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, partic...
详细信息
parallel high-performance computing relies on cache -efficient, branch-free algorithms that are often expressed as imperative computations over multi-dimensional arrays. Numerous problem domains, spanning from image p...
详细信息
ISBN:
(纸本)9798350364613;9798350364606
parallel high-performance computing relies on cache -efficient, branch-free algorithms that are often expressed as imperative computations over multi-dimensional arrays. Numerous problem domains, spanning from image processing to graph analytics, and from state space exploration in combinatorial optimization to computer Chess, require carefully crafted algorithms that capitalize on patterns inherent in the underlying problem structure. A renowned technique, SIMD-Within-A-Register (SWAR), harnesses integer arithmetic to attain significant hardware parallelism. However, this approach typically demands labor-intensive efforts from domain experts with specialized knowledge of the underlying hardware architecture. We therefore present a compiler-driven approach that automates the transformation of conventional array -based C-code into highly tuned integer arithmetic, exploiting SWAR parallelism without the requirement of tedious manual optimization efforts. Our approach achieves substantial performance improvements, exhibiting an average speedup of 30x compared to conventional array-based implementations.
Efficient and flexible cloud computing is widely used in distributed systems. However, in the Internet of Things (IoT) environment with heterogeneous capabilities, the performance of cloud computing may decline due to...
详细信息
ISBN:
(数字)9789819708598
ISBN:
(纸本)9789819708581;9789819708598
Efficient and flexible cloud computing is widely used in distributed systems. However, in the Internet of Things (IoT) environment with heterogeneous capabilities, the performance of cloud computing may decline due to limited communication resources. As located closer to the end, edge computing is used to replace cloud computing to provide timely and stable services. To accomplish distributed system and privacy preserving, Federated Learning (FL) has been combined with edge computing. However, due to the large number of clients, the amount of data transmitted will also grow exponentially. How to reduce the communication overhead in FL is still a big problem. As a major method to reduce the communication overhead, compressing the transmission parameters can effectively reduce the communication overhead. However, the existing methods do not consider the possible internal relationship between neurons. In this paper, we propose Neuron Pruning-Based FL for communication-efficient distributed training, which is a model pruning method to compress model parameters transmitted in FL. In contrast to the previous methods, we use dimensionality reduction method as the importance factor of neurons, and take advantage of the correlation between them to carry out model pruning. Our analysis results show that NPBFL can reduce communication overhead while maintaining classification accuracy.
The proceedings contain 11 papers. The topics discussed include: hybrid forecasting model for smart grid using exogenous variables;IoRT enabled gesture mimicking robotic hand based on computer vision;optimizing membra...
ISBN:
(纸本)9798331515850
The proceedings contain 11 papers. The topics discussed include: hybrid forecasting model for smart grid using exogenous variables;IoRT enabled gesture mimicking robotic hand based on computer vision;optimizing membrane distillation configuration for superior heat and mass transfer;a modular distributed operating system architecture for scalable mesh topologies;effect of cotton dust morphology on photovoltaic module performance;simulation design of a photovoltaic-based reliable street lighting system using PVSyst software;optimized sizing of grid connected PV systems for distributed generation in remote Sindh and Balochistan;bagging as defense mechanism against adversarial attack;and design of a digital twin based feedback controller for a parallel mechanism based six degree of freedom machining bed.
暂无评论