As the scale of data continues to grow, the training of machine learning models presents new challenges. In this paper, a machine learning model is established, which is optimized by stochastic gradient descent algori...
详细信息
Stream join is a fundamental data operator for processing real-time data, but it faces computational challenges during stream inequality join (theta join operators) due to frequent updates in indexing data structures....
详细信息
ISBN:
(纸本)9798400704130
Stream join is a fundamental data operator for processing real-time data, but it faces computational challenges during stream inequality join (theta join operators) due to frequent updates in indexing data structures. To tackle this problem, we identify three key insights: 1) identifying skewed data distributions in real-time and implementing dedicated indexing structures for skewed keys to reduce index update costs;2) leveraging optimized data structures, including insert-efficient mutable and search-efficient immutable structures to optimize the search stream join process and 3) adopting learned indexes instead of conventional ones, which can provide up to 4x better performance. In this Ph.D. work, we propose novel solutions for distributed and multi-core stream join processing, including an indexing solution that uses a space-efficient dedicated filter and a two-stage data structure that effectively holds and processes sliding window items (bounded streaming contents). We are also exploring the adoption and benefits of learned indexes for real-time stream join processing. Despite non-trivial challenges like state management for distributed processing, processing guarantees, and efficient concurrency mechanisms, experiments on distributed stream processing systems show superior performance compared to state-of-the-art solutions.
Temporal Graph Neural networks (TGNNs) have achieved success in real-world graph-based applications. The increasing scale of dynamic graphs necessitates distributed training. However, deploying TGNNs in a distributed ...
详细信息
ISBN:
(纸本)9789819757787;9789819757794
Temporal Graph Neural networks (TGNNs) have achieved success in real-world graph-based applications. The increasing scale of dynamic graphs necessitates distributed training. However, deploying TGNNs in a distributed setting poses challenges due to the temporal dependencies in dynamic graphs, the need for computation balance during distributed training, and the non-ignorable communication costs across disjointed trainers. In this paper, we propose DisTGL, a distributed temporal graph neural network learning system. Leveraging a temporal-aware partitioning scheme and a series of enhanced communication techniques, DisTGL ensures efficient distributed computation and minimizes communication overhead. Based on that, DisTGL facilitates fast TGNN training and downstream tasks. An evaluation of DisTGL using various TGNN models shows that i) DisTGL achieves acceleration of up to 10x compared to existing TGNN frameworks;and ii) the proposed distributed dynamic graph partitioning reduces cross-machine operations by 25%, while the optimized communication reduce the costs by 1.5-2.5x.
Register allocation is a crucial step in the compilation pipeline that decides what program values occupy which physical registers. Single-path code's use of predicated instructions instead of branching control-fl...
详细信息
ISBN:
(纸本)9798350371291;9798350371284
Register allocation is a crucial step in the compilation pipeline that decides what program values occupy which physical registers. Single-path code's use of predicated instructions instead of branching control-flow means register allocation must also allocate predicate registers. In this paper, we improve the original single-path transformation to allow generic register allocators to allocate predicate registers. Our improved transformation splits register allocation into two. First, the general-purpose registers are allocated as usual using a generic register allocator. Then, the main steps of the single-path transformation are performed while still using virtual predicate registers. Lastly, register allocation is rerun using the generic allocator to allocate the predicate registers. Our results show the improved single-path transformation increasing performance by up to 80 % and reducing code size by up to 43 % compared to the original transformation that uses a custom predicate allocator.
The proceedings contain 47 papers. The topics discussed include: ElasticRoom: multi-tenant DNN inference engine via co-design with resource-constrained compilation and strong priority scheduling;efficient all-to-all c...
ISBN:
(纸本)9798400704130
The proceedings contain 47 papers. The topics discussed include: ElasticRoom: multi-tenant DNN inference engine via co-design with resource-constrained compilation and strong priority scheduling;efficient all-to-all collective communication schedules for direct-connect topologies;ESG: pipeline-conscious efficient scheduling of DNN workflows on serverless platforms with shareable GPUs;ETS: deep learning training iteration time prediction based on execution trace sliding window;IDT: intelligent data placement for multi-tiered main memory with reinforcement learning;FaaSKeeper: learning from building serverless services with ZooKeeper as an example;accelerating function-centric applications by discovering, distributing, and retaining reusable context in workflow systems;and Faast: an efficient serverless framework made snapshot-based function response fast.
To enable large-scale and efficient deployment of artificial intelligence (AI), the combination of AI and edge computing has spawned Edge Intelligence, which leverages the computing and communication capabilities of e...
详细信息
Deep Neural networks (DNNs) process big datasets achieving high accuracy on incredibly complex tasks. However, this progress has led to a scalability impasse, as DNNs require massive amounts of processing power and lo...
详细信息
ISBN:
(纸本)9781665455800
Deep Neural networks (DNNs) process big datasets achieving high accuracy on incredibly complex tasks. However, this progress has led to a scalability impasse, as DNNs require massive amounts of processing power and local memory to be trained, making them impossible or impractical to be used on a single device. This situation has led to the design of distributed training architectures, where the DNN and the training data can be split among multiple processors. How to choose the appropriate distributed training architecture, however, remains an open question. To help bring insights into this debate, in this work we design a distributed Training Simulator (DTS) that estimates the training time of a DNN in a distributed architecture through a mathematical model of the distributed architecture and resource-allocation heuristics. We illustrate the power of the proposed DTS through the implementation of five different distributed architectures, Pipeline Learning, Federated Learning, Split Learning, Parallel Split Learning, and Federated Split Learning, and we validate the accuracy of the training estimates using three different datasets of varying complexity and two different DNNs. Finally, we present a trade-off analysis to demonstrate the coherence of DTS estimates for diverse high-performance computing scenarios by comparing these estimates with the behaviors of a real computer cluster.
This paper develops a distributed control protocol for the distributed optimal consensus problem of multi-agent systems. The dynamics of multi-agent systems are heterogeneous linear. Considering that multi-agent syste...
详细信息
ISBN:
(纸本)9789819743988;9789819743995
This paper develops a distributed control protocol for the distributed optimal consensus problem of multi-agent systems. The dynamics of multi-agent systems are heterogeneous linear. Considering that multi-agent systems often face various external disturbances in practical applications, which may seriously affect the performance and stability of the system. The control protocol proposed in this paper pays special attention to dealing with external disturbances in linear dynamic systems. Theoretical results show that the distributed control protocol is robust under the external disruption and drives the states of each agent to the auxiliary variable in a finite time. Further, with the aid of Lyapunov method, the states of close-loop system globally converge to the optimal solution of the non-smooth optimization problem. Finally, the simulation results demonstrate the effectiveness of proposed control protocol.
This paper presents a comparison to low-cost passive impedance matching networks, encompassing three different techniques of the pi model: lumped, distributed and hybrid. Additionally, three different optimization app...
详细信息
ISBN:
(纸本)9798350350876;9798350350869
This paper presents a comparison to low-cost passive impedance matching networks, encompassing three different techniques of the pi model: lumped, distributed and hybrid. Additionally, three different optimization approaches including simulation are compared within impedance transformation of a practical case study involving a commercial Bluetooth transceiver and an SMD antenna. The performance of each technique and approach is analyzed in terms of the scattering parameters, the bandwidth of the network, PCB footprint and associated costs. The study underscores the crucial role of electromagnetic simulations in the design of matching networks, due to its reliability. It also exhibits the hybrid technique as the most favorable option, both technically and practically. The results and insights derived from this study contribute to the optimization methods of impedance matching techniques tailored for low-cost FR4 substrate applications in wireless communication systems.
Breast cancer is one of the commonly occurring malignant tumors and poses a serious threat to women's health. To address this important clinical problem, an innovative cross-modal graph neural network model, PreGA...
详细信息
暂无评论