检索结果-内蒙古大学图书馆

Automatic compilation of loops to exploit operator parallelism on configurable arithmetic logic units

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2002年第1期13卷 45-66页

作者： Ramasubramanian, N Subramanian, R Pande, S Microsoft Corp Redmond WA 98052 USA Xilinx Inc San Jose CA 95124 USA Georgia Inst Technol Coll Comp Atlanta GA 30332 USA

Configurable Arithmetic Logic Units (ALUs) offer opportunities for adapting the underlying hardware to support the varying amount of parallelism in the computation. The problem of identifying the optimal parallel configurations (a configuration is defined as a given hardware implementation of different operators along with their multiplicities) at different steps in a program is a very complex issue but, if solved, allows the power of these ALUs to be maximally used. This paper focuses on developing an automatic compilation framework for configuration analysis to exploit operator parallelism within loop nests. The focus of this work is on performing configuration analysis to minimize costly reconfiguration overheads. In our framework, we initially carry out some operator and loop transformations to expose more opportunities for configuration reuse. We then present a two pass solution. The first pass attempts to generate either maximal cutsets (a cutset is defined as a group of statements that execute under a given configuration) or maximally parallel configurations by performing an analysis on the program dependency graph (PDG) of a loop nest. The second pass analyzes the trade-offs between the costs and benefits of reconfigurations across different cutsets and attempts to eliminate the reconfiguration overheads by merging cutsets. This methodology is implemented in the SUIF compilation system and is tested using some loops extracted from Perfect benchmarks and Livermore kernels. Good speedups are obtained, showing the merit of the proposed method. The method also scales well with the loop sizes and the amount of space available on FPGAs for configurable logic.

关键词： compilers parallel computing operator parallelism reconfigurable systems loop transformation FPGAs

来源：评论

学校读者我要写书评

暂无评论

Exact and Approximate operator parallelism

引用

CANADIAN MATHEMATICAL BULLETIN-BULLETIN CANADIEN DE MATHEMATIQUES 2015年第1期58卷 207-224页

作者： Zamani, Ali Moslehian, Mohammad Sal Ferdowsi Univ Mashhad Dept Pure Math Ctr Excellence Anal Algebra Struct Mashhad 91775 Iran

Extending the notion of parallelism we introduce the concept of approximate parallelism in normed spaces and then substantially restrict ourselves to the setting of Hilbert space operators endowed with the operator norm. We present several characterizations of the exact and approximate operator parallelism in the algebra B(H) of bounded linear operators acting on a Hilbert space H. Among other things, we investigate the relationship between the approximate parallelism and norm of inner derivations on B(H). We also characterize the parallel elements of a C*-algebra by using states. Finally we utilize the linking algebra to give some equivalent assertions regarding parallel elements in a Hilbert C*-module.

关键词： C*-algebra approximate parallelism operator parallelism Hilbert C*-module

来源：评论

学校读者我要写书评

暂无评论

An elastic reconfiguration strategy for operators in distributed stream computing systems

引用

JOURNAL OF SUPERCOMPUTING 2025年第5期81卷 1-31页

作者： Sun, Dawei Fan, Yinuo Guan, Chengjun Rong, Jia Gao, Shang Buyya, Rajkumar China Univ Geosci Sch Informat Engn Beijing 10083 Peoples R China Monash Univ Dept Data Sci & AI Clayton Vic 3800 Australia Deakin Univ Sch Informat Technol Geelong Vic 3216 Australia Univ Melbourne Sch Comp & Informat Syst Quantum Cloud Comp & Distributed Syst qCLOUDS Lab Parkville Vic Australia

Low latency and high throughput are crucial for distributed stream computing systems. Existing operator reconfiguration strategies often have poor performance under resource-limited and latency-constraint scenarios. The challenge lies in the elasticity of operator parallelism and reconfiguration of operators that balances performance constraints and performance improvement. To address these issues, we propose Er-Stream, an elastic reconfiguration strategy for various application scenarios. This paper discusses the Er-Stream from the following aspects: (1) We model task topology as a queuing network to evaluate system latency, and construct a communication cost model to formalize the reconfiguration problem;(2) we proposed an elastic strategy for operator parallelism to rationally utilize the available resources and reduce the processing latency of topology;(3) we proposed a reconfiguration strategy for operators to reduce the communication cost, and set thresholds added to control its trigger frequency;(4) we design and implement Er-Stream and integrated it into Apache Storm. We evaluate key metrics such as latency, throughput, resource usage, and CPU utilization in a real-world distributed stream computing environment. Results demonstrate significant improvements achieved by Er-Stream. In comparison with Storm's existing strategies, it reduces average system latency by up to 30%, increases average system throughput by 1.89 times, lowers average resource usage by 26.6%, and increases CPU utilization by 19.8%.

关键词： operator parallelism Elastic strategy operator reconfiguration Distributed stream computing Storm

来源：评论

学校读者我要写书评

暂无评论

AutoDDL: Automatic Distributed Deep Learning With Near-Optimal Bandwidth Cost

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2024年第8期35卷 1331-1344页

作者： Chen, Jinfan Li, Shigang Guo, Ran Yuan, Jinhui Hoefler, Torsten Swiss Fed Inst Technol Dept Comp Sci CH-8092 Zurich Switzerland Beijing Univ Posts & Telecommun Sch Comp Sci Beijing 100876 Peoples R China OneFlow Beijing 100004 Peoples R China

Recent advances in deep learning are driven by the growing scale of computation, data, and models. However, efficiently training large-scale models on distributed systems requires an intricate combination of data, operator, and pipeline parallelism, which exerts heavy burden on machine learning practitioners. To this end, we propose AutoDDL, a distributed training framework that automatically explores and exploits new parallelization schemes with near-optimal bandwidth cost. AutoDDL facilitates the description and implementation of different schemes by utilizing OneFlow's Split, Broadcast, and Partial Sum (SBP) abstraction. AutoDDL is equipped with an analytical performance model combined with a customized Coordinate Descent algorithm, which significantly reduces the scheme searching overhead. We conduct evaluations on Multi-Node-Single-GPU and Multi-Node-Multi-GPU machines using different models, including VGG and Transformer. Compared to the expert-optimized implementations, AutoDDL reduces the end-to-end training time by up to 31.1% and 10% for Transformer and up to 17.7% and 71.5% for VGG on the two parallel systems, respectively.

关键词： Parallel processing Costs Deep learning Training Tensors Computational modeling Pipelines Distributed deep learning operator parallelism data parallelism pipeline parallelism

来源：评论

学校读者我要写书评

暂无评论

Elastic Scaling of Stateful operators Over Fluctuating Data Streams

引用

IEEE TRANSACTIONS ON SERVICES COMPUTING 2024年第6期17卷 3555-3568页

作者： Wu, Minghui Sun, Dawei Gao, Shang Li, Keqin Buyya, Rajkumar China Univ Geosci Sch Informat Engn Beijing 100083 Peoples R China Deakin Univ Sch Informat Technol Waurn Ponds Vic 3216 Australia SUNY Coll New Paltz Dept Comp Sci New Paltz NY 12561 USA Univ Melbourne Sch Comp & Informat Syst Cloud Comp & Distributed Syst CLOUDS Lab Melbourne Vic 3010 Australia

Elastic scaling of parallel operators has emerged as a powerful approach to reduce response time in stream applications with fluctuating inputs. Many state-of-the-art works focus on stateless operators and change the operator parallelism from one aspect. They often lack efficient management of operator states and overlook the costs associated with resource over-provisioning. To overcome these limitations, we introduce Es-Stream for elastic scaling of stateful operators over fluctuating data streams, which includes: 1) We observe that under-provisioning of operator parallelism leads to data pile-up, resulting in longer system latency, while over-provisioning of operator parallelism causes idle instances and additional resource consumption. 2) The Es-Stream system scales in two dimensions: the parallelism of operators and the number of resources. It dynamically adjusts operators to an optimal parallelism while scaling the resources used by the stream application. 3) When the parallelism of stateful operators changes, upstream operators backup downstream operators' state and cache the emitted data tuples at dynamic time intervals, ensuring the operator parallelism is adjusted in a low-overhead way. 4) Experimental results demonstrate that Es-Stream provides promising performance improvements, reducing the maximum system latency by 3x and saving the maximum state recovery time by 2x, compared to existing state-of-the-art works.

关键词： Streams Parallel processing Topology Resource management Data models Computational modeling System performance Distributed stream computing operator parallelism resource scaling state management stateful operator

来源：评论

学校读者我要写书评

暂无评论

An Optimal Model for Optimizing the Placement and parallelism of Data Stream Processing Applications on Cloud-Edge Computing 32

An Optimal Model for Optimizing the Placement and Parallelis...

引用

32nd IEEE International Symposium on Computer Architecture and High-Performance Computing (SBAC-PAD) / 11th Workshop on Applications for Multi-Core Architectures (WAMCA)

作者： de Souza, Felipe Rodrigo de Assuncao, Marcos Dias Caron, Eddy Veith, Alexandre da Silva Univ Lyon EnsL INRIA UCBLCNRSLIP Lyon 07 France Univ Toronto Toronto ON Canada

ISBN: (纸本)9781728199245

The Internet of Things has enabled many application scenarios where a large number of connected devices generate unbounded streams of data, often processed by data stream processing frameworks deployed in the cloud. Edge computing enables offloading processing from the cloud and placing it close to where the data is generated, thereby reducing the time to process data events and deployment costs. However, edge resources are more computationally constrained than their cloud counterparts, raising two interrelated issues, namely deciding on the parallelism of processing tasks (a.k.a. operators) and their mapping onto available resources. In this work, we formulate the scenario of operator placement and parallelism as an optimal mixed-integer linear programming problem. The proposed model is termed as Cloud-Edge data Stream Placement (CESP). Experimental results using discrete-event simulation demonstrate that CESP can achieve an end-to-end latency at least similar or equal to 80% and monetary costs at least similar or equal to 30% better than traditional cloud deployment.

关键词： Data Stream Processing operator Placement operator parallelism End-to-end Latency Edge Computing

来源：评论

学校读者我要写书评

暂无评论

Scalable Joint Optimization of Placement and parallelism of Data Stream Processing Applications on Cloud-Edge Infrastructure 18th

Scalable Joint Optimization of Placement and Parallelism of ...

引用

18th International Conference on Service-Oriented Computing (ICSOC)

作者： de Souza, Felipe Rodrigo Veith, Alexandre Da Silva de Assuncao, Marcos Dias Caron, Eddy Univ Lyon EnsL UCBL INRIACNRSLIP F-69342 Lyon 07 France Univ Toronto Toronto ON Canada

ISBN: (纸本)9783030653101;9783030653095

The Internet of Things has enabled many application scenarios where a large number of connected devices generate unbounded streams of data, often processed by data stream processing frameworks deployed in the cloud. Edge computing enables offloading processing from the cloud and placing it close to where the data is generated, whereby reducing both the time to process data events and deployment costs. However, edge resources are more computationally constrained than their cloud counterparts. This gives rise to two interrelated issues, namely deciding on the parallelism of processing tasks (a.k.a. operators) and their mapping onto available resources. In this work, we formulate the scenario of operator placement and parallelism as an optimal mixed integer linear programming problem. To overcome the issue of scalability with the optimal model, we devise a resource selection technique that reduces the number of resources evaluated during placement and parallelization decisions. Experimental results using discrete-event simulation demonstrate that the proposed model coupled with the resource selection technique is 94% faster than solving the optimal model alone, and it produces solutions that are only 12% worse than the optimal, yet it performs better than state-of-the-art approaches.

关键词： Data stream processing operator placement operator parallelism End-to-end latency Edge computing

来源：评论

学校读者我要写书评

暂无评论

Using Containers to Execute SQL Queries in a Cloud 11

Using Containers to Execute SQL Queries in a Cloud

引用

11th IEEE/ACM International Conference on Utility and Cloud Computing (UCC-Companion) / 5th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT)

作者： Holland, David Zhang, Weining Univ Texas San Antonio Dept Comp Sci San Antonio TX 78249 USA

ISBN: (纸本)9781728103594

Emergent software container technology is available on any cloud and opens up new opportunities to execute and scale data computationally intensive applications wherever data is located. However, many traditional databases hosted on clouds have not scaled well. In this paper, a framework and methodology to containerize a SQL query is presented, so that, a single SQL query can be scaled and executed by a network of cooperating containers, achieving operator parallelism and other significant performance gains. Experiments of a framework prototype are reported and compared to a real-world database control baseline. Preliminary results on a research cloud shows up to 3-orders of magnitude performance gain compared to running the same query on a single VM hosted baseline control database.

关键词： Containers SQL database query evaluation operator parallelism cloud computing

来源：评论

学校读者我要写书评

暂无评论

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines 21

Chimera: Efficiently Training Large-Scale Neural Networks wi...

引用

International Conference for High Performance Computing, Networking, Storage and Analysis (SC21)

作者： Li, Shigang Hoefler, Torsten Swiss Fed Inst Technol Zurich Switzerland

ISBN: (纸本)9781450384421

Training large deep learning models at scale is very challenging. This paper proposes Chimera, a novel pipeline parallelism scheme which combines bidirectional pipelines for efficiently training large-scale models. Chimera is a synchronous approach and therefore no loss of accuracy, which is more convergence-friendly than asynchronous approaches. Compared with the latest synchronous pipeline approach, Chimera reduces the number of bubbles by up to 50%;benefiting from the sophisticated scheduling of bidirectional pipelines, Chimera has a more balanced activation memory consumption. Evaluations are conducted on Transformer based language models. For a GPT-2 model with 1.3 billion parameters running on 2,048 GPU nodes of the Piz Daint supercomputer, Chimera improves the training throughput by 1.16x-2.34x over the state-of-the-art synchronous and asynchronous pipeline approaches.

关键词： distributed deep learning pipeline parallelism data parallelism operator parallelism model parallelism

来源：评论

学校读者我要写书评

暂无评论

Orchestrating scheduling, grouping and parallelism to enhance the performance of distributed stream computing system

引用

EXPERT SYSTEMS WITH APPLICATIONS 2024年 254卷

作者： Sun, Dawei Chen, Haiyang Gao, Shang Buyya, Rajkumar China Univ Geosci Sch Informat Engn Beijing 100083 Peoples R China Deakin Univ Sch Informat Technol Waurn Ponds Vic 3216 Australia Univ Melbourne Sch Comp & Informat Syst Cloud Comp & Distributed Syst CLOUDS Lab Parkville Australia

In a big data stream computing environment, the arrival rate of data streams usually fluctuates over time, posing a great challenge to the elasticity of system. The performance of stream computing system is crucial, especially when dealing with unbounded and fluctuating data streams. Most prior studies have primarily focused on one or two aspects to enable elasticity, often lacking prompt and comprehensive performance optimization. This limitation could lead to a tuning bottleneck, preventing the system's performance from consistently reaching its optimal state. Additionally, many stream computing systems are not intelligently adaptive in real time due to the challenges of manual parameter reconfiguration for fluctuating streams. To better address these issues, we propose a framework named Sgp-Stream, which orchestrates scheduling, grouping and parallelism (Sgp). To enhance the system performance. We conduct the following research: (1) Running experiments to evaluate the impact of different factors such as scheduling, grouping and parallelism on system performance. Results show that factors at a single level usually have an upper limit on tuning system performance, and better overall performance can be achieved by coordinating multi-level factors. (2) Establishing quantitative models for stream application that consider computational cost and communication cost, multi-dimensional featured data stream, data center resources, and latency & throughput performance. (3) Demonstrating the effectiveness of the proposed runtime-aware data stream grouping based on smooth weighted polling, elastic adaptive scheduling based on Linear Deterministic Greedy and elastic scaling strategy based on Gradient Descent in Sgp-Stream, for continuous performance optimization.(4) Evaluating the application latency, throughput and resource utilization objectives using a real -world elastic stream computing system and twitter data set. Experimental results show that, compared to existing stat

关键词： Stream application scheduling Stream grouping operator parallelism Stream computing Big data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：