检索结果-内蒙古大学图书馆

A dataflow programming Language and its Compiler for Streaming Systems

Procedia Computer Science 2014年 29卷 1289-1298页

作者： Haitao Wei Stéphane Zuckerman Xiaoming Li Guang R. Gao University of Delaware Newark DE U.S.A

The dataflow programming paradigm shows an important way to improve programming productivity for streaming systems. In this paper we propose COStream, a programming language based on synchronous data flow execution model for data-driven application. We also propose a compiler framework for COStream on general-purpose multi-core architectures. It features an inter-thread software pipelining scheduler to exploit the parallelism among the cores. We implemented the COStream compiler framework on x86 multi-core architecture and performed experiments to evaluate the system.

关键词： dataflow programming COStream Compiler Streaming

来源：评论

学校读者我要写书评

暂无评论

In-Line Compositional Visual programming 8

In-Line Compositional Visual Programming

引用

8th International Conference on the Art, Science, and Engineering of programming (programming)

作者： Homer, Michael Victoria Univ Wellington Wellington New Zealand

ISBN: (纸本)9798400706349

Concatenative programming inherently expresses composition of sub-tasks of a pipeline, but one uncommonly seen model of this paradigm includes all data values inline within the program. A visual environment for editing and evaluating programs in this model would inherently display state in place, and allow for easy tracing of data flow through the program by watching the values literally move as evaluation steps took place. We propose a visual approach for programming in this style, with function calls and data values interleaved on a single "track", with specific concrete arguments always adjacent when a function term is evaluated and various affordances for editing, evaluating, and debugging. We then show how extensions to this model to multiple tracks can ease programming in the model and even make available some more inscrutable programming-language features, such as concurrency and effect systems, in a more accessible way.

关键词： visual programming dataflow programming end-user programming concatenative programming

来源：评论

学校读者我要写书评

暂无评论

A Reactive System-specific Compilation Chain from Synchronous dataflow Models to FPGA Netlist 10

A Reactive System-specific Compilation Chain from Synchronou...

引用

10th International Conference on Space Mission Challenges for Information Technology

作者： Winandy, Ines Dion, Arnaud Garoche, Pierre-Loic Manni, Florent Univ Toulouse Federat ENAC ISAE SUPAERO ONERA Toulouse France CNES Ctr Natl Etud Spatiales Toulouse France

ISBN: (纸本)9798350384529;9798350384512

Modern Field Programmable Gate Arrays (FPGAs) offer a solution to several issues related to real-time on-board systems, such as guaranteed execution time. They are currently considered as target platforms for space applications. However, the complexity of producing circuits on these components poses a challenge to their widespread adoption. To address this issue, high-level synthesis tools provide another layer of abstraction above the logic circuit design process, for example compiling C code into Hardware Description Languages such as VHDL or Verilog. However, high-level synthesis results are poorly predictable and do not guarantee the efficient use of recent FPGA capabilities provided by new primitives like digital signal processor or random-access memory. In this paper we propose a compilation chain dedicated to reactive systems, ie. controllers, providing a more predictable synthesis process for critical embedded control applications. The implemented solution demonstrates timing performance equivalent to the traditional synthesis process with a more predictable result.

关键词： High Level Synthesis (HLS) Embedded systems Guidance Navigation and Control (GNC) dataflow programming Computer-aided Design (CAD)

来源：评论

学校读者我要写书评

暂无评论

VeilGraph: incremental graph stream processing

引用

JOURNAL OF BIG DATA 2022年第1期9卷 1-29页

作者： Coimbra, Miguel E. Esteves, Sergio Francisco, Alexandre P. Veiga, Luis Univ Lisbon INESC ID IST Rua Alves Redol 9 P-1000029 Lisbon Portugal

Graphs are found in a plethora of domains, including online social networks, the World Wide Web and the study of epidemics, to name a few. With the advent of greater volumes of information and the need for continuously updated results under temporal constraints, it is necessary to explore alternative approaches that further enable performance improvements. In the scope of stream processing over graphs, we research the trade-offs between result accuracy and the speedup of approximate computation techniques. The relationships between the frequency of graph algorithm execution, the update rate and the type of update play an important role in applying these techniques. Herein we present VeilGraph, through which we conducted our research. We showcase an innovative model for approximate graph processing implemented in Apache Flink. We analyse the feasibility of our model and evaluate it with the case study of the PageRank algorithm, the most famous measure of vertex centrality used to rank websites in search engine results. Our experiments show that VeilGraph can often reduce latency closely to half (speedup of 2.0x), while achieving result quality above 95% when compared to results of the traditional version of PageRank executing in Apache Flink with Gelly (i.e. without any summarization or approximation techniques). In some cases, depending on the workload, speedups against Apache Flink reach up to 3.0x (i.e. yielding a reduction of up to 66% in latency). We have found VeilGraph implementation on Flink to be scalable, as it is able to improve performance up to 10X speedups, when more resources are employed (16 workers), achieving better speedups with scale for larger graphs, which are the most relevant.

关键词： Graph processing Approximate processing Stream processing Summarization dataflow programming Distributed computation

来源：评论

学校读者我要写书评

暂无评论

Parsl+CWL: Towards Combining the Python and CWL Ecosystems

Parsl+CWL: Towards Combining the Python and CWL Ecosystems

引用

2024 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC Workshops 2024

作者： Karle, Nishchay Clifford, Ben Babuji, Yadu Chard, Ryan Katz, Daniel S. Chard, Kyle University of Chicago Department of Computer Science Chicago United States Argonne National Laboratory Lemont United States University of Illinois Urbana-Champaign NCSA & CS & iSchool UrbanaIL United States

ISBN: (纸本)9798350355543

The Common Workflow Language (CWL) is a widely adopted language for defining and sharing computational workflows. It is designed to be independent of the execution engine on which workflows are executed. In this paper, we describe our experiences integrating CWL with Parsl, a Python-based parallel programming library designed to manage execution of workflows across diverse computing environments. We propose a new method that converts CWL CommandLineTool definitions into Parsl apps, enabling Parsl scripts to easily import and use tools represented in CWL. We describe a Parsl runner that is capable of executing a CWL CommandLineTool directly. We also describe a proof-of-concept extension to support inline Python in a CWL workflow definition, enabling seamless use in Parsl's Python ecosystem. We demonstrate the benefits of this integration by presenting example CWL CommandLineTool definitions that show how they can be used in Parsl, and comparing performance of executing an image processing workflow using the Parsl integration and other CWL runners. © 2024 IEEE.

关键词： commandlinetool common workflow language common workflow language (cwl) computational reproducibility cross-platform execution cwl dataflow programming dynamic workflows high performance computing (hpc) inline expressions parallel computing parsl python programming resource management scalability scientific workflows workflow workflow automation workflow ecosystem workflow execution workflow integration workflow management systems workflow portability

来源：评论

学校读者我要写书评

暂无评论

Branching Compositional Data Transformations in jq, Visually 2

Branching Compositional Data Transformations in jq, Visually

引用

2nd ACM SIGPLAN International Workshop on programming Abstractions and Interactive Notations, Tools, and Environments (PAINT)

作者： Homer, Michael Victoria Univ Wellington Wellington New Zealand

ISBN: (纸本)9798400703997

jq is a widely-used command-line tool for filtering and transforming JSON data, in the vein of sed and awk, including a bespoke programming language for writing the filters. The paradigm of that language is unusual: while its appearance is somewhere between a shell pipeline and JavaScript, the language is pure-functional and essentially concatenative, the pipelines branch and interleave data invisibly, implicit output flattening obscures all these effects, and most users are unaware of any of its semantics, leading to confusion when encountering any of these latent elements and difficulty in constructing appropriate non-trivial filters, all the while common debugging techniques are also obstructed by the paradigm. These confusions can be eliminated by visually demonstrating the recursively forking nature of evaluation on actual data, and allowing manipulations of the program or data to be reflected live, but no such tool exists for jq or any similar approaches. We present a visualisation of jq's execution model that makes the branching nature manifest, illustrating its effects on concrete values provided by the user, and editing affordances that allow manipulating the program with reference to real data.

关键词： jq dataflow programming object pipelines stream processing

来源：评论

学校读者我要写书评

暂无评论

Targeting a light-weight and multi-channel approach for distributed stream processing

引用

JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING 2022年 167卷 77-96页

作者： Venugopal, Vinu Ellampallil Theobald, Martin Tassetti, Damien Chaychi, Samira Tawakuli, Amal Int Inst Informat Technol Bangalore India Univ Luxembourg Esch Sur Alzette Luxembourg Univ Nantes Nantes France

Processing high-throughput data-streams has become a major challenge in areas such as real-time event monitoring, complex dataflow processing, and big data analytics. While there has been tremendous progress in distributed stream processing systems in the past few years, the high-throughput and low latency (a.k.a. high sustainable-throughput) requirement of modern applications is pushing the limits of traditional data processing infrastructures. This paper introduces a new distributed stream processing engine (DSPE), called Asynchronous Iterative Routing (or simply "AIR "), which implements a light-weight, dynamic sharding protocol. AIR expedites direct and asynchronous communication among all the worker nodes via a channel-like communication protocol on top of the Message Passing Interface (MPI), thereby completely avoiding the need for a dedicated driver node. The system adopts a new progress-tracking protocol, called hew-meld, which has been experimentally observed to show a low processing latency on our asynchronous master-less architecture when compared to the conventional low-watermark technique. The current version of AIR is also equipped with two fault tolerance and recovery strategies namely checkpointing & rollback and replication. With its unique design, AIR scales out particularly well to multi core HPC architectures;specifically, we deployed it on clusters with up to 16 nodes and 448 cores (thus reaching a peak of 435.3 million events and 55.14 GB of data processed per second), which we found to significantly outperform existing DSPEs. (c) 2022 Elsevier Inc. All rights reserved.

关键词： Distributed stream processing systems dataflow programming Asynchronous iterative routing Stateful windowed operators

来源：评论

学校读者我要写书评

暂无评论

Domain-Specific Visual Language for Data Engineering Quality 1

Domain-Specific Visual Language for Data Engineering Quality

引用

1st International Workshop on programming Abstractions and Interactive Notations, Tools, and Environments (PAINT)

作者： De Meo, Alexis Homer, Michael Trove Wellington New Zealand Victoria Univ Wellington Sch Engn & Comp Sci Wellington New Zealand

ISBN: (纸本)9781450399104

Data engineering pipelines process large amounts of information, and ensuring that the quality and integrity of the data is maintained throughout is critical for technical, business, and social reasons. Conventional data quality assurance approaches require a large amount of fine-grained testing code, which is laborious, easy to get out of sync, and inscrutable to non-technical stakeholders. An executable higher-level visual approach to expressing quality requirements can serve as a shared representation of these constraints and their implications for all parties, eliminating repetition while increasing accessibility and maintainability. We present a visual programming language for expressing data quality requirements within a pipeline declaratively, structured as a diagram of compositional data flow, transformation, and validation steps.

关键词： data engineering visual programming dataflow programming

来源：评论

学校读者我要写书评

暂无评论

Interleaved 2D Notation for Concatenative programming 1

Interleaved 2D Notation for Concatenative Programming

引用

1st International Workshop on programming Abstractions and Interactive Notations, Tools, and Environments (PAINT)

作者： Homer, Michael Victoria Univ Wellington Sch Engn & Comp Sci Wellington New Zealand

ISBN: (纸本)9781450399104

Concatenative languages use implicit argument passing to provide a concise expression of programs comprising many composed transformation functions. However, they are sometimes regarded as "write-only" languages because understanding code requires mentally simulating the manipulations of the argument stack to identify where values are produced and consumed. All of this difficulty can be avoided with a notation that presents both the functions and their operands simultaneously, which can also ease editing by making available values and functions directly apparent. This paper presents a two-dimensional notation for these programs, comprising alternating rows of functions and operands with arguments and return values indicated by physical layout, and a tool for interactive live editing of programs in this notation.

关键词： concatenative programming visual programming dataflow programming

来源：评论

学校读者我要写书评

暂无评论

Calling Cards: Concrete Visual End-User programming 22

Calling Cards: Concrete Visual End-User Programming

引用

6th International Conference on the Art, Science, and Engineering of programming (programming)

作者： Homer, Michael Victoria Univ Wellington Sch Engn & Comp Sci Wellington New Zealand

ISBN: (纸本)9781450396561

Creating a program that performs even a simple task and shows the result is unapproachable to most people, and even trained programmers face a burden to create a new program. In this paper, we introduce a prototype system and model for live visual dataflow programming where intermediate steps are visible and all components are tangible and manifest. Our system aims to allow a user to dive in and immediately have a working program that can be incrementally extended.

关键词： end-user programming exploratory programming visual programming dataflow programming

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：