The existence of a programming error is often indicated by the occurrence of a dataflow anomaly. The detection of such anomalies can be used for error detection and the upgrading of software quality. A new, efficient...
详细信息
The existence of a programming error is often indicated by the occurrence of a dataflow anomaly. The detection of such anomalies can be used for error detection and the upgrading of software quality. A new, efficient algorithm is proposed that is capable of detecting anomalous dataflow patterns in a program represented by a graph. The algorithm based on static analysis scans the paths entering and leaving each node of the graph, thus revealing anomalous data action combinations. Fosdick and Osterweil (1976) proposed an algorithm implementing this type of approach. The proposed approach presents a general framework that not only fills a gap in the previous approach, but also offers both time and space improvements.
In a typical data pipeline, the dataflow starts from the first node, where the data is initiated, and moves to the last node in the pipeline, where the processed data will be stored. Due to the sheer number of involve...
详细信息
In a typical data pipeline, the dataflow starts from the first node, where the data is initiated, and moves to the last node in the pipeline, where the processed data will be stored. Due to the sheer number of involved participants, it is crucial to protect the dataflow integrity in the pipeline. While previous studies have outlined solutions to this matter, the solution for an untrusted data pipeline is still left unexplored, which motivates us to propose SIGNORA. Our proposal combines the concept of a chain of signatures with blockchain receipt to provide dataflow integrity. The chain of signatures provides a non-repudiation guarantee from participants, while the hash of the data and signatures is anchored in the blockchain for a non-tampering guarantee through blockchain receipt. Aside from that, SIGNORA also satisfies essential requirements of running data pipeline processing in an open and untrusted environment, such as (i) providing reliable identity management, (ii) solving the trust and accountability issues through a reputation system, (iii) supporting various devices through multiple cryptographic algorithms (i.e., ECDSA, EdDSA, RSA, and HMAC), and (iv) off-chain processing. Our experiment results show that SIGNORA can provide dataflow integrity provisioning in multiple scenarios of data payload size with reasonable overhead. Furthermore, the cost of smart contract methods has also been analyzed, and several off-chain solutions have been addressed to reduce transaction costs. Finally, the reputation system can adapt to the history of nodes' activities by increasing their scores when they actively perform honest behavior while reducing their scores when they become inactive. Therefore, SIGNORA can provide a high degree of accountability for participants collaborating in an untrusted environment.
Managing privacy in the IoT presents a significant challenge. We make the case that information obtained by auditing the flows of data can assist in demonstrating that the systems handling personal data satisfy regula...
详细信息
Managing privacy in the IoT presents a significant challenge. We make the case that information obtained by auditing the flows of data can assist in demonstrating that the systems handling personal data satisfy regulatory and user requirements. Thus, components handling personal data should be audited to demonstrate that their actions comply with all such policies and requirements. A valuable side-effect of this approach is that such an auditing process will highlight areas where technical enforcement has been incompletely or incorrectly specified. There is a clear role for technical assistance in aligning privacy policy enforcement mechanisms with data protection regulations. The first step necessary in producing technology to accomplish this alignment is to gather evidence of dataflows. We describe our work producing, representing and querying audit data and discuss outstanding challenges.
NASA's Goddard Earth Sciences data and Information Services Center has developed the Goddard Interactive Online Visualization ANd aNalysis Infrastructure or "Giovanni," an asynchronous Web-service-based ...
详细信息
NASA's Goddard Earth Sciences data and Information Services Center has developed the Goddard Interactive Online Visualization ANd aNalysis Infrastructure or "Giovanni," an asynchronous Web-service-based workflow management system for Earth science data. Giovanni has been providing an intuitive and responsive interface for visualizing, analyzing, and intercomparing multisensor data using only a Web browser to scientists and other users. Giovanni supports many types of single- and multiparameter visualizations and statistical analyses. The interface also provides users with capabilities for downloading images and data in multiple formats. Giovanni supports open and standard data protocols and formats. Finally, Giovanni provides users with a data lineage that describes, in detail, the algorithms used in processing the data including caveats and other scientifically pertinent information.
The concept of data flow computing is applied to digital signal processing (DSP). A dataflow signal processor (DFSP) architecture is presented. The principles of data flow computing are carefully considered in order ...
详细信息
The concept of data flow computing is applied to digital signal processing (DSP). A dataflow signal processor (DFSP) architecture is presented. The principles of data flow computing are carefully considered in order to conform with the special properties of DSP. The bus oriented architecture is easily configured to meet various performance requirements. The DFSP architecture is most suitable for nonrecursive algorithms. Typical tasks of this nature are transforms and FIR filters. A simulation model of the DFSP architecture has been developed. Simulation results of two application examples are given.
We describe the timely dataflow model for distributed computation and its implementation in the Naiad system. The model supports stateful iterative and incremental computations. It enables both low-latency stream proc...
详细信息
We describe the timely dataflow model for distributed computation and its implementation in the Naiad system. The model supports stateful iterative and incremental computations. It enables both low-latency stream processing and high-throughput batch processing, using a new approach to coordination that combines asynchronous and fine-grained synchronous execution. We describe two of the programming frameworks built on Naiad: GraphLINQ for parallel graph processing, and differential dataflow for nested iterative and incremental computations. We show that a general-purpose system can achieve performance that matches, and sometimes exceeds, that of specialized systems.
Rate-optimal scheduling of iterative data-flow graphs requires the computation of the iteration period bound. According to the formal definition, the total computational delay in each directed loop in the graph has to...
详细信息
Rate-optimal scheduling of iterative data-flow graphs requires the computation of the iteration period bound. According to the formal definition, the total computational delay in each directed loop in the graph has to be calculated in order to determine that bound. As the number of loops cannot be expressed as a polynomial function of the number of nodes in the graph, this definition cannot be the basis of an efficient algorithm. This paper presents a polynomial-time algorithm for the computation of the iteration period bound based on longest path matrices and their multiplications.
The Cheeps Imaging System is a compact, modular platform for acquisition, processing, and display of digital video sequences and model-based representations of moving scenes, and is intended as both a laboratory tool ...
详细信息
The Cheeps Imaging System is a compact, modular platform for acquisition, processing, and display of digital video sequences and model-based representations of moving scenes, and is intended as both a laboratory tool and a prototype architecture for future programmable video decoders. Rather than using a large number of general-purpose processors and dividing up image processing tasks spatially, Cheeps abstracts out a set of basic, computationally intensive stream operations that may be performed in parallel and embodies them in specialized hardware. We review the Cheeps architecture, describe the software system that has been developed to perform resource management, and present the results of some performance tests.
This article presents a framework based on a flow-based programming (FBP) paradigm to design data-stream processing applications for Nuclear Physics (NP). The developed framework encourages a functional decomposition ...
详细信息
This article presents a framework based on a flow-based programming (FBP) paradigm to design data-stream processing applications for Nuclear Physics (NP). The developed framework encourages a functional decomposition of the overall data-processing application into small monofunctional artifacts that are easy to understand, develop, and debug. The fact that these artifacts (actors) are programmatically independent means that they can be scaled and optimized independently, which is difficult for monolithic application components. One of the advantages of this approach is fault tolerance, where independent actors can come and go in the data stream without stopping or crashing the entire application. Because actors are loosely coupled and data carries context, they can run in heterogeneous environments and utilize wide-ranging accelerators. This article describes the main design concepts of this framework, presenting a proof-of-concept application and the results of processing on-beam calorimeter streaming data.
暂无评论