Pattern detection is an active field in big data streams analytics with numerous ongoing challenges. Actually, due to the great velocity and variety of data, new patterns can appear and change over time. Existing stat...
详细信息
ISBN:
(数字)9781510635784
ISBN:
(纸本)9781510635784
Pattern detection is an active field in big data streams analytics with numerous ongoing challenges. Actually, due to the great velocity and variety of data, new patterns can appear and change over time. Existing state-ofthe-art solutions consist in updating the pattern detection model regularly in order to integrate newly appeared and validated patterns. However, in several applications, such as security and defense, patterns can represent anomalies. Therefore, it becomes crucial to detect new patterns (i.e. new anomalies), as early as possible, in order to react at the right moment. Consequently, emergent pattern detection becomes a very challenging task. To tackle this challenge, we propose EPDA (Emergent Pattern Detection Algorithm): a new and validated algorithm for detecting emergent patterns in data streams. The originality of EPDA consists in exploiting frequent pattern mining techniques by proposing new statistical measures in order to estimate the evolution of emergent patterns over time. To perform this detection in a real-time, EPDA runs on the well-known Apache STORM distributed real-time computation system. To better fit our algorithm, we propose a new Apache STORM topology which is composed of one Spouts level and two Bolts levels. Experiments on a real data stream have shown the relevance of the proposed measures and the efficiency of our algorithm in a prediction task and in terms of execution time.
As real-time systems are becoming increasingly distributed, it becomes important to understand their structural robustness with respect to timing uncertainty. Structural robustness, a concept that arises by virtue of ...
详细信息
ISBN:
(纸本)9780769542980
As real-time systems are becoming increasingly distributed, it becomes important to understand their structural robustness with respect to timing uncertainty. Structural robustness, a concept that arises by virtue of multi-stage execution, refers to the robustness of end-to-end timing behavior of an execution graph towards unexpected timing violations in individual execution stages. A robust topology is one where such violations minimally affect end-to-end execution delay. The paper shows that the manner in which resources are allocated to execution stages can make a difference in robustness. Algorithms are presented and evaluated for resource allocation that improve the robustness of execution graphs. Evaluation shows that such algorithms are able to significantly reduce deadline misses due to unpredictable timing violations. Hence, the approach is important for soft real-time systems, systems where timing uncertainty exists, or where worst-case timing is not entirely verified.
Based on system execution traces, this paper presents a dynamic approach for visualizing and debugging timing constraint violations occurring in distributed real-time systems. The system execution traces used for visu...
详细信息
Based on system execution traces, this paper presents a dynamic approach for visualizing and debugging timing constraint violations occurring in distributed real-time systems. The system execution traces used for visualization and debugging are collected during the execution of a target program in such a way that its run-time behavior is not interfered with. This is made possible by our non-interference distributed real-time monitoring system which is capable of collecting system's run-time traces by monitoring and fetching the data passing through the internal buses of a target system. After the run-time data has been collected, the visualization and debugging activities then proceeded. The timing behavior of a target program is visualized as two graphs - the Colored Process Interaction Graph (CPIG) and the Dedicated Colored Process Interaction Graph (DCPIG). The CPIG depicts the timing behavior of a target program by graphically representing interprocess relationships during their communication and synchronization. The DCPIG can reduce visualization and debugging complexity by focusing on the portion of a target program which has direct or indirect correspondence with an imposed timing constraint. With the help of the CPIG and the DCPIG, a timing analysis method is used for computing the system-related timing statistics and analyzing the causes of timing constraint violations. A visualization and debugging system, called VDS, has been implemented using Open Windows on Sun-4's/UNIX workstations.
暂无评论