Real-time Batch Scheduling in Data-Parallel Complex Event Processing

Titelangaben

Mayer, Ruben ; Tariq, Muhammad Adnan ; Rothermel, Kurt:
Real-time Batch Scheduling in Data-Parallel Complex Event Processing.
University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology
Stuttgart , 2016 . - 14 S. - (Technical Report Computer Science ; 2016/04 )

Volltext

Link zum Volltext (externe URL):

Abstract

Distributed Complex Event Processing has emerged as a well-established paradigm to detect situations of interest to an application from basic sensor streams, building an operator graph between sensors and applications. To enable operators to cope with high workload, the incoming data streams are split into---possibly overlapping---partitions which are processed in parallel by a set of operator instances. However, with increasing parallelization degree the network becomes a bottleneck, because events that are part of multiple different partitions are duplicated to multiple operator instances. In this paper, we address this problem and propose batch scheduling of overlapping partitions, i.e., assigning them to the same operator instance. Albeit reducing communication overhead, batch scheduling increases the processing latency of events---and thus inhibits the timely detection of situations---by inducing higher computational load on the operator instance. Controlling the trade-off between communication overhead and latency is challenging and cannot be solved with traditional reactive approaches. To this end, we propose an analytical batch scheduling controller building on prediction. Evaluations show that our approach is able to significantly save bandwidth and keep a latency bound in the operator instances.