Latency-aware placement of stream processing operators in modern-day stream processing frameworks

The rise of the Internet of Things has substantially increased the number of interconnected devices at the edge of the network. As a result, a large number of computations are now distributed in the compute continuum, spanning from the edge to the cloud, generating vast amounts of data. Stream processing is typically employed to process this data in near real-time due to its efficiency in handling continuous streams of information in a scalable manner. However, many stream processing approaches do not consider the underlying network devices of the compute continuum as candidate resources for processing data. Moreover, many existing works do not consider the incurred network latency of performing computations on multiple devices in a distributed way. To avoid this, we formulate an optimization problem for utilizing the complete compute continuum resources and design heuristics to solve this problem efficiently. Furthermore, we integrate our heuristics into Apache Storm and perform experiments that show latency- and throughput-related benefits compared to alternatives.

Subjects

Apache storm | Compute continuum | Data stream processing | Edge computing | Internet of Things

DDC Class

004: Computer Sciences

620.3: Vibrations

621.3: Electrical Engineering, Electronic Engineering

519: Applied Mathematics, Probabilities

Publication version

publishedVersion

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

1-s2.0-S0743731525000085-main.pdf

Type

Main Article

Size

1.8 MB

Format

Adobe PDF

Options

Latency-aware placement of stream processing operators in modern-day stream processing frameworks