Towards Fine Grain Resource Management for Stream Processing

Distributed Stream Processing engines such as Apache Flink handle increasing load using horizontal scaling. Each operator is assigned to a number of task slots. The homogeneous nature of task slots in terms of memory contrasts with the heterogeneous needs of operators and of their individual tasks. This thesis proposes to study the integration of fine-grained heterogeneous resource allocation in scaling policies applied to Flink over Kubernetes.