osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Flink 1.5 + resource elasticity resulting in overloaded workers


Hi, We recently upgraded to flink 1.6 and seem to be suffering from the issue described in this email

http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Flink-1-5-job-distribution-over-cluster-nodes-tt23364.html

Our workers have 8 slots and some workers are fully loaded and as a consequence get to cope with heavy load during peak times. while other workers sit completely idle. and have 0 jobs assigned to its slots

Is there any workaround for this . short of reducing the number of slots on a worker ? 

We need to have double the slots we need available in order to cope with availability zone maintenance . So if we where to reduce the number of slots we'd have to add new nodes that would then mostly sit idle