osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Buffer congestion in a downstream operator hangs SourceContext.collect in an (in?)finite loop


Hello,

I have a scenario, where FlinkKafkaProducer instance isn't able to write to a certain partition (the broker for the partition is down, or whatever), it's buffer eventually becomes full, and the whole topology freezes, because the root source (nonparallel) get's blocked on it, however it could still serve other branches/partitions.

Please see https://stackoverflow.com/questions/50667499/how-to-keep-broadcasting-from-non-parallel-source-when-one-of-the-downstreams-ha for more details. 

What can be wrong, could you suggest anything?

Respectfully,
Oleh.