osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Multiple stream operator watermark handling


Is there mechanism for a multiple stream operator to ignore watermarks from one of the streams?

The use case is a multiple stream operator that consumes a primary stream and a secondary control stream.  The control stream may only receive messages in rare occasion, and possibly never.  The default behavior of the operator is to only emit the lowest of the last watermark received from each input stream.  That means that event time fails to advance if there are no control messages.  

I also notice that FLIP-17, the Side Input proposal, does not address this issue, either in the Wiki or in the Google Docs.

Assuming there is no currently prescribed way to handle this, are folks taking care of this by introducing a new Assigner after the multiple input operator to generate watermarks?