osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Flink join stream where one stream is coming 5 minutes late


Hi Abhijeet,

If you want to perform window-join in the DataStream API, the window configurations on both sides must be exactly the same.

For your case, maybe you can try adding a 5 mins delay on event times (and watermarks) of the faster stream. 

Hope that helps.

Best,
Xingcan


On Nov 26, 2018, at 12:44 PM, Abhijeet Kumar <abhijeet.kumar@xxxxxxxxxxxx> wrote:

I already mentioned above, one is coming late by 5 mins. Maybe my approach isn't correct, and I asked to correct me if I'm wrong. 

On 26-Nov-2018, at 5:56 PM, Taher Koitawala <taher.koitawala@xxxxxxxxx> wrote:

May I ask why you want to have 2 differences between window time? What's the use case?

On Mon 26 Nov, 2018, 5:53 PM Abhijeet Kumar <abhijeet.kumar@xxxxxxxxxxxx wrote:
Hello Team,

I've to join two stream where one stream is coming late. So, I planned doing it by creating two windows, for first window the size will be 5 minutes and for the other it will be 10 minutes. Then, I'll perform the join operation. Is my solution correct because I don't know whether one window will sync with other or not because one is getting finished earlier and another is late. If there is a better way to achieve the same, please let me know. 

Thanks,


Abhijeet Kumar
Software Development Engineer,
Sentienz Solutions Pvt Ltd
Cognitive Data Platform - Perceive the Data !