OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Slide Window Compute Optimization


Hi Yennie,

AFAIK, the sliding window will in fact duplicate elements into multiple different streams. There's a discussion thread regarding this [1]. 
We are looking into some performance improvement, can you provide some more info regarding your use case?

--
Rong

[1] https://issues.apache.org/jira/browse/FLINK-7001

On Thu, Jul 5, 2018 at 3:30 AM Kostas Kloudas <k.kloudas@xxxxxxxxxxxxxxxxx> wrote:
Hi,

You are correct that with sliding windows you will have 3600 “open windows” at any point.
Could you describe a bit more what you want to do?

If you simply want to have an update of something like a counter every second, then you can 
implement your own logic with a ProcessFunction that allows to handle state and timers in a 
custom way (see [1]).

Hope this helps,
Kostas

 

On Jul 5, 2018, at 12:12 PM, YennieChen88 <chenyanying3@xxxxxx> wrote:

Hi,
   I want to use slide windows of 1 hour window size and 1 second step
size. I found that once a element arrives, it will be processed in 3600
windows serially through one thread. It takes serveral seconds to finish one
element processing,much more than my expection. Do I have any way to
optimizate it?
   Thank you very much for your reply.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/