OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Understanding checkpoint behavior


Thank you for your response Piotr. I plan to upgrade to Flink 1.5.x early
next year.

Two follow-up questions for now. 

1. 
" When operator snapshots are taken, there are two parts: the synchronous
and the asynchronous parts. "
I understand that when the operator snapshot is being taken, the processing
of that operator is stopped as taking this snapshot is synchronous part. Is
there any other synchronous part in the snapshot / checkpoint process?


2. 
Based on the test I mentioned above, my understanding is that for a window
operator, when all events that belongs to checkpoint N and the checkpoint
barrier N are received by window operator (but pending for window to be
triggered), then checkpoint barrier N will be immediately emitted to the
sink operator (so snapshot can be completed) while the events are still
pending to be evaluated by window operator.

Can you please confirm my understanding as I was initially confused by the
following second statement (emits all pending outgoing records) under
Barriers section in this doc
https://ci.apache.org/projects/flink/flink-docs-release-1.2/internals/stream_checkpointing.html
?

"When an intermediate operator has received a barrier for snapshot n from
all of its input streams, it emits itself a barrier for snapshot n into all
of its outgoing streams."

" Once the last stream has received barrier n, the operator emits all
pending outgoing records, and then emits snapshot n barriers itself. "

 Thanks,
Pranjal



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/