osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Introduce Barriers in stream source


Hi,

I am implementing a source and I want to use checkpointing and would like to restore the job from these external checkpoints. I used Kafka for my tests and it worked fine.

However, I would like to know if I have my own source what do I need to do. I am sure that I will need to implement CheckpointedFunctions (initializeState and snapshotState) . Based on what I read and looked at implemetation of Kafka source I do not need to do anything else apart from this.
Maybe notifyCheckpointComplete also.

But I would like to confirm if I will need to implement something to create Barrier in my source.

Thanks