[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [QUESTION] retrial in sources by the runners

Runners are responsible for retry semantics, they should catch the failure and choose whether they want to retry or not. I think your reading of the code is correct.

Some I/O layers do retry but that is more about attempting to continue processing within a bundle instead of failing and having the runner choose to retry the bundle or not.

On Thu, Aug 30, 2018 at 5:57 AM Etienne Chauchot <echauchot@xxxxxxxxxx> wrote:
Hi all,

I have a question concerning retrial of sources. I've looked at the code of direct runner and spark runner on bounded sources. As far as I can tell, if there is a failure in reading a record from the reader of the source, there will be no retrial from the runner, there will just be an exception thrown. The only retrial that can take place with the source is the one that could be done by the reader itself.

Can you confirm that I did not miss something?