[jira] [Created] (FLINK-10205) Batch Job: InputSplit Fault tolerant for DataSourceTask
JIN SUN created FLINK-10205:
Summary: Batch Job: InputSplit Fault tolerant for DataSourceTask
Issue Type: Sub-task
Reporter: JIN SUN
Today DataSource Task pull InputSplits from JobManager to achieve better performance, however, when a DataSourceTask failed and rerun, it will not get the same splits as its previous version. this will introduce inconsistent result or even data corruption.
Furthermore, if there are two executions run at the same time (in batch scenario), this two executions should process same splits.
we need to fix the issue to make the inputs of a DataSourceTask deterministic. The propose is save all splits into ExecutionVertex and DataSourceTask will pull split from there.
This message was sent by Atlassian JIRA