osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

REST job submission


Hi to all,
in our ETL we need to call an external (REST) service once a job ends: we
extract informations about accumulators and we update the job status.
However this is only possible if using the CLI client: if we call the job
via the REST API o Web UI (that is very useful to decouple our UI from the
Flink cluster) then this is not possible, because the REST API cannot
execute any code after env.execute().
I think that this is a very huge limitation: first of all, when writing
(and debugging) a Flink job, you assume that you can call multiple times
execute() and use the returned JobExecutionResult.
In second instance, the binary client and the rest client behaves
differently (with the CLI client everything works as expected).

What do you think about this? Is this a bug or not?

PS: I think also that the REST client should not be aware of any jar or
class instance, it should just call the job manager with the proper class
name and jar id (plus other options of course).

Cheers,
Flavio