Re: About how to pause the running task
There are no semantics or concept of pause within a task, though you can
clear/kill tasks which is essentially sending a poison pill to kill the
Even if BaseOperator (common to all tasks) was to implement pausing
semantics, they probably wouldn't be implemented for most operators. Is it
even possible to pause a bash script externally (through SIGTSTP
<https://en.wikipedia.org/wiki/SIGTSTP>, but does that always work)? a
python script? Hive job? A Spark job? Personally I don't think that feature
would get much use. You end up freezing and not releasing a worker slot.
You can pause a DAG though. FYI when a DAG is paused, it only means it will
stop scheduling new tasks.
On Thu, Apr 26, 2018 at 10:14 AM, Song Liu <songliu@xxxxxxxxxxx> wrote:
> A DAG is composed of many tasks, when this DAG is started, how to pause
> the current running task ?