Re: Regarding airflow scheduler starting PID periodically.
I am not sure if I understood... you want to trigger DAGs based on the PIDs
of the tasks?
As you say (I am not an expert on OSes here) you don't control this.
I think that the TriggerDagRunOperator (https://airflow.apache.org/co
de.html#airflow.operators.TriggerDagRunOperator) would work for you. Right?
On Tue, May 8, 2018 at 5:06 PM nitish sharma <nitishcse412@xxxxxxxxx> wrote:
> Hi Team,
> I have a situation where i want to create dag with unique dag id and run
> that dag once as soon as it is created.
> For eg : 1.py , 2.py are two different dags. 1.py (has 3 tasks) and
> 2.py(has 2 tasks).
> Once these are run i would have next set of dags with me for eg 3.py, 4.py,
> 5.py and so on.
> I see a challenger here as scheduler keeps generating process ID to
> generate tasks