OSDir

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Regarding airflow scheduler starting PID periodically.


If it is different than the TriggerDagRunOperator idea that Guillermo
mentioned, could you elaborate on your specific use case?

On Thu, May 10, 2018 at 2:40 AM Guillermo Rodríguez Cano <wschutz@xxxxxxxxx>
wrote:

> Hello,
>
> I am not sure if I understood... you want to trigger DAGs based on the PIDs
> of the tasks?
>
> As you say (I am not an expert on OSes here) you don't control this.
>
> I think that the TriggerDagRunOperator (https://airflow.apache.org/co
> de.html#airflow.operators.TriggerDagRunOperator
> <https://airflow.apache.org/code.html#airflow.operators.TriggerDagRunOperator>)
> would work for you. Right?
>
> /Guillermo
>
>
> On Tue, May 8, 2018 at 5:06 PM nitish sharma <nitishcse412@xxxxxxxxx>
> wrote:
>
> > Hi Team,
> >
> > I have a situation where i want to create dag with unique dag id and run
> > that dag once as soon as it is created.
> >
> > For eg :  1.py , 2.py are two different dags. 1.py (has 3 tasks) and
> > 2.py(has 2 tasks).
> >
> > Once these are run i would have next set of dags with me for eg 3.py,
> 4.py,
> > 5.py and so on.
> >
> > I see a challenger here as scheduler keeps generating process ID to
> > generate tasks
> >
> >
>
-- 
*Taylor Edmiston*
Blog <http://blog.tedmiston.com> | Stack Overflow CV
<https://stackoverflow.com/story/taylor> | LinkedIn
<https://www.linkedin.com/in/tedmiston/> | AngelList
<https://angel.co/taylor>