[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: TriggerDagRunOperator sub tasks are scheduled to run after few hours


I cannot connect this to timezone issues yet. The trigger dagrun operator seems to work fine (checked the examples).

B.

> On 10 Sep 2018, at 08:19, Deng Xiaodong <xd.deng.r@xxxxxxxxx> wrote:
> 
> Hi Goutam,
> 
> Seems it’s due to the timezone setting?
> 
> If that’s the case, you may try either adjusting your timezone setting or
> “manually” adjusting using execution_date argument in TriggerDagRunOperator.
> 
> 
> XD
> 
> On Mon, Sep 10, 2018 at 14:00 Goutam Kumar Sahoo <
> goutamkumar.sahoo@xxxxxxxxxxx> wrote:
> 
>> HI All,
>> 
>> 
>> 
>> Could anyone please address this issue ? We are really stuck here and not
>> able to proceed further
>> 
>> 
>> 
>> Thanks & Regards
>> 
>> Goutam Sahoo
>> 
>> 
>> 
>> *From:* Goutam Kumar Sahoo
>> *Sent:* Friday, September 07, 2018 4:51 PM
>> *To:* 'dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx' <dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>;
>> 'dev@xxxxxxxxxxxxxxxxxx' <dev@xxxxxxxxxxxxxxxxxx>
>> *Subject:* TriggerDagRunOperator sub tasks are scheduled to run after few
>> hours
>> 
>> 
>> 
>> HI Experts
>> 
>> 
>> 
>> In our project, we are trying to replicate the existing  job scheduling
>> implemented in Microsoft Orchestrater to new scheduler called “Apache
>> Airflow” .
>> 
>> 
>> 
>> During this replication process we have an requirement to create Master
>> DAG which in turn will call Other child DAGs based on different condition.
>> We referred to the existing example (example_trigger_controller_dag,
>> example_trigger_target_dag) available in Airflow and tried running them
>> manually.
>> 
>> 
>> 
>> When we triggered the controller DAG manually , it is got triggered
>> immediately with execution time created in local timezone (PDT) whereas the
>> child DAG is scheduled to run few hours later. Even though child DAGRUN is
>> showing as running, it is doing nothing and waiting for the time to
>> satisfy.  Our requirement is to trigger the Target/Child DAG run as soon it
>> is triggered by controller DAG
>> 
>> 
>> 
>> 
>> 
>> If you see here , the trigger time and execution time is having a 7 hours
>> GAP.
>> 
>> 
>> 
>> We have tried to solve this many different ways but couldn’t succeed. We
>> have asked for help in many forums but didn’t get any satisfactory answer
>> form any of them
>> 
>> 
>> 
>> 
>> 
>> Please help us ASAP as this issue is one of the show stopper for us .
>> 
>> 
>> 
>> 
>> 
>> 
>>