Re: Disable Processing of DAG file
This config line
controls how often the scheduler scan the DAG folder and tries to discover/
For doing dag file processing part, scheduler does parse the DAG file
everytime before it schedules tasks through DagFileProcessor.
On Mon, May 28, 2018 at 10:14 PM, Ananth Durai <vananth22@xxxxxxxxx> wrote:
> It is an interesting question. On a slightly related note, Correct me if
> I'm wrong, AFAIK we require restarting airflow scheduler in order pick any
> new DAG file changes by the scheduler. In that case, should the scheduler
> do the DAGFileProcessing every time before scheduling the tasks?
> On 28 May 2018 at 21:46, ramandumcs@xxxxxxxxx <ramandumcs@xxxxxxxxx>
> > Hi All,
> > We have a use case where there would be 100(s) of DAG files with schedule
> > set to "@once". Currently it seems that scheduler processes each and
> > file and creates a Dag Object.
> > Is there a way or config to tell scheduler to stop processing certain
> > files.
> > Thanks,
> > Raman Gupta