[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Single Airflow Instance Vs Multiple Airflow Instance


We incorportated multi-tentacy in how we deploy dags. 

With version 0.9 you can have the following structure:
DAG_HOME/
 Project1/
  - Dag_1
  - Lib_for_dag_1
 Project2/
  - Dag_1
  - lib_for_dag_1

So that the team owning Project1 can independently and without inferring with others deploy (Project2) their own dags. If you incorporate this structure in your deployment process you could do this.

It works fine. 

-----Original Message-----
From: gauthiermartin86@ <gmail.com gauthiermartin86@xxxxxxxxx> 
Sent: Wednesday, June 6, 2018 4:24 PM
To: dev@xxxxxxxxxxxxxxxxxx
Subject: Single Airflow Instance Vs Multiple Airflow Instance

Hi Everyone, 

We have been experimenting with airflow for about 6 months now.
We are planning to have multiple departments to use it. Since we don't have any internal experience with Airflow we are wondering if single instance per department is more suited than single instance with multi-tenancy? We have been aware about the upcoming release of airflow 1.10 and changes that will be made to the RBAC which will be more suited for multi-tenancy.

Any advice on this ? Any tips could be helpful to us.

This e-mail message and any attachments are confidential and are for the exclusive use of the addressee only.  If you are not the intended recipient, you should not use the content, place any reliance on it or disclose it to anyone else.  Please notify the sender immediately by replying to it and then ensure that it is deleted from your system (including any attachments).