OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Single Airflow Instance Vs Multiple Airflow Instance


An important consideration here is that there are several settings that are
cluster-wide. In particular, cluster-wide concurrency settings could result
in Team B's DAG refusing to schedule based on an error in Team A's DAG.

Do your teams follow similar practices in how eagerly they ship code, or
have similar SLAs for resolving issues? If so, you are probably fine using
co-tenancy. If not, you should probably talk about it first to make sure
the teams are okay with co-tenancy.

On Wed, Jun 6, 2018 at 11:24 AM, gauthiermartin86@xxxxxxxxx <
gauthiermartin86@xxxxxxxxx> wrote:

> Hi Everyone,
>
> We have been experimenting with airflow for about 6 months now.
> We are planning to have multiple departments to use it. Since we don't
> have any internal experience with Airflow we are wondering if single
> instance per department is more suited than single instance with
> multi-tenancy? We have been aware about the upcoming release of airflow
> 1.10 and changes that will be made to the RBAC which will be more suited
> for multi-tenancy.
>
> Any advice on this ? Any tips could be helpful to us.
>