Re: Airflow - YARN as an executor?
Happy to have it as a contrib executor. However, I personally think yarn is a dead end. It has a lot of catching up to do and all the momentum is with kubernetes.
Verstuurd vanaf mijn iPad
> Op 24 apr. 2018 om 22:13 heeft Ruslan Dautkhanov <dautkhanov@xxxxxxxxx> het volgende geschreven:
> With Hadoop 3's Docker on YARN support, I think YARN becomes
> somewhat a competitor for Kubernetes.
> Great job on adding k8s support to Airflow.
> Very similarly I see Airflow could integrate with YARN and use
> its infrastructure as an "executor" .. have anyone explored feasibility of
> this approach?
> Ruslan Dautkhanov