osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Airflow - YARN as an executor?


With Hadoop 3's Docker on YARN support, I think YARN becomes
somewhat a competitor for Kubernetes.

Great job on adding k8s support to Airflow.

Very similarly I see Airflow could integrate with YARN and use
its infrastructure as an "executor" .. have anyone explored feasibility of
this approach?


Thanks!
Ruslan Dautkhanov