[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Pentaho to Airflow


Hey Arash -

There are some common operators built-in
<https://github.com/apache/incubator-airflow/tree/master/airflow/operators>
to Airflow and some in contrib
<https://github.com/apache/incubator-airflow/tree/master/airflow/contrib/operators>
as well.

We also maintain a community sourced GitHub org of Airflow plugins (mostly
hooks and operators) at https://github.com/airflow-plugins.

Are there specific sources/destinations you're looking for to match what
you use in Pentaho?

Best,
Taylor

*Taylor Edmiston*
Blog <https://blog.tedmiston.com/> | CV
<https://stackoverflow.com/cv/taylor> | LinkedIn
<https://www.linkedin.com/in/tedmiston/> | AngelList
<https://angel.co/taylor> | Stack Overflow
<https://stackoverflow.com/users/149428/taylor-edmiston>


On Tue, Jun 5, 2018 at 8:57 PM, Arash Soheili <tonyarash@xxxxxxxxx> wrote:

> Hi,
>
> I'm new to Airlfow and helping to setup our organization to transition away
> from using Pentaho Data Integration for our ETL. Although there are a lot
> of things I don't like about Pentaho they do have some nice standard
> modules like batch databae insert/update which are common ETL tasks.
>
> As I'm new to Airflow I haven't seen any standard Operators for this kind
> of task which I would think would be a common use case in Airflow or any
> ETL. Am I missing this information or is it expected upon each Airflow
> users to implement their own standard operators for this kind of operation.
> I would think this should at some point become part of Airflow codebase.
>
> Arash
>