OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Beam SQL: Integrating runners & IO


Thanks for the great writeup Kenn! I really like the part about pushing the TableProvider abstraction into core and the IOs. This would make it really easy to extend the IOs supported by your SQL shell just by adding the appropriate IOs to the classpath. It would also make testing much easier as we could create mock IOs that don't depend on SQL for some of the JDBC integration tests.

Andrew

On Mon, Jun 11, 2018 at 10:44 AM Kenneth Knowles <klk@xxxxxxxxxx> wrote:
Hi all,

Andrew mentioned something super cool about what he and Anton have done with Beam SQL: it now implements a JDBC driver (via Calcite Avatica).

And since the sqlline client is tiny, it is just baked in. So you can java -jar the JDBC driver and run a little shell. But the shell only has the direct runner, and to make a nice experience for SQL users who don't want to deal with Java, there are some challenges.

I wrote up a really quick doc for brainstorming

https://s.apache.org/beam-sql-packaging

Please lend you comments and expertise!

Kenn