Re: [DISCUSS] Improvements to the Unified SQL Connector API
Thanks a lot for the proposal. I like the idea to unify table definitions.
I think we can drop the table type since the type can be derived from the
sql, i.e, a table be inserted can only be a sink table.
I left some minor suggestions in the document, mainly include:
- Maybe we also need to allow define properties for tables.
- Support specify Computed Columns in a table
- Support define keys for sources.
On Thu, Oct 4, 2018 at 4:09 PM Shuyi Chen <suez1224@xxxxxxxxx> wrote:
> Thanks a lot for the proposal, Timo. I left a few comments. Also, it seems
> the example in the doc does not have the table type (source, sink and both)
> property anymore. Are you suggesting drop it? I think the table type
> properties is still useful as it can restrict a certain connector to be
> only source/sink, for example, we usually want a Kafka topic to be either
> read-only or write-only, but not both.
> On Mon, Oct 1, 2018 at 1:53 AM Timo Walther <twalthr@xxxxxxxxxx> wrote:
> > Hi everyone,
> > as some of you might have noticed, in the last two releases we aimed to
> > unify SQL connectors and make them more modular. The first connectors
> > and formats have been implemented and are usable via the SQL Client and
> > Java/Scala/SQL APIs.
> > However, after writing more connectors/example programs and talking to
> > users, there are still a couple of improvements that should be applied
> > to unified SQL connector API.
> > I wrote a design document  that discusses limitations that I have
> > observed and consideres feedback that I have collected over the last
> > months. I don't know whether we will implement all of these
> > improvements, but it would be great to get feedback for a satisfactory
> > API and for future priorization.
> > The general goal should be to connect to external systems as convenient
> > and type-safe as possible. Any feedback is highly appreciated.
> > Thanks,
> > Timo
> > 
> "So you have to trust that the dots will somehow connect in your future."