[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Improvements to the Unified SQL Connector API


Thanks a lot for the proposal. I like the idea to unify table definitions.
I think we can drop the table type since the type can be derived from the
sql, i.e, a table be inserted can only be a sink table.

I left some minor suggestions in the document, mainly include:
- Maybe we also need to allow define properties for tables.
- Support specify Computed Columns in a table
- Support define keys for sources.

Best, Hequn

On Thu, Oct 4, 2018 at 4:09 PM Shuyi Chen <suez1224@xxxxxxxxx> wrote:

> Thanks a lot for the proposal, Timo. I left a few comments. Also, it seems
> the example in the doc does not have the table type (source, sink and both)
> property anymore. Are you suggesting drop it? I think the table type
> properties is still useful as it can restrict a certain connector to be
> only source/sink, for example, we usually want a Kafka topic to be either
> read-only or write-only, but not both.
> Shuyi
> On Mon, Oct 1, 2018 at 1:53 AM Timo Walther <twalthr@xxxxxxxxxx> wrote:
> > Hi everyone,
> >
> > as some of you might have noticed, in the last two releases we aimed to
> > unify SQL connectors and make them more modular. The first connectors
> > and formats have been implemented and are usable via the SQL Client and
> > Java/Scala/SQL APIs.
> >
> > However, after writing more connectors/example programs and talking to
> > users, there are still a couple of improvements that should be applied
> > to unified SQL connector API.
> >
> > I wrote a design document [1] that discusses limitations that I have
> > observed and consideres feedback that I have collected over the last
> > months. I don't know whether we will implement all of these
> > improvements, but it would be great to get feedback for a satisfactory
> > API and for future priorization.
> >
> > The general goal should be to connect to external systems as convenient
> > and type-safe as possible. Any feedback is highly appreciated.
> >
> > Thanks,
> >
> > Timo
> >
> > [1]
> >
> >
> https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit?usp=sharing
> >
> >
> --
> "So you have to trust that the dots will somehow connect in your future."