Re: [DISCUSS] Improvements to the Unified SQL Connector API
Thanks for the proposal!
I like the proposed changes a lot, especially support for reading/writing key data of systems that have a key/value split will be very nice to have.
> On 2. Oct 2018, at 11:58, Timo Walther <twalthr@xxxxxxxxxx> wrote:
> Thanks for the feedback Fabian. I updated the document and addressed your comments.
> I agree that tables which are stored in different systems need more discussion. I would suggest to deprecate the field mapping interfaces in this release and remove it in the next release.
> Am 02.10.18 um 11:06 schrieb Fabian Hueske:
>> Thanks for the proposal Timo!
>> I've done a pass and added some comments (mostly asking for clarification,
>> Overall, this is going into a very good direction.
>> I think the tables which are stored in different systems and using a format
>> definition to define other formats require some more discussions.
>> However, these are also not the features that we would start with.
>> >From a compatibility point of view, an important question to answer would
>> be whether we can drop the support for field mapping, i.e., do we have
>> users who take advantage of mapping format fields to fields with a
>> different name in the schema.
>> Besides that, all existing functionality is preserved although the syntax
>> changes a bit.
>> Am Mo., 1. Okt. 2018 um 10:53 Uhr schrieb Timo Walther <twalthr@xxxxxxxxxx>:
>>> Hi everyone,
>>> as some of you might have noticed, in the last two releases we aimed to
>>> unify SQL connectors and make them more modular. The first connectors
>>> and formats have been implemented and are usable via the SQL Client and
>>> Java/Scala/SQL APIs.
>>> However, after writing more connectors/example programs and talking to
>>> users, there are still a couple of improvements that should be applied
>>> to unified SQL connector API.
>>> I wrote a design document  that discusses limitations that I have
>>> observed and consideres feedback that I have collected over the last
>>> months. I don't know whether we will implement all of these
>>> improvements, but it would be great to get feedback for a satisfactory
>>> API and for future priorization.
>>> The general goal should be to connect to external systems as convenient
>>> and type-safe as possible. Any feedback is highly appreciated.