OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Improvements to the Unified SQL Connector API


Thanks for the proposal!

I like the proposed changes a lot, especially support for reading/writing key data of systems that have a key/value split will be very nice to have.

> On 2. Oct 2018, at 11:58, Timo Walther <twalthr@xxxxxxxxxx> wrote:
> 
> Thanks for the feedback Fabian. I updated the document and addressed your comments.
> 
> I agree that tables which are stored in different systems need more discussion. I would suggest to deprecate the field mapping interfaces in this release and remove it in the next release.
> 
> Regards,
> Timo
> 
> 
> Am 02.10.18 um 11:06 schrieb Fabian Hueske:
>> Thanks for the proposal Timo!
>> 
>> I've done a pass and added some comments (mostly asking for clarification,
>> details).
>> Overall, this is going into a very good direction.
>> I think the tables which are stored in different systems and using a format
>> definition to define other formats require some more discussions.
>> However, these are also not the features that we would start with.
>> 
>> >From a compatibility point of view, an important question to answer would
>> be whether we can drop the support for field mapping, i.e., do we have
>> users who take advantage of mapping format fields to fields with a
>> different name in the schema.
>> Besides that, all existing functionality is preserved although the syntax
>> changes a bit.
>> 
>> Best,
>> Fabian
>> 
>> Am Mo., 1. Okt. 2018 um 10:53 Uhr schrieb Timo Walther <twalthr@xxxxxxxxxx>:
>> 
>>> Hi everyone,
>>> 
>>> as some of you might have noticed, in the last two releases we aimed to
>>> unify SQL connectors and make them more modular. The first connectors
>>> and formats have been implemented and are usable via the SQL Client and
>>> Java/Scala/SQL APIs.
>>> 
>>> However, after writing more connectors/example programs and talking to
>>> users, there are still a couple of improvements that should be applied
>>> to unified SQL connector API.
>>> 
>>> I wrote a design document [1] that discusses limitations that I have
>>> observed and consideres feedback that I have collected over the last
>>> months. I don't know whether we will implement all of these
>>> improvements, but it would be great to get feedback for a satisfactory
>>> API and for future priorization.
>>> 
>>> The general goal should be to connect to external systems as convenient
>>> and type-safe as possible. Any feedback is highly appreciated.
>>> 
>>> Thanks,
>>> 
>>> Timo
>>> 
>>> [1]
>>> 
>>> https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit?usp=sharing
>>> 
>>> 
>