osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Please review : Re: PubSub connector (FLINK-9311)


Hi Niels and Richard,

I would be very happy about having a PubSub connector in Flink. Having it in Flink means that you don't have manual effort for tracking API changes and I think having a production user is incentive enough for them (you) to maintain the connector.

I'm afraid we don't have much PubSub knowledge in the Flink community but I will try and talk to some folks to have this reviewed.

Best,
Aljoscha

> On 14. Sep 2018, at 15:40, Niels Basjes <Niels@xxxxxxxxx> wrote:
> 
> Hi all,
> 
> We (Richard and I) would really appreciate it if you guys could review
> the new feature we created (see below).
> It is something we really need in production and thought it would be
> best if it can be a native part of the Flink toolset.
> Please indicate what we need to change/improve in order to get this committed.
> 
> Thank you.
> 
> Niels Basjes
> 
> On Mon, Sep 10, 2018 at 10:12 AM Richard Deurwaarder <richard@xxxxxxx> wrote:
>> 
>> Hello everyone,
>> 
>> A while back I opened this Jira issue:
>> https://issues.apache.org/jira/browse/FLINK-9311. It is regarding a new
>> Flink connector for Google PubSub.
>> 
>> At Bol.com, the company I work for, we are going to use this connector to
>> do our financial processing. My colleague, Niels Basjes, and I have
>> finished implementing this and we are about ready to start running it in
>> production.
>> 
>> We would like to donate this code and have opened a pull request (
>> https://github.com/apache/flink/pull/6594). The pull request contains:
>> 
>>   - a SourceFunction (with 2 test versions)
>>   - a SinkFunction
>>   - an example application
>>   - and End-to-End tests using a docker container
>> 
>> Yanghua has taken a glance at the code and already provided us with some
>> feedback. We would like to invite others to provide additional feedback.
>> 
>> I hope you find this useful and will consider merging the PR!
>> 
>> Best,
>> 
>> Richard Deurwaarder
> 
> 
> 
> -- 
> Best regards / Met vriendelijke groeten,
> 
> Niels Basjes