[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: BigTable IO in python SDK


Thanks for the JIRA link, I did some necessary JIRA clean-up to prevent future confusion.

On Wed, Sep 12, 2018 at 1:27 PM Solomon Duskis <sduskis@xxxxxxxxxx> wrote:
There's another JIRA here, that's been updated: https://issues.apache.org/jira/browse/BEAM-3342

Solomon Duskis | Google Cloud Bigtable Tech Lead | sduskis@xxxxxxxxxx | 914-462-0531


On Wed, Sep 12, 2018 at 4:26 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
Cham, thanks for pointing this out. I wasn't aware that Solomon was working on it. Solomon, could you update the JIRA with any progress and work with Antonio on getting a BigTable connector working.

On Wed, Sep 12, 2018 at 11:17 AM Chamikara Jayalath <chamikara@xxxxxxxxxx> wrote:
I believe Solomon Duskis is actively working with several other folks on this. Please collaborate with them on this.

Thanks,
Cham

On Wed, Sep 12, 2018 at 10:25 AM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:
If you setup a JIRA account, I can assign https://issues.apache.org/jira/browse/BEAM-2395 to you to track that your working on it.

Lots of basic questions can be answered also by looking at our contribution guide: https://beam.apache.org/contribute/
Like, where is the code, how to build, how to test, how to use an IDE, ...

On Wed, Sep 12, 2018 at 9:48 AM Antonio Momblan <antonio.momblan@xxxxxxxxx> wrote:
Thank you Lukasz ;)

I'll go through the authoring guide, and come back here in case of trouble.

Best regards,
/Antonio

El mié., 12 sept. 2018 a las 18:27, Lukasz Cwik (<lcwik@xxxxxxxxxx>) escribió:
I just want to make sure that you had seen our I/O authoring guide: https://beam.apache.org/documentation/io/authoring-overview/

I'm sure people would welcome the contribution and as you have questions, just reach out to the dev@xxxxxxxxxxxxxxx list as you have now.



On Wed, Sep 12, 2018 at 5:32 AM Antonio Momblan <antonio.momblan@xxxxxxxxx> wrote:
Dear members of the list,

Currently I need to create different pipelines, reading data from BigTable (bounded set of data in a range query), and saving the transformed data back to BigTable.

There is no IO connector for BigTable in the python SDK, as of today. 
However, I've noticed there is some work in progress to have it at some point in the future: https://issues.apache.org/jira/browse/BEAM-2395

Since this is work in progress, but seems to be stalled since June 2017, I would like to explore the possibility of contributing to this feature.

I've already had a look at the python sdk codebase, and I have an idea of how I would do it to fulfill my needs. But I would also appreciate some gidelines and support from the dev-list, so we could make this work also useful for the rest of the community.

Thanks in advance!

Best regards,
/Antonio