osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: dynamic tables in cassandra sink


If you restart the job each month you can build the string dynamically. If you want data to flow into the log based on a date in the record you will need to do something fancier. I have not used the casandra connector so I can’t help on the details. Can you subclass the connector and build the query dynamically for each data item? Or build a custom sink that just writes to casandra?

Michael

Sent from my iPad

On May 3, 2018, at 5:31 PM, Meghashyam Sandeep V <vr1meghashyam@xxxxxxxxx> wrote:

Hi there,

I have a flink stream from kafka writing to Cassandra. We use monthly tables in Cassandra to avoid TTL and tombstones that come with it. Tables would be like table_05_2018, table_06_2018 and so on. How do I dynamically register this table name in the following snippet?

CassandraSink
.addSink(dataStream)
.setQuery("INSERT INTO table_05_2018(id, text) VALUES (?,?);")
.setHost("127.0.0.1")
.build();