osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to migrate Kafka Producer ?


Hi Edward,

AFAIK we do not support migrating state from one connector to another
one, which is in fact the case for kafka 0.11 and the "universal" one.

You might try to use the project bravo[1] to migrate the state manually,
but unfortunately you have to understand the internals of both of the
connectors. I pull also Piotr to the thread, maybe he can provide more
straightforward workaround.

Best,

Dawid

[1] https://github.com/king/bravo

On 18/12/2018 14:33, Edward Rojas wrote:
> Hi,
>
> I'm planning to migrate from kafka connector 0.11 to the new universal kafka
> connector 1.0.0+ but I'm having some troubles.
>
> The kafka consumer seems to be compatible but when trying to migrate the
> kafka producer I get an incompatibility error for the state migration. 
> It looks like the producer uses a list state of type
> "NextTransactionalIdHint", but this class is specific for each Producer
> (FlinkKafkaProducer011.NextTransactionalIdHint  vs
> FlinkKafkaProducer.NextTransactionalIdHint) and therefore the states are not
> compatible.
>
>
> I would like to know what is the recommended way to perform this kind of
> migration without losing the state ?
>
> Thanks in advance,
> Edward
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Attachment: signature.asc
Description: OpenPGP digital signature