osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Kafka consumer, is there a way to filter out messages using key only?


Hi, I am using 1.7 on K8S.

I have a huge amount of data in kafka, but I only need a tiny portion of it.
It is a keyed stream, the value in JSON encoded. I want to avoid deserialization of the value, since it is very expensive. Can I only filter based on the key?
I know there is a KeyedDeserializationSchema, but can I use it to filter data?

Hao Sun
Team Lead
1019 Market St. 7F
San Francisco, CA 94103