osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Flink Serialization and case class fields limit


Hey squirrels,

I've started to study more in-depth Flink Serialization and its "type system".

I have a generated case class using scalapb that has more than 30 fields; I've seen that Flink still uses the CaseClassSerializer, the TypeInformation is CaseClassTypeInfo, even if in the docs[1] is written differently (22 fields limit). I'd have expected a GenericTypeInfo, but all is well because the CaseClassSerializer is faster than Kryo. Did I misunderstand the documentation or don't the limitation apply anymore? 

Another thing, I've read "Juggling with Bits and Bytes"[2] blog post an I would like to replicate the experiment with some tailored changes to deep dive even better in the topic. Is the source code in Github or somewhere else?


Thank you,
Andrea