[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Stream collector serialization performance

Hi all,

I’m currently using the keyed process function, I see there’s serialization happening when I collect the object / update the object to rocksdb. For me the performance of serialization seems to be the bottleneck.
By default, POJO serializer is used, and the timecost of collect / update to rocksdb is roughly 1:1, Then I switch to kryo by setting getConfig.enableForceKryo(). Now the timecost of update to rocksdb decreases significantly to roughly 0.3, but the collect method seems not improving. Can someone help to explain this?

 My Object looks somehow like this:

Class A {
String f1 // 20 * string fields
List<B> f2. // 20 * list of another POJO object
Int f3 // 20 * ints fields
Class B {
String f // 5 * string fields


This communication may contain privileged or other confidential information of Red. If you have received it in error, please advise the sender by reply e-mail and immediately delete the message and any attachments without copying or disclosing the contents. Thank you.