osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Stream collector serialization performance


Hi all,

I’m currently using the keyed process function, I see there’s serialization happening when I collect the object / update the object to rocksdb. For me the performance of serialization seems to be the bottleneck.
By default, POJO serializer is used, and the timecost of collect / update to rocksdb is roughly 1:1, Then I switch to kryo by setting getConfig.enableForceKryo(). Now the timecost of update to rocksdb decreases significantly to roughly 0.3, but the collect method seems not improving. Can someone help to explain this?

 My Object looks somehow like this:

Class A {
String f1 // 20 * string fields
List<B> f2. // 20 * list of another POJO object
Int f3 // 20 * ints fields
}
Class B {
String f // 5 * string fields
}

Best,
Mingliang

本邮件及其附件含有小红书公司的保密信息,仅限于发送给以上收件人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件!
This communication may contain privileged or other confidential information of Red. If you have received it in error, please advise the sender by reply e-mail and immediately delete the message and any attachments without copying or disclosing the contents. Thank you.