osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Sources of Reference objects in Druid


The discussion has started here:
https://github.com/apache/incubator-druid/pull/6677#discussion_r237182258
(TL;DR: there are a lot of references from java.lang.ref package, including
finalizers, in the heaps of Druid nodes, that is bad for GC).

Now I'm 90% sure that the major source of finalizable objects is Curator:
https://issues.apache.org/jira/browse/CURATOR-487

We are not exposed to that, but those who use Zstd compression with Druid
could have a lot of finalizable objects coming from Zstd compression, see
https://issues.apache.org/jira/browse/COMPRESS-473 and
https://github.com/luben/zstd-jni/issues/83

Zstd compression / decompression could also be a subject of double-free
race through finalize(): https://github.com/luben/zstd-jni/issues/82

I didn't check other compression/decompression/encryption/decryption parts
used by Druid yet but there is a big chance that some of them are
problematic too.