Question about sketches aggregation in druid
My name is Eshcar Hillel from Oath research. I'm currently working with Lee Rhodes on committing a new concurrent implementation of the theta sketch to the sketches-core library.I was wondering whether this implementation can help boost the union operation that is applied to multiple sketches at query time in druid.From what I see in the code the sketch aggregator uses the SynchronizedUnion implementation, which basically uses a lock at every single access (update/read) of the union operation. We believe a thread-safe implementation of the union operation can help decrease the inherent overhead of the lock.
I will be happy to join the meeting today and briefly discuss this option.