The reasoning unbounded threadpool is explained as:
On Thu, Nov 8, 2018 at 11:41 PM Dan Halperin <dhalperi@xxxxxxxxxx> wrote:
On Thu, Nov 8, 2018 at 2:12 PM Udi Meiri <ehudm@xxxxxxxxxx> wrote:Both options risk delaying worker shutdown if the executor's shutdown() hasn't been called, which is I guess why the executor in GcsOptions.java creates daemon threads.My guess (and it really is a guess at this point) is that this was a fix for DirectRunner issues - want that to exit quickly!On Thu, Nov 8, 2018 at 1:02 PM Lukasz Cwik <lcwik@xxxxxxxxxx> wrote:Not certain, it looks like we should have been caching the executor within the GcsUtil as a static instance instead of creating one each time. Could have been missed during code review / slow code changes over time. GcsUtil is not well "loved".On Thu, Nov 8, 2018 at 11:00 AM Udi Meiri <ehudm@xxxxxxxxxx> wrote:HI,I've identified a memory leak when GcsUtil.java instantiates a ThreadPoolExecutor (https://issues.apache.org/jira/browse/BEAM-6018).The code uses the getExitingExecutorService wrapper, which leaks memory. The question is, why is that wrapper necessary if executor.shutdown(); is later unconditionally called?