osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GitHub] jihoonson opened a new pull request #5996: Fix NPE while handling CheckpointNotice in KafkaSupervisor


jihoonson opened a new pull request #5996: Fix NPE while handling CheckpointNotice in KafkaSupervisor
URL: https://github.com/apache/incubator-druid/pull/5996
 
 
   Fixes https://github.com/apache/incubator-druid/issues/5992.
   
   In this PR, the supervisor has been changed to ignore inactive taskGroup (which is not in `taskGroups` map) while processing a checkpointNotice if it's a known taskGroup (which means it is in either `pendingCompletionTaskGroups` or `partitionGroups` maps). See `CheckpointNotice. isValidTaskGroup()`.
   
   Additionally, 
   
   - Removed unnecessary `sequenceTaskGroup` map from `KafkaSupervisor`. In `KafkaSupervisor`, there is a `ConcurrentHashMap` called `taskGroups` which maps a taskGroupId to an _active_ taskGroup. This is very similar to `sequenceTaskGroup` which maps a baseSequenceName to an active taskGroup because baseSequenceName is unique for a taskGroup. Also, `sequenceTaskGroup` was being changed along with `taskGroups`. We don't have to maintain two `ConcurrentHashMap`s which can even cause a race condition.
   - Since `sequenceTaskGroup` has been removed, the checkpoint request now contains a taskGroupId which is assigned to `KafkaIndexTask`s via `KafkaIOConfig`.
   - Fixed wrong `Nullable` annotation in `Supervisor.checkpoint()` method.
   - Changed the `map.putIfAbsent()` & `map.get()` pattern to `map.computeIfAbsent()`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@xxxxxxxxxxxxxxxx


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx