[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[BucketingSink] notify on moving into pending/ final state

Hi mates, I got a proposal about functionality of BucketingSink.

During implementation of one of our tasks we got the following need - create a meta-file, with the path and additional information about the file, created by BucketingSink, when it’s been moved into final place.
Unfortunately such behaviour is currently not available for us. 

We’ve implemented our own Sink, that provides an opportunity to register notifiers, that will be called, when file state is changing, but current API doesn’t allow us to add such behaviour using inheritance ...

It seems, that such functionality could be useful, and could be a part of BucketingSink API
What do you sink, should I make a PR ?

Sincerely yours,
Rinat Sharipov
Software Engineer at 1DMP CORE Team

mobile: +7 (925) 416-37-26

make your data clever