OSDir

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Eventual PAssert


I left some comments but I like the idea of moving the validation into the pipeline instead of writing matchers that execute outside from within the test driver program.

This fits well with future SplittableDoFn work as this will allow us test sources and sinks and their side effects completely from within a pipeline.


On Mon, May 14, 2018 at 1:57 PM Anton Kedin <kedin@xxxxxxxxxx> wrote:
Hi,

While working on an integration test for Pubsub-related functionality I couldn't find a good solution to test the pipelines that don't reliably stop.

I propose we extend PAssert to support eventual verification. In this case some success/failure predicate is being constantly evaluated against all elements of the pipeline until it's met. At that point the result gets communicated to the main program/test.

Example API:

PAssert

 .thatEventually(pcollection)

 .containsInAnyOrder(e1, e2, e3)

 .synchronizingOver(signalOverPubsub());

 .timeoutAfter(10 min)



Comments, thoughts, things that I missed?

Regards,
Anton