osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Enrich testing doc with more unit test examples using AbstractStreamOperator


Hi Tony,

I think this would be great - we’ve been building out tests using AbstractStreamOperator, and the lack of documentation has made it challenging.

For example, there was this exchange I had with Piotr about a month ago:

> You made a small mistake when restoring from state using test harness, that I myself have also done in the past. Problem is with an ordering of those calls:
> 
>         result.open();
>         if (savedState != null) {
>             result.initializeState(savedState);
>         }
> 
> Open is supposed to be called after initializeState, and if you look into the code of AbstractStreamOperatorTestHarness#open, if it is called before initialize, it will initialize harness without any state.
> 
> Unfortunate is that this is implicit behaviour that doesn’t throw any error (test harness is not part of a Flink’s public api). I will try to fix this: https://issues.apache.org/jira/browse/FLINK-10159 <https://issues.apache.org/jira/browse/FLINK-10159>
— Ken

> On Sep 25, 2018, at 3:30 AM, Tony Wei <tony19920430@xxxxxxxxx> wrote:
> 
> Hi all,
> 
> It seems that there are more and more users from user mailing list ask how
> to do unit test with Flink
> features like states or timer. And the community usually tends to suggest
> them using
> `AbstractStreamOperator` and provide an example from Flink github repo.
> Here I sort out some
> examples and write them down in the testing documentation [1]. And I would
> link to contribute back
> to the Flink.
> 
> The reason why I ask it first in dev mailing list is that
> `AbstractStreamOperator` is an internal API and
> could be changed at any time. I'm not sure if it is worth to provide these
> examples on testing
> document, so I want to collect some feedbacks before I go to open a JIRA
> ticket.
> 
> If this is feasible and valuable, then I will open the corresponding JIRA
> ticket and we can discuss
> more details of what examples are good to have in the document or how to
> structure the content.
> 
> I would really appreciate any feedback from you. Thanks in advance.
> 
> Best Regards,
> Tony Wei
> 
> [1]
> https://github.com/apache/flink/compare/master...tony810430:flink-testing-doc

--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
Custom big data solutions & training
Flink, Solr, Hadoop, Cascading & Cassandra