osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Does NeedsRunner work on parameterized tests?


We hit a similar issue when trying to refactor some tests to use the 'Enclosed' runner. It seems the Gradle test filtering is not compatible with the Enclosed runner See BEAM-4375 [1] and pr/5440 [2]. 

I'm not familiar with the Parametrized runner, but perhaps it has similar issues.

[2] https://github.com/apache/beam/pull/5440

On Mon, Dec 17, 2018 at 6:15 PM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Also FWIW when I tried to repro this, I consistently got many failures and a hang when I tried to run the whole suite via `./gradlew needsRunnerTests`. Filed https://issues.apache.org/jira/browse/BEAM-6256 with some details.

Kenn

On Mon, Dec 17, 2018 at 8:54 PM Reuven Lax <relax@xxxxxxxxxx> wrote:
It looks like maybe this is a JUnit bug: https://github.com/gradle/gradle/issues/1153

The fix is targeted for JUnit 4.13, which is still not fully released (and is intended to be the final JUnit 4 release). 4.13.-beta1 is out though, and it might be worth trying that out to see if it fixes this. Does anyone know how we control the version of JUnit used?

On Mon, Dec 17, 2018 at 5:39 PM Reuven Lax <relax@xxxxxxxxxx> wrote:
Ok, this is worrying - we're silently skipping some important test suites. Does anyone know what might be causing us to skip them?

On Mon, Dec 17, 2018 at 5:03 PM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Flipping through the latest Java PreCommit Cron, it does look like NeedsRunner tests within the Parameterized sub-suites of TextIOReadTest are not executed.


Kenn

On Sat, Dec 15, 2018 at 12:16 PM Reuven Lax <relax@xxxxxxxxxx> wrote:
Hi,

I tried to switch one of our tests (AvroIOTest) over to be a parameterized test. However after doing so, I cannot get the NeedsRunner tests to execute! All attempts fail with the following:

No tests found for given includes: [org.apache.beam.sdk.io.AvroIOTest](filter.includeTestsMatching)

I also can't find evidence that Jenkins will execute this either.

Am I missing something, or do our NeedsRunner tests no longer work for parameterized tests? If the latter this is concerning, as there are a number of NeedsRunner paramaterized tests in our codebase (e.g. the TextIO tests).

Reuven


--