osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ANNOUNCEMENT] Nexmark included to the CI


Awesome Etienne, this is really important for the (user) community to have that visibility since it is one of the most important aspect of the Beam's quality, kudo!

Romain Manni-Bucau
@rmannibucau |  Blog | Old BlogGithub | LinkedIn | Book


Le jeu. 12 juil. 2018 à 10:59, Jean-Baptiste Onofré <jb@xxxxxxxxxxxx> a écrit :
It's really great to have these dashboards and integration in Jenkins !

Thanks Etienne for driving this !

Regards
JB

On 11/07/2018 15:13, Etienne Chauchot wrote:
>
> Hi guys,
>
> I'm glad to announce that the CI of Beam has much improved ! Indeed
> Nexmark is now included in the perfkit dashboards.
>
> At each commit on master, nexmark suites are run and plots are created
> on the graphs.
>
> I've created 2 kind of dashboards:
> - one for performances (run times of the queries)
> - one for the size of the output PCollection (which should be constant)
>
> There are dashboards for these runners:
> - spark
> - flink
> - direct runner
>
> Each dashboard contains:
> - graphs in batch mode 
> - graphs in streaming mode
> - graphs for the 13 queries.
>
> That gives more than a hundred of graphs (my right finger hurts after so
> many clics on the mouse :) ). It is detailed that much so that anyone
> can focus on the area they have interest in.
> Feel free to also create new dashboards with more aggregated data.
>
> Thanks to Lukasz and Cham for reviewing my PRs and showing how to use
> perfkit dashboards.
>
> Dashboards are there:
>
> https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424
> https://apache-beam-testing.appspot.com/explore?dashboard=5699257587728384
> <https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712>https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712
>
> https://apache-beam-testing.appspot.com/explore?dashboard=5099379773931520
> https://apache-beam-testing.appspot.com/explore?dashboard=5731568492478464
> https://apache-beam-testing.appspot.com/explore?dashboard=5163657986048000
>
>
> Enjoy, 
>
> Etienne
>
>

--
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com