osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Codespeed deployment for Flink


Great addition Piotr and Nico. This is a really nice tool for the community
to monitor performance regressions in Flink.

On Tue, Sep 25, 2018 at 8:41 PM Peter Huang <huangzhenqiu0825@xxxxxxxxx>
wrote:

> It is a great tool. Thanks for the contribution.
>
> On Tue, Sep 25, 2018 at 11:39 AM Jin Sun <isunjin@xxxxxxxxx> wrote:
>
> > Great tool!
> >
> > > On Sep 24, 2018, at 10:59 PM, Zhijiang(wangzhijiang999) <
> > wangzhijiang999@xxxxxxxxxx.INVALID> wrote:
> > >
> > > Thanks @Piotr Nowojski  and @Nico Kruber for the good job!
> > >
> > > I already benefit from this benchmark in the previous PRs. Wish the
> > visualization tool becoming stronger to benefit more for the community!
> > >
> > > Best,
> > > Zhijiang
> > > ------------------------------------------------------------------
> > > 发件人:Piotr Nowojski <piotr@xxxxxxxxxxxxxxxxx>
> > > 发送时间:2018年9月21日(星期五) 22:59
> > > 收件人:dev <dev@xxxxxxxxxxxxxxxx>
> > > 抄 送:Nico Kruber <nico@xxxxxxxxxxxxxxxxx>
> > > 主 题:Codespeed deployment for Flink
> > >
> > > Hello community,
> > >
> > > For almost a year in data Artisans Nico and I were maintaining a setup
> > > that continuously evaluates Flink with benchmarks defined at
> > > https://github.com/dataArtisans/flink-benchmarks <
> > https://github.com/dataArtisans/flink-benchmarks>. With growing interest
> > > and after proving useful a couple of times, we have finally decided to
> > > publish the web UI layer of this setup. Currently it is accessible via
> > > the following (maybe not so?) temporarily url:
> > >
> > > http://codespeed.dak8s.net:8000 <http://codespeed.dak8s.net:8000/>
> > >
> > > This is a simple web UI to present performance changes over past and
> > > present commits to Apache Flink. It only has a couple of views and the
> > > most useful ones are:
> > >
> > > 1. Timeline
> > > 2. Comparison (I recommend to use normalization)
> > >
> > > Timeline is useful for spotting unintended regressions or unexpected
> > > improvements. It is being updated every six hours.
> > > Comparison is useful for comparing a given branch (for example a
> pending
> > > PR) with the master branch. More about that later.
> > >
> > > The codespeed project on it’s own is just a presentation layer. As
> > > mentioned before, the only currently available benchmarks are defined
> in
> > > the flink-benchmarks repository and they are executed periodically or
> on
> > > demand by Jenkins on a single bare metal machine. The current setup
> > > limits us only to micro benchmarks (they are easier to
> > > setup/develop/maintain and have a quicker feedback loop compared to
> > > cluster benchmarks) but there is no reason preventing us from setting
> up
> > > other kinds of benchmarks and upload their results to our codespeed
> > > instance as well.
> > >
> > > Regarding the comparison view. Currently data Artisans’ Flink mirror
> > > repository at https://github.com/dataArtisans/flink <
> > https://github.com/dataArtisans/flink> is configured to
> > > trigger benchmark runs on every commit/change that happens on the
> > > benchmark-request branch (We chose to use dataArtisans' repository here
> > > because we needed a custom GitHub hook that we couldn’t add to the
> > > apache/flink repository). Benchmarking usually takes between one and
> two
> > > hours. One obvious limitation at the moment is that there is only one
> > > comparison view, with one comparison branch, so trying to compare two
> > > PRs at the same time is impossible. However we can tackle
> > > this problem once it will become a real issue, not only a theoretical
> > one.
> > >
> > > Piotrek & Nico
> > >
> >
> >
>