Re: What is the way to prove performance improvement?
There is a some small set of benchmarks defined in https://github.com/dataArtisans/flink-benchmarks <https://github.com/dataArtisans/flink-benchmarks> , however their scope is limited and after briefly looking at your PRs, I wouldn’t expect them to cover your cases. However if you could define there some jmh micro benchmark to cover your cases that would be nice. It would be a shame if someone would accidentally revert/brake your improvements in the future.
> On 18 Jul 2018, at 08:52, 陈梓立 <wander4096@xxxxxxxxx> wrote:
> Hi Till,
> Thanks for your reply! I will try to add ones later.
> Till Rohrmann <trohrmann@xxxxxxxxxx> 于2018年7月18日周三 下午2:48写道：
>> Hi Tison,
>> at the moment there is formal way to verify performance improvements. What
>> you can do is to provide your measurements by adding the graphs to the PR
>> thread and specify the setup. Then others could try to verify these numbers
>> by running their own benchmark.
>> On Wed, Jul 18, 2018 at 1:34 AM 陈梓立 <wander4096@xxxxxxxxx> wrote:
>>> Hi all,
>>> Recently I pull 3 PRs about performance improvements. Unit tests
>>> will verify their correctness, and in the real scenario, we have
>>> report to confirm that they do help for performance.
>>> I wonder what is the formal way to verify a performance improvement. Is
>>> to give out a benchmark report, or run a standard benchmark, or add
>>> performance test(I don't know how to do it), or anything else.
>>> Looking forward for your reply.
>>>  https://github.com/apache/flink/pull/6339
>>>  https://github.com/apache/flink/pull/6345
>>>  https://github.com/apache/flink/pull/6353