osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: A new Beam Runner on Apache Nemo


Hi Wonook,

First of all, welcome to the Beam community! It is great to see another Runner emerging.

If you're planning to contribute your Runner to Beam, you should verify the compatibility with the ValidatesRunner integration tests. Then open a PR with documentation, a Runner page, and updates to the matrix.

If you're planning to leave the Runner outside Beam for the time being, please submit a Runner page for the Beam website. The page should contain information on how to use the Runner and a link to the external web site with up-to-date information.

Feel free to ask here or in our Slack channel if you have more questions.

I'm also curious, have you looked into integrating portability with the Nemo Runner?

Thanks,
Max

On 16.11.18 06:51, 송원욱 wrote:
Hello all!

I'm a member of the Apache Nemo community, another Apache project for processing big data focusing on easy-to-use, flexible optimizations for various deployment environments. More information can be seen on our website <http://nemo.apache.org>. We've been building the system for quite a while now, and we have been using Apache Beam as one of the programming layers that we support for writing data processing applications. We have already taken a look at the capability matrix <https://beam.apache.org/documentation/runners/capability-matrix/> of Beam runners, and the runner authoring guide <https://beam.apache.org/contribute/runner-guide/>, and we have been successful in implementing a large portion of the capability criteria.

With the progress, we wish to be able to list our runner as one of the Beam runners, to  be able to notify the users that our system supports Beam, and that Beam users have another option to choose from for running their data processing applications. It would be lovely to know the details of the process required for it!

Thanks!


      Wonook