OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Where is the mapreduce runner?


Hi,

as said, the M/R runner should be updated to leverage the portability
layer (especially Job API).

I will create the Jira and I will take look to evaluate the effort.

Regards
JB

On 20/06/2018 12:37, Reminia Scarlet wrote:
> I'm very glad that if there's anything I can do to push the release of
> MR runner. 
> 
> 
> 
> 
> On Wed, Jun 20, 2018 at 4:53 PM, Jean-Baptiste Onofré <jb@xxxxxxxxxxxx
> <mailto:jb@xxxxxxxxxxxx>> wrote:
> 
>     Hi,
> 
>     Pei and I worked on the MapReduce runner. There's some work to be done
>     (like leveraging the new Job API, etc).
> 
>     No short-term plan to include on master for now (so no release soon).
>     I'm working on this runner only on spare time.
> 
>     If you are interested to contribute, please let me know.
> 
>     Thanks,
>     Regards
>     JB
> 
>     On 20/06/2018 08:23, Reminia Scarlet wrote:
>     > @Kenneth  When will the runner be released?  Does the community have a
>     > plan on that ?
>     > 
>     > I've read the second link. The MR runner artifact cant be found in maven
>     > central. Maybe it 
>     > 
>     > has not beed deployed to maven.
>     > 
>     > On Wed, Jun 20, 2018 at 11:49 AM, Kenneth Knowles <klk@xxxxxxxxxx <mailto:klk@xxxxxxxxxx>
>     > <mailto:klk@xxxxxxxxxx <mailto:klk@xxxxxxxxxx>>> wrote:
>     > 
>     >     The MapReduce runner is still on a feature branch.
>     >     See https://beam.apache.org/contribute/#mapreduce-runner
>     <https://beam.apache.org/contribute/#mapreduce-runner>
>     >     <https://beam.apache.org/contribute/#mapreduce-runner
>     <https://beam.apache.org/contribute/#mapreduce-runner>>
>     > 
>     >     We should remove unreleased runners from the capability matrix.
>     > 
>     >     May I ask how you arrived at
>     >     https://beam.apache.org/documentation/runners/mapreduce/
>     <https://beam.apache.org/documentation/runners/mapreduce/>
>     >     <https://beam.apache.org/documentation/runners/mapreduce/
>     <https://beam.apache.org/documentation/runners/mapreduce/>> ?
>     > 
>     >     Kenn
>     > 
>     >     On Tue, Jun 19, 2018 at 8:36 PM Reminia Scarlet
>     >     <reminia.scarlet@xxxxxxxxx <mailto:reminia.scarlet@xxxxxxxxx>
>     <mailto:reminia.scarlet@xxxxxxxxx
>     <mailto:reminia.scarlet@xxxxxxxxx>>> wrote:
>     > 
>     >         I've found that mapreduce runner is supported  in beam from
>     >         below links:
>     > 
>     >         https://beam.apache.org/documentation/runners/mapreduce/
>     <https://beam.apache.org/documentation/runners/mapreduce/>
>     >         <https://beam.apache.org/documentation/runners/mapreduce/
>     <https://beam.apache.org/documentation/runners/mapreduce/>>
>     >
>     >       
>      https://beam.apache.org/documentation/runners/capability-matrix/
>     <https://beam.apache.org/documentation/runners/capability-matrix/>
>     >       
>      <https://beam.apache.org/documentation/runners/capability-matrix/
>     <https://beam.apache.org/documentation/runners/capability-matrix/>>
>     >
>     >         But I cant find the runner on maven.
>     >
>     >
> 
>     -- 
>     Jean-Baptiste Onofré
>     jbonofre@xxxxxxxxxx <mailto:jbonofre@xxxxxxxxxx>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com