osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to use "PortableRunner" in Python SDK?


Thanks Ankur and Maximilian.

Just for reference in case other people encountering the same error message, the "permission denied" error in my original email is exactly due to docker inside docker issue that Ankur mentioned.      Thanks Ankur! Didn't make the link when you said it, had to discover that in a hard way (I thought it is due to my docker installation messed up).

On Tue, Nov 6, 2018 at 1:53 AM Maximilian Michels <mxm@xxxxxxxxxx> wrote:
Hi,

Please follow https://beam.apache.org/roadmap/portability/#python-on-flink

Cheers,
Max

On 06.11.18 01:14, Ankur Goenka wrote:
> Hi,
>
> The Portable Runner requires a job server uri to work with. The current
> default job server docker image is broken because of docker inside
> docker issue.
>
> Please refer to
> https://beam.apache.org/roadmap/portability/#python-on-flink for how to
> run a wordcount using Portable Flink Runner.
>
> Thanks,
> Ankur
>
> On Mon, Nov 5, 2018 at 3:41 PM Ruoyun Huang <ruoyun@xxxxxxxxxx
> <mailto:ruoyun@xxxxxxxxxx>> wrote:
>
>     Hi, Folks,
>
>           I want to try out Python PortableRunner, by using following
>     command:
>
>     *sdk/python: python -m apache_beam.examples.wordcount
>       --output=/tmp/test_output   --runner PortableRunner*
>
>           It complains with following error message:
>
>     Caused by: java.lang.Exception: The user defined 'open()' method
>     caused an exception: java.io.IOException: Cannot run program
>     "docker": error=13, Permission denied
>     at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
>     at
>     org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
>     at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
>     ... 1 more
>     Caused by:
>     org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.util.concurrent.UncheckedExecutionException:
>     java.io.IOException: Cannot run program "docker": error=13,
>     Permission denied
>     at
>     org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4994)
>
>     ... 7 more
>
>
>
>     My py2 environment is properly configured, because DirectRunner
>     works.  Also I tested my docker installation by 'docker run
>     hello-world ', no issue.
>
>
>     Thanks.
>     --
>     ================
>     Ruoyun  Huang
>


--
================
Ruoyun  Huang