[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to use "PortableRunner" in Python SDK?


The Portable Runner requires a job server uri to work with. The current default job server docker image is broken because of docker inside docker issue.

Please refer to https://beam.apache.org/roadmap/portability/#python-on-flink for how to run a wordcount using Portable Flink Runner.


On Mon, Nov 5, 2018 at 3:41 PM Ruoyun Huang <ruoyun@xxxxxxxxxx> wrote:
Hi, Folks,

     I want to try out Python PortableRunner, by using following command: 

sdk/python: python -m apache_beam.examples.wordcount   --output=/tmp/test_output   --runner PortableRunner

     It complains with following error message: 

Caused by: java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Cannot run program "docker": error=13, Permission denied
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
... 1 more
Caused by: org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.util.concurrent.UncheckedExecutionException: java.io.IOException: Cannot run program "docker": error=13, Permission denied
at org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4994)

... 7 more

My py2 environment is properly configured, because DirectRunner works.  Also I tested my docker installation by 'docker run hello-world ', no issue. 

Ruoyun  Huang