osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to use "PortableRunner" in Python SDK?


Hi,

Please follow https://beam.apache.org/roadmap/portability/#python-on-flink

Cheers,
Max

On 06.11.18 01:14, Ankur Goenka wrote:
Hi,

The Portable Runner requires a job server uri to work with. The current default job server docker image is broken because of docker inside docker issue.

Please refer to https://beam.apache.org/roadmap/portability/#python-on-flink for how to run a wordcount using Portable Flink Runner.

Thanks,
Ankur

On Mon, Nov 5, 2018 at 3:41 PM Ruoyun Huang <ruoyun@xxxxxxxxxx <mailto:ruoyun@xxxxxxxxxx>> wrote:

    Hi, Folks,

          I want to try out Python PortableRunner, by using following
    command:

*sdk/python: python -m apache_beam.examples.wordcount  --output=/tmp/test_output   --runner PortableRunner*

          It complains with following error message:

    Caused by: java.lang.Exception: The user defined 'open()' method
    caused an exception: java.io.IOException: Cannot run program
    "docker": error=13, Permission denied
    at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
    at
    org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
    ... 1 more
    Caused by:
    org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.util.concurrent.UncheckedExecutionException:
    java.io.IOException: Cannot run program "docker": error=13,
    Permission denied
    at
    org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4994)

    ... 7 more



    My py2 environment is properly configured, because DirectRunner
    works.  Also I tested my docker installation by 'docker run
    hello-world ', no issue.


    Thanks.
-- ================
    Ruoyun  Huang