In the long run, we should get rid of the Docker-inside-Docker approach,
which was only intended for testing anyways. It would be cleaner to
start the SDK harness container alongside with JobServer container.
Short term, I think it should be easy to either fix the permissions of
the mounted "docker" executable or use a Docker image for the JobServer
which comes with Docker pre-installed.
JIRA: https://issues.apache.org/jira/browse/BEAM-6020
Thanks for reporting this Ruoyun!
-Max
On 08.11.18 00:10, Ruoyun Huang wrote:
> Thanks Ankur and Maximilian.
>
> Just for reference in case other people encountering the same error
> message, the "permission denied" error in my original email is exactly
> due to dockerinsidedocker issue that Ankur mentioned. Thanks Ankur!
> Didn't make the link when you said it, had to discover that in a hard
> way (I thought it is due to my docker installation messed up).
>
> On Tue, Nov 6, 2018 at 1:53 AM Maximilian Michels <mxm@xxxxxxxxxx
> <mailto:mxm@xxxxxxxxxx>> wrote:
>
> Hi,
>
> Please follow
> https://beam.apache.org/roadmap/portability/#python-on-flink
>
> Cheers,
> Max
>
> On 06.11.18 01:14, Ankur Goenka wrote:
> > Hi,
> >
> > The Portable Runner requires a job server uri to work with. The
> current
> > default job server docker image is broken because of docker inside
> > docker issue.
> >
> > Please refer to
> > https://beam.apache.org/roadmap/portability/#python-on-flink for
> how to
> > run a wordcount using Portable Flink Runner.
> >
> > Thanks,
> > Ankur
> >
> > On Mon, Nov 5, 2018 at 3:41 PM Ruoyun Huang <ruoyun@xxxxxxxxxx
> <mailto:ruoyun@xxxxxxxxxx>
> > <mailto:ruoyun@xxxxxxxxxx <mailto:ruoyun@xxxxxxxxxx>>> wrote:
> >
> > Hi, Folks,
> >
> > I want to try out Python PortableRunner, by using following
> > command:
> >
> > *sdk/python: python -m apache_beam.examples.wordcount
> > --output=/tmp/test_output --runner PortableRunner*
> >
> > It complains with following error message:
> >
> > Caused by: java.lang.Exception: The user defined 'open()' method
> > caused an exception: java.io.IOException: Cannot run program
> > "docker": error=13, Permission denied
> > at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498)
> > at
> >
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
> > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
> > ... 1 more
> > Caused by:
> >
> org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.util.concurrent.UncheckedExecutionException:
> > java.io.IOException: Cannot run program "docker": error=13,
> > Permission denied
> > at
> >
> org.apache.beam.repackaged.beam_runners_java_fn_execution.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4994)
> >
> > ... 7 more
> >
> >
> >
> > My py2 environment is properly configured, because DirectRunner
> > works. Also I tested my docker installation by 'docker run
> > hello-world ', no issue.
> >
> >
> > Thanks.
> > --
> > ================
> > Ruoyun Huang
> >
>
>
>
> --
> ================
> Ruoyun Huang
>