osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: executing the pipeline from datalab


+ user, dev to bcc

Eila,

Is it possible that you are using an old version? I remember pending was missing in the dictionary and was added later. If that is not the reason, could you file a JIRA issue?

Thank you,
Ahmet


On Fri, Mar 23, 2018 at 6:15 AM, Jean-Baptiste Onofré <jb@xxxxxxxxxxxx> wrote:
Hi Eila,

can you please address this kind of question to the user mailing list ?

Thanks !
Regards
JB

On 03/23/2018 02:08 PM, OrielResearch Eila Arich-Landkof wrote:
> Hello all,
>
> When I run the pipeline with 4 samples (very small dataset), I don't get any
> error on DirectRunner or DataflowRunner
>
> When I run it with 50 samples dataset, I get the following error for the
> run.wait_until_finished()
> What does this error mean? 
> Thanks,
> Eila
>
> KeyErrorTraceback (most recent call last)
> <ipython-input-38-4d2dc0c0717f> in <module>() 1 result = pc.run()---->
> 2result.wait_until_finish()/usr/local/envs/py2env/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.pyc
> in wait_until_finish(self, duration) 771 while thread.isAlive():772
> time.sleep(5.0)--> 773if self.state != PipelineState.DONE:774 # TODO(BEAM-1290):
> Consider converting this to an error log based on the775 # resolution of the
> issue./usr/local/envs/py2env/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.pyc
> in state(self) 741 } 742 --> 743return (api_jobstate_map[self._job.currentState]
> if self._job.currentState 744 else PipelineState.UNKNOWN) 745 KeyError:
> CurrentStateValueValuesEnum(JOB_STATE_PENDING, 9)
>
>
>
>
> --
> Eila
> www.orielresearch.org <http://www.orielresearch.org>
> https://www.meetup.com/Deep-Learning-In-Production/

--
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com