osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Passing parameter to DoFn in Python


Yes, DoFns are normal Python classes. To do this you would write

class SampleFn(beam.DoFn):
    def __init__(self, samplePath):
        self.samplePath = samplePath

    def process(self, element):
        # use self.samplePath here, will get to remote workers via pickling

On Tue, Apr 10, 2018 at 4:27 PM OrielResearch Eila Arich-Landkof <eila@xxxxxxxxxxxxxxxxx> wrote:
Hi all,

Is it possible to pass a string parameter with DoFn function and what would be the syntax?

The call should look something like that:
beam.ParDo(SampleFn(samplePath))

how would the class definition be updated?
class SampleFn(beam.DoFn):
  def process(self,element):


Thanks,

--