osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: delayed emit (timer) in py-beam?


Hey Austin,

This API is not yet implemented in the Python SDK.  I am working on this feature:  the next step from my end is to finish a reference implementation in the local DirectRunner.  As you note, the doc at https://s.apache.org/beam-python-user-state-and-timers describes the design.

You can track progress on the mailing list thread here: https://lists.apache.org/thread.html/51ba1a00027ad8635bc1d2c0df805ce873995170c75d6a08dfe21997@%3Cdev.beam.apache.org%3E

Best,
Charles

On Mon, Jul 30, 2018 at 3:34 PM Austin Bennett <whatwouldaustindo@xxxxxxxxx> wrote:
What's going on with timers and python?  

Am looking at building a pipeline (assuming another group in my company will grant access to the Kafka topic):

Kafka -> beam -> have beam wait 24 hours -> do transform(s) and emit a record.  If I read things correctly that's not currently possible in python on beam.  What all is needed?  (trying to figure out whether that is something that I am capable of and there is room for me to help with).  Looking for similar functionality to https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/ (though don't need alternate routing, nor is that example in python).