[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Pass complex type to FlinkPipelineOptions

You can't set it via the command line. You'll need to programmatically set it.

Normally complex types are configured via using a JSON map as shown in this commit: https://github.com/apache/beam/commit/f1d4d41279ad93370a6151902fae6dbd13dde53b#diff-03b47195a404600b2f4576d4ab5fbbb3

This requires the pipeline option to have a JSON -> Complex type -> JSON mapping defined. Please consider contributing one if you find it useful.

On Thu, Jul 19, 2018 at 1:48 AM Jozef Vilcek <jozo.vilcek@xxxxxxxxx> wrote:

I am using Beam with Flink and want to set `stateBacked` via pipeline options available here

The property is an abstract class. I was not able to figure out so far what to pass to command line for initialization. How to provide such complex abstract type? Is it possible?
.. I am trying to use rocksdb backend with incremental checkpointing ..