osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: DataSet - Broadcast set in output format


Hi Bastien,

the OutputFormat specifies how a given record is written to an external system. The DataSink using these formats do not support using broadcast variables. This is currently a limitation of Flink.

What you could do is to introduce a mapper before your sink which enriches the records with respect to the broadcast variable. The OutputFormat could then react to this additional information.

Cheers,
Till

On Thu, Nov 22, 2018 at 2:57 PM bastien dine <bastien.dine@xxxxxxxxx> wrote:
Hello,

I would like to use a broadcast variable in my outputformat (to pass some information, and control execution flow)
How would I do it ? 
.output does not have a .withBroadcast function as it does not extends SingleInputUdfOperator


------------------

Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io