osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

BigQuery FILE_LOADS recover from errors


Hi all!!

On my pipeline I want to dump some data into BQ using FILE_LOADS write method and I can't see how would I recover from errors (i.e. on the pipeline detect which records couldn't be inserted and store it somewhere else for further inspection) as the WriteTables transform throws an Exception after BatchLoads.MAX_RETRY_JOBS retries...

How could I approach that?

Thanks!