Re: BigQueryIO Failed Insert
Looking at the code, not found is considered a transient error which will be retried until the table is created. Try using neverRetry and see if that gives you the results you expect?
On 2018/05/04 17:16:39, Strahinja Vladetic <strahinja.vladetic@xxxxxxxxx> wrote:
> Hey all,
> I have been running into an issue with streaming writes to Big Query using
> a custom destination and I can't seem to find an appropriate solution in
> the docs so I was hoping someone here could help.
> The Issue: Specifying a table destination which does not exist causes the
> message to fail to be
> inserted and there is no way to handle the error.
> The problem is that the message does not appear in the `getFailedInserts()`
> function so I don't see a way of getting a handle on this error so that I
> could, for example, direct the message to a poison or deadletter queue.
> Some code for reference....
> Is there a way of handling this use case in the current beam SDK?
> Strahinja Vladetic