OSDir

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Jackson serialisation of GenericJson subclasses


Hi everyone!!

I'm working on BEAM-4257 issue and the approach I'm following is to create a new class 'BigQueryInsertError' that also extends 'GenericJson' and that contains three keys 'TableRow row', 'TableDataInsertAllResponse.InsertErrors error', and 'TableReference ref' and use this type as the contained items returned by WriteResults.getFailedInserts

I have now to create a Coder for this new type and I'm following the TableRowJsonCoder way https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowJsonCoder.java#L34 by relying on Jackson's ObjectMapper and StringUtf8Encoder.

The problem is that I always get errors when deserialising as it deserialises the inner TableRow as a LinkedHashMap and fails when trying to assign it. Here you can see the full stacktrace: https://pastebin.com/MkUD9L3W

Testing it a bit further I've spotted other GenericJson subclasses that cannot be encoded/decoded following that method. For example TableDataInsertAllResponse.InsertErrors itself. See the example below:

TableDataInsertAllResponse.InsertErrors err = new TableDataInsertAllResponse.InsertErrors().setIndex(0L);
ObjectMapper mapper = new ObjectMapper().disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
mapper.readValue(mapper.writeValueAsString(err), TableDataInsertAllResponse.InsertErrors.class);

Fails with a similar error, but in this case is because it deserialises the index into an int: https://pastebin.com/bzXMR3z5

So a couple of questions here:
* Which is the appropriate way of encoding/decoding GenericJson subclasses? (Maybe this issues can be tackled using Jackson's type annotations, but I'm quite a newbie on Jackson and I couldn't figure out how)
* This will (hopefully) be my very first contribution to Apache Beam and I'd like to get some feedback/comments/ideas/... on the issue and the suggested solution.

Thanks everyone!