[Java] Fix default values of fields with UNION schemas where the NULL schema comes first#3660
Open
lw-mcno wants to merge 1 commit intoapache:mainfrom
Open
Conversation
… schema comes first
The assumption in JacksonUtils#toObject(JsonNode, Schema), that the actual data schema in a UNION schema (of some data with NULL), always comes first, is incorrect.
For example, a field with this schema will have its default value converted incorrectly:
```
{
"name": "exampleDateTime",
"type": ["null", { "type": "long", "logicalType": "local-timestamp-millis" }],
"default": 1746088255000
}
```
Here the "null" schema explicitly comes first and the assumption is no longer correct (JacksonUtils tries to interpret the value 1746088255000 using the NULL schema).
Another example using Avro IDL (nullable field, containing an array of nullable items):
```
record SomeRecord {
union{array<int?>, null} optArrayOptItemVal = [4];
}
```
In this case, the "optArrayOptItemVal" field fulfills the assumption, but the array item's schema does not. On my machine, with Avro 1.12.1, the item's schema is the following:
```
"type": ["null", { "type": "int" }]
```
which also fails the non-NULL schema first assumption.
97004ed to
e15f7cb
Compare
Author
|
I've updated the PR with fixed formatting on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The assumption in JacksonUtils#toObject(JsonNode, Schema), that the actual data schema in a UNION schema (of some data with NULL), always comes first, is incorrect.
For example, a field with this schema will have its default value converted incorrectly:
Here the "null" schema explicitly comes first and the assumption is no longer correct (JacksonUtils tries to interpret the value 1746088255000 using the NULL schema).
Another example using Avro IDL (nullable field, containing an array of nullable items):
In this case, the "optArrayOptItemVal" field fulfills the assumption, but the array item's schema does not. On my machine, with Avro 1.12.1, the item's schema is the following:
which also fails the non-NULL schema first assumption.
All of the above examples are successfully parsed or converted through the Avro IDL library to a Schema object.
What is the purpose of the change
Fix the retrieval of schema fields' default values (JacksonUtils) when the schema(s) consist of UNIONs with NULLs (as first type in the UNION).
Verifying this change
This change added tests and can be verified as follows:
Documentation