`@JsonInclude(NON_EMPTY)` makes DoclingDocument serialization lossy

[rkudryashov]

Serve keeps "text": "" on formula items (do_formula_enrichment=false). The Java DoclingDocument still has text as "", but serialization drops it:

DoclingDocument doc = response.getDocument().getJsonContent();
// doc still has text == ""
byte[] bytes = jsonMapper.writeValueAsBytes(doc); // "text" omitted in JSON

Include.ALWAYS on the mapper does not help — class-level @JsonInclude(NON_EMPTY) wins. No raw json_content accessor either.

Wanted: lossless serialize, or raw json_content bytes/JsonNode.

Related: docling#3780, docling-java#386

[edeandrea]

@all-contributors add @rkudryashov for issue

[allcontributors]

@edeandrea

I couldn't determine any contributions to add, did you specify any contributions?
Please make sure to use valid contribution names.

[edeandrea]

@all-contributors add @rkudryashov for bug

[allcontributors]

@edeandrea

I've put up a pull request to add @rkudryashov! 🎉

[edeandrea]

Hi @rkudryashov I'm not sure I fully understand the issue? Could you explain a bit more?

Are you saying that "on the wire" the field is present in the json but with an empty value, but after deserialization the field is null?

[rkudryashov]

@edeandrea Not quite — after deserialization the Java field is still "" (not null). The loss happens on the way back out:

1. Serve wire JSON: "text": ""
2. DoclingDocument in memory: text == ""
3. jsonMapper.writeValueAsBytes(doc) → text key absent (class-level @JsonInclude(NON_EMPTY))

Setting Include.ALWAYS on the JsonMapper/ObjectMapper does not help, because @JsonInclude(NON_EMPTY) on the model class takes precedence.
So with the public API there is no straightforward way to persist a lossless DoclingDocument JSON that keeps "text": "" — writeValueAsBytes(doc) will drop those fields.

[edeandrea]

Let me ask a question - do you consider an empty string different from null?

[rkudryashov]

Yes. For us "" and null/absent are different.

text: "" is valid DoclingDocument state (e.g. formulas with enrichment off). After Jackson serialize + S3, empty strings are dropped → text is missing.

When we rehydrate that JSON we hit two failures:

1. Schema validation — we validate against the official DoclingDocument JSON Schema (DoclingDocument.json). Missing text fails validation; "text": "" passes.
2. Chunking — hybrid / json_docling chunking then fails on the same invalid document.