Implement error collection for deserialization (#1196)

[sri-adarsh-kumar]

PR: Implement error collection for deserialization (#1196)

AI Disclosure

This is an AI-generated solution. The requirements were specified by a human developer based on a 9-year-old feature request from @odrotbohm, and the implementation was fully built by Claude Code (an AI coding agent).

Overview

Adds opt-in error collection capability to Jackson deserialization, allowing applications to collect multiple deserialization errors instead of failing fast on the first error. This addresses issue #1196 requested by Oliver Drotbohm in 2016.

Target branch: 3.x (for Jackson 3.1.0)

Motivation

Currently, Jackson fails fast on the first deserialization error, requiring a fix-retry cycle to discover all problems in a payload. This is inefficient for:

• API validation: Return all validation errors to clients in one response
• Batch processing: Log all errors but continue processing
• Development tooling: Show developers all issues at once
• Data migration: Identify all problematic records upfront

Key Features

1. Opt-in Per-Call Design

• No global configuration - must explicitly call collectErrors() on ObjectReader
• Thread-safe - each call gets its own isolated error bucket
• Backward compatible - default behavior unchanged

2. DoS Protection

• Default limit: 100 errors per call (configurable)
• Hard failure when limit exceeded, with collected errors as suppressed exceptions
• Prevents memory exhaustion from malicious payloads

3. Best-Effort Collection

• Collects recoverable errors (type mismatches, unknown properties, etc.)
• Non-recoverable errors (malformed JSON, structural issues) still fail immediately
• Failed fields get default values (0 for primitives, null for objects)

4. JSON Pointer Paths (RFC 6901)

• Each error includes path to problematic field: /items/0/price
• Proper escaping: ~ → ~0, / → ~1
• Makes it easy to locate issues in complex nested structures

5. Thread Safety

• Per-call error buckets - safe to reuse ObjectReader across threads
• No shared state between calls

API Usage

Basic Usage

ObjectMapper mapper = new JsonMapper();
ObjectReader reader = mapper.readerFor(Order.class).collectErrors();

try {
    Order result = reader.readValueCollecting(json);
    // Success - all fields valid
} catch (DeferredBindingException ex) {
    System.out.println("Found " + ex.getProblems().size() + " problems:");
    for (CollectedProblem problem : ex.getProblems()) {
        System.out.println(problem.getPath() + ": " + problem.getMessage());
        // Can also access problem.getRawValue() to see bad input
    }
}

Custom Error Limit

// Collect up to 10 errors before hard failure
ObjectReader reader = mapper.readerFor(Order.class).collectErrors(10);

Multiple Input Types

// Supports all standard input types
Order order1 = reader.readValueCollecting(jsonString);
Order order2 = reader.readValueCollecting(jsonBytes);
Order order3 = reader.readValueCollecting(file);
Order order4 = reader.readValueCollecting(inputStream);
Order order5 = reader.readValueCollecting(reader);

Implementation Details

New Classes

1. CollectedProblem

Immutable value object representing a single deserialization error:

• path: JSON Pointer to problematic field (e.g., /items/0/price)
• message: Human-readable error message
• rawValue: The invalid input value that caused the error
• targetType: Expected Java type

2. DeferredBindingException

Aggregate exception thrown when errors are collected:

• problems: List of all collected errors
• partialResult: The partially deserialized object (with default values for failed fields)
• limitReached: Flag indicating if error limit was hit
• Extends DatabindException for compatibility

3. CollectingProblemHandler

Stateless problem handler that collects errors during deserialization:

• Implements DeserializationProblemHandler interface
• No shared state - all state passed via DeserializationContext
• Handles type mismatches, unknown properties, and other recoverable errors
• Returns default values to allow deserialization to continue

4. ObjectReader Extensions

New methods:

• collectErrors(): Enable collection with default limit (100)
• collectErrors(int maxProblems): Enable collection with custom limit
• readValueCollecting(...): Read with error collection (5 overloads for different input types)

Design Decisions

1. Per-call buckets: Each readValueCollecting() call gets a fresh error list stored in DeserializationContext attributes
2. Default values: Failed fields use Java defaults (0, false, null) to allow deserialization to continue
3. Suppressed exceptions: When hard failure occurs (limit reached, non-recoverable error), collected problems are attached as suppressed exceptions
4. No recursion: Skips children of unknown properties to avoid cascading errors

Testing

Comprehensive test suite with 31 tests covering:

Core Functionality

• Default fail-fast behavior (2 tests): Unchanged when not using error collection
• Per-call bucket isolation (2 tests): Successive and concurrent calls don't share errors
• Thread safety (1 test): 10 threads processing in parallel with isolated buckets

Edge Cases

• JSON Pointer escaping (4 tests): Tilde, slash, combined, array indices
• Limit behavior (3 tests): Default limit, custom limit, under limit
• Unknown properties (2 tests): Collection and child skipping
• Type coercions (6 tests): int, long, double, float, boolean, boxed types
• Root-level errors (2 tests): Non-recoverable, path formatting
• Hard failures (1 test): Suppressed exception attachment
• Message formatting (2 tests): Single error, multiple errors
• Input types (6 tests): byte[], File, InputStream, Reader, null handling, empty JSON

Backward Compatibility

• 100% backward compatible - no changes to existing behavior
• Feature is completely opt-in via explicit method calls
• New exception type extends existing DatabindException
• No performance impact when not using the feature
• No breaking changes to any existing APIs

Performance Considerations

• Negligible overhead when disabled: Feature only activates when explicitly requested
• Per-call allocation: Each call allocates a small ArrayList for errors (default capacity: 10)
• Early termination: Stops collecting at configured limit to prevent DoS
• No synchronization: Thread-safe through isolation, not locking

Use Cases

API Validation

@PostMapping("/orders")
public ResponseEntity<?> createOrder(@RequestBody String json) {
    try {
        Order order = orderReader.readValueCollecting(json);
        return ResponseEntity.ok(orderService.save(order));
    } catch (DeferredBindingException ex) {
        List<ValidationError> errors = ex.getProblems().stream()
            .map(p -> new ValidationError(p.getPath().toString(), p.getMessage()))
            .toList();
        return ResponseEntity.badRequest().body(errors);
    }
}

Batch Processing

for (String record : records) {
    try {
        Data data = reader.readValueCollecting(record);
        process(data);
    } catch (DeferredBindingException ex) {
        logger.error("Record validation failed with {} errors:",
                     ex.getProblems().size());
        ex.getProblems().forEach(p ->
            logger.error("  {}: {}", p.getPath(), p.getMessage()));
        // Continue with next record
    }
}

Limitations

1. Best-effort: Not all errors can be collected

   • Malformed JSON (missing braces) fails immediately
   • Structural problems fail immediately
   • Type conversion errors ARE collected
   • Unknown property errors ARE collected (if enabled)

2. Default values: Failed fields get defaults

   • Primitives: 0, false
   • Objects: null
   • May affect downstream validation logic

3. Error limit: Default 100 errors

   • Prevents DoS but may not catch all issues in very large payloads
   • Configurable per call

Future Enhancements

Potential future improvements (not in this PR):

• Structured error types (beyond just messages)
• Integration with Bean Validation (JSR-380)
• Custom error formatters
• Error severity levels
• Configurable default value strategies

---

Generated by: Claude Code (Anthropic)
Specified by: @sri-adarsh-kumar
Original feature request: @odrotbohm (2016)

[pjfanning]

again, avoid using fully qualified class names - use imports so the code can be kept concise

[JooHyukKim]

-1 on this PR itself due to the sheer amount of line changes.

Ai code assistants should really get trained on splitting tasks to safer(or at least feeling so) bits of progression

[sri-adarsh-kumar]

@JooHyukKim

Thanks for the feedback.

-1 on this PR itself due to the sheer amount of line changes.

If it's some consolation, the actual logic change in ObjectReader and CollectingProblemHandler is only like 1/3 of the total changes. Others are from domain classes, documentation and tests. As this is not a small change (atleast from my POV) this seemed reasonable for me.

Ai code assistants should really get trained on splitting tasks to safer(or at least feeling so) bits of progression

Reg Coding Assistants and splitting tasks, this is indeed true. Once their context window starts filling up their answers are not that reliable.

Let me explain my process how I overcome this (something I also do in my job).

1. Specify the requirements to Claude Sonnet 4.5 (in Claude Code) along with any context (in this case the Github issue and the underlying discussion) and ask it to make a technical spec.

2. Review the spec with GPT-5 and continuously iterate on it between Sonnet 4.5, GPT-5 and myself. Spec will itself have sub-sections and checklist that the Coding Agent can build piece by piece.

3. Build the finalised spec using the Coding Agent (Sonnet 4.5).

4. Self review manually as well as with GPT-5 until the solution is in a reviewable state

5. Push the solution for peer review

If you see any code blocks that could be improved/simplified or if you have any feedback to my development process (mentioned above), I would be happy to learn and improve on it in future.

[sri-adarsh-kumar]

@cowtowncoder Just wanted to check in as the PR hasn't been reviewed yet.
Please let me know if there is any input needed from my side.

[cowtowncoder]

@sri-adarsh-kumar I haven't had time to re-review, but not waiting for anything from your side. Biggest question is that of how API should work, functionality be exposed.

[cowtowncoder]

Should we not instead throw "full" DeferredBindingException? To avoid losing 100 (etc) collected problems?

[sri-adarsh-kumar]

This is how it should be. This is also verified under CollectingErrorsTest#defaultLimit() test

[cowtowncoder]

Instead of this, let's make use of DeserializationContext.setAttribute() -- DeserializationContext is per-call, mutable.

And to have access to the DC, need to refactor to have something like

protected <T> T readValueCollectingProblems(JsonParser p, DeserializationContextExt ctxt) 

that all public readValueCollectingProblems() call with parser, context created.

Finally, we should NOT call readValue(parser) because that would just create yet another DeserializationContext. This means it'll be necessary to call one _bind() or _bindAndClose().

[sri-adarsh-kumar]

Refactored readValueCollectingProblems so every overload shares a single DeserializationContextExt, stores the bucket via ctxt.setAttribute(...), and calls _bind/_collectingBindAndClose directly without creating new ObjectReader instances.

[cowtowncoder]

Ok. Phew! Had another pass and now I understand this much better. I like the fact this one of remaining "Most Wanted" issues. There are a few things to address so added quite a few notes. But we are getting there. :)

(and once again, I am EXCITED about solving this long-standing HIGHLY VOTED FOR feature request!)