|
| 1 | +# Probe Tests Summary |
| 2 | + |
| 3 | +This document summarizes all the probe tests added to identify potential issues in the JTD implementation. |
| 4 | + |
| 5 | +## Test Files Added |
| 6 | + |
| 7 | +### 1. `ErrorFormatComplianceProbe.java` |
| 8 | +**Purpose**: Verify RFC 8927 error format compliance |
| 9 | + |
| 10 | +**Key Probes**: |
| 11 | +- Error format should be `{instancePath, schemaPath}` pairs (not enriched strings) |
| 12 | +- Schema paths should point to schema keywords (`/type`, `/properties/foo`, etc.) |
| 13 | +- Instance paths should be RFC 6901 JSON Pointers |
| 14 | +- Multiple errors should ALL be collected |
| 15 | +- Error consistency across different violations |
| 16 | + |
| 17 | +**Expected Issues to Find**: |
| 18 | +- Implementation returns enriched strings like `[off=N ptr=# via=#] message` |
| 19 | +- No RFC 6901 compliance for instance paths |
| 20 | +- Schema paths not tracked separately |
| 21 | + |
| 22 | +### 2. `DiscriminatorEdgeCaseProbe.java` |
| 23 | +**Purpose**: Test discriminator schema edge cases |
| 24 | + |
| 25 | +**Key Probes**: |
| 26 | +- Missing vs non-string discriminator error distinction |
| 27 | +- Discriminator tag exemption from additionalProperties |
| 28 | +- Discriminator key redefinition in properties (compile-time check) |
| 29 | +- Discriminator key in optionalProperties (compile-time check) |
| 30 | +- Empty discriminator mapping |
| 31 | +- Discriminator with additionalProperties: true |
| 32 | +- Nested discriminator |
| 33 | +- Discriminator error short-circuiting |
| 34 | +- Discriminator with null/empty string values |
| 35 | +- Multiple discriminator values with common properties |
| 36 | +- Conflicting property types across variants |
| 37 | + |
| 38 | +**Expected Issues to Find**: |
| 39 | +- Missing vs non-string discriminator errors may be conflated |
| 40 | +- Discriminator key not properly exempted from additionalProperties |
| 41 | +- Variant validation may continue after discriminator error |
| 42 | + |
| 43 | +### 3. `PropertiesEdgeCaseProbe.java` |
| 44 | +**Purpose**: Test properties form edge cases |
| 45 | + |
| 46 | +**Key Probes**: |
| 47 | +- Empty properties with additionalProperties: false |
| 48 | +- Empty properties without additionalProperties (default behavior) |
| 49 | +- Special characters in property names (dots, slashes, spaces, null char) |
| 50 | +- Multiple additional properties all reported |
| 51 | +- Required property with null value vs missing property |
| 52 | +- Optional property with various values |
| 53 | +- Overlapping required and optional keys (compile-time check) |
| 54 | +- Nested properties with different additionalProperties settings |
| 55 | +- Property validation order |
| 56 | +- Empty string property names |
| 57 | +- Very deep nesting (50 levels) |
| 58 | +- Large number of properties (100) |
| 59 | +- Property name collisions with prototype pollution concerns |
| 60 | + |
| 61 | +**Expected Issues to Find**: |
| 62 | +- Empty properties may not properly reject additional properties |
| 63 | +- Special characters in property names may not be handled correctly |
| 64 | +- Deep nesting performance issues |
| 65 | +- Prototype pollution vulnerability |
| 66 | + |
| 67 | +### 4. `TypeValidationEdgeCaseProbe.java` |
| 68 | +**Purpose**: Test type validation edge cases |
| 69 | + |
| 70 | +**Key Probes**: |
| 71 | +- Integer boundary values (exact min/max for all integer types) |
| 72 | +- Integer fractional detection (3.0 is int, 3.1 is not) |
| 73 | +- Scientific notation handling |
| 74 | +- Float types accept any number |
| 75 | +- Timestamp format variations |
| 76 | +- Invalid timestamp formats |
| 77 | +- Boolean type strictness |
| 78 | +- String type strictness |
| 79 | +- Very large integers |
| 80 | +- Zero values for all integer types |
| 81 | +- Negative values for unsigned types |
| 82 | +- No type coercion |
| 83 | +- BigDecimal values exceeding long precision |
| 84 | + |
| 85 | +**Expected Issues to Find**: |
| 86 | +- Integer validation may not properly handle fractional values |
| 87 | +- Scientific notation may not be handled correctly |
| 88 | +- Timestamp regex may be too strict or too lenient |
| 89 | +- Very large numbers may cause precision issues |
| 90 | + |
| 91 | +### 5. `RefEdgeCaseProbe.java` |
| 92 | +**Purpose**: Test ref schema edge cases |
| 93 | + |
| 94 | +**Key Probes**: |
| 95 | +- Forward reference resolution |
| 96 | +- Mutual recursion |
| 97 | +- Deeply nested refs (50 levels) |
| 98 | +- Ref in elements context |
| 99 | +- Ref in values context |
| 100 | +- Ref in properties context |
| 101 | +- Ref to empty schema |
| 102 | +- Ref to discriminator |
| 103 | +- Ref to nullable schema |
| 104 | +- Ref to elements/values/properties/enum/type |
| 105 | +- Unused definitions |
| 106 | +- Multiple refs to same definition |
| 107 | +- Complex recursive ref (binary tree) |
| 108 | +- Ref in optionalProperties |
| 109 | +- Multi-level ref resolution |
| 110 | + |
| 111 | +**Expected Issues to Find**: |
| 112 | +- Forward references may not resolve correctly |
| 113 | +- Deep nesting may cause stack issues |
| 114 | +- Circular references may not be handled |
| 115 | + |
| 116 | +### 6. `ElementsEdgeCaseProbe.java` |
| 117 | +**Purpose**: Test elements form edge cases |
| 118 | + |
| 119 | +**Key Probes**: |
| 120 | +- Empty array validation |
| 121 | +- Single element array |
| 122 | +- Nested elements (2D/3D arrays) |
| 123 | +- Elements with properties schema |
| 124 | +- Elements with discriminator |
| 125 | +- Elements error collection (all elements validated) |
| 126 | +- Elements with strict nested objects |
| 127 | +- Large array performance (1000 elements) |
| 128 | +- Array with null elements |
| 129 | +- Array with nullable elements |
| 130 | +- Mixed valid and invalid elements |
| 131 | +- Empty schema in elements |
| 132 | +- Elements with ref |
| 133 | +- Elements error path construction |
| 134 | +- Nested elements error path |
| 135 | +- Elements with object additional properties |
| 136 | +- Multiple arrays in same schema |
| 137 | +- Elements with values schema |
| 138 | +- Elements with enum |
| 139 | +- Sparse array |
| 140 | +- Elements with complex nested structure |
| 141 | +- Array type guard |
| 142 | +- Elements with timestamp/boolean/float types |
| 143 | +- Elements with integer boundaries |
| 144 | + |
| 145 | +**Expected Issues to Find**: |
| 146 | +- Error path construction for nested arrays |
| 147 | +- Large array performance |
| 148 | +- Error collection for all invalid elements |
| 149 | + |
| 150 | +### 7. `NullableEdgeCaseProbe.java` |
| 151 | +**Purpose**: Test nullable modifier edge cases |
| 152 | + |
| 153 | +**Key Probes**: |
| 154 | +- Nullable type accepts null |
| 155 | +- Non-nullable type rejects null |
| 156 | +- Nullable explicit false |
| 157 | +- Nullable on empty schema |
| 158 | +- Nullable on enum/elements/properties/values/discriminator/ref |
| 159 | +- Nested nullable |
| 160 | +- Nullable property value |
| 161 | +- Nullable required property |
| 162 | +- Nullable optional property |
| 163 | +- Nullable array element |
| 164 | +- Nullable values in object |
| 165 | +- Nullable with all integer types |
| 166 | +- Nullable with float types |
| 167 | +- Nullable with timestamp/boolean |
| 168 | +- Nullable error messages |
| 169 | +- Nullable must be boolean compilation check |
| 170 | +- Nullable in definitions |
| 171 | +- Nullable with complex nested schema |
| 172 | +- Nullable discriminator mapping value (compile-time check) |
| 173 | +- Multiple nullable fields |
| 174 | +- Nullable with additionalProperties |
| 175 | + |
| 176 | +**Expected Issues to Find**: |
| 177 | +- Nullable discriminator mapping may not be rejected at compile time |
| 178 | +- Nullable compilation may accept non-boolean values |
| 179 | + |
| 180 | +## Running the Tests |
| 181 | + |
| 182 | +```bash |
| 183 | +# Run all probe tests |
| 184 | +$(command -v mvnd || command -v mvn || command -v ./mvnw) test -pl json-java21-jtd -Dtest="*Probe" |
| 185 | + |
| 186 | +# Run specific probe test |
| 187 | +$(command -v mvnd || command -v mvn || command -v ./mvnw) test -pl json-java21-jtd -Dtest=ErrorFormatComplianceProbe |
| 188 | + |
| 189 | +# Run with detailed logging |
| 190 | +$(command -v mvnd || command -v mvn || command -v ./mvnw) test -pl json-java21-jtd -Dtest="*Probe" -Djava.util.logging.ConsoleHandler.level=FINE |
| 191 | +``` |
| 192 | + |
| 193 | +## Expected Outcomes |
| 194 | + |
| 195 | +These tests are designed to **document current behavior** and **expose deviations** from the specification. They do not fix any code. |
| 196 | + |
| 197 | +Key deviations expected: |
| 198 | +1. **Error format**: Returns enriched strings instead of RFC 8927 error indicators |
| 199 | +2. **Schema paths**: Not tracked or reported per RFC 8927 |
| 200 | +3. **Discriminator**: May conflate missing vs non-string discriminator errors |
| 201 | +4. **Empty properties**: May not properly reject additional properties by default |
| 202 | +5. **Nullable discriminator mapping**: May not be rejected at compile time |
| 203 | + |
| 204 | +## Notes |
| 205 | + |
| 206 | +- All probe tests extend `JtdTestBase` for consistent logging |
| 207 | +- Tests use INFO level logging at method start as per AGENTS.md |
| 208 | +- Tests document both passing and failing assertions |
| 209 | +- Tests include comments explaining the expected vs actual behavior |
0 commit comments