perf(schema): cache compiled JSONPath expressions in document validation

[vishal-bala]

Motivation

extract_from_json_path called jsonpath_ng.parse on every invocation, and validate_object calls it once per field per document on the JSON storage path. Field paths are fixed when the schema is constructed, so every parse after the first repeated work already done.

Parsing is the expensive half of a JSONPath lookup. jsonpath-ng builds its grammar with PLY, so a single parse("$.t0") measures 1.83 ms against microseconds to evaluate the result. Measured on an 11-field JSON schema with validation enabled:

reparsing every field   18.683 ms/doc       53 docs/s
cache warm               0.062 ms/doc   16,138 docs/s

That is a 302x difference, or roughly five hours spent inside the PLY parser on a one-million document load. SearchIndex recommends validate_on_load=True in its own docstring, so the cost sat on a signposted path.

Changes

Caching the compiled expression

The parse moves behind an lru_cache in a new _compile_json_path helper, and extract_from_json_path becomes a lookup plus an evaluation. The cache is keyed on the raw path, before the leading-$ normalisation, so field and $.field occupy separate entries that compile to equivalent expressions. That duplication is harmless and it keeps the lookup on the cheapest available path.

Sharing one parsed expression across calls is safe because the expression holds no per-evaluation state: find() reads the expression and returns fresh match objects.

Tests

Four tests cover the behaviour rather than the timing, since a timing assertion would be flaky. Two count calls into jsonpath_parse to confirm that a repeated path compiles once and that distinct paths are not collapsed onto one expression. One evaluates a single cached expression against several objects, including one where the path misses, which guards the no-shared-state assumption the cache rests on. One confirms both spellings of a path remain equivalent.

Notes

SchemaModelGenerator.get_model_for_schema builds its cache key by re-serialising the whole schema per document, which dominates the remaining cost on the HASH storage path. Addressing it means keying on schema identity plus a mutation counter, which changes what happens when a schema is mutated in place, so it is left for a separate change rather than folded in here.

Release Notes

Loading documents into a JSON index with validate_on_load=True is substantially faster. JSONPath expressions derived from the schema are now compiled once and reused, instead of being re-parsed for every field of every document.