Skip to content

Quoted string keys break extract/Delta; escaping approach needs compatibility direction #648

Open

Description

@chivector

On current dev (59acdd9), a dictionary key containing both single and double quotes produces a path that cannot be consumed by parse_path, extract, or Delta.

from deepdiff import DeepDiff, Delta, extract

key = '''Bob's "plan"'''
before, after = {key: 1}, {key: 2}
diff = DeepDiff(before, after)
path = next(iter(diff['values_changed']))
print(path)
extract(after, path)  # KeyError
# Independently, before + Delta(diff, bidirectional=True, raise_errors=True)
# raises DeltaError instead of producing after.

Expected: the returned path identifies the original key and the diff can be replayed in both directions. The same failure occurs for added/removed quoted keys. This was reproduced on Windows / CPython 3.12.14.

I prepared a prototype and regression tests using explicit u-prefixed Python string literals only for keys containing both quote types. This separates escaped literals from existing raw-backslash path semantics, including nested keys ending in a backslash. A canonical str.__repr__ call also handles string subclasses.

The prototype has a compatibility trade-off that needs your direction: old malformed paths still worked as opaque exclude_paths strings. Changing their canonical spelling means persisted old exclusions stop matching, while exclusions regenerated from the new output work. Ordinary keys, single-quote-type keys and old raw-backslash paths retain their existing spelling and parsing.

Would a canonical escaped representation with migration guidance be acceptable, or should the solution preserve old opaque exclusion strings as well? I have not opened a PR because this compatibility question is unresolved.

Validation of the prototype:

  • Unchanged source: 23 new regression failures; fixed related path/model/glob/search/Delta/text tests: 577 passed, 3 skipped.
  • Broader declared-dependency suite including slow tests: 1340 passed, 8 skipped, 2 deselected, 95% source coverage. Security payload tests and restricted-unpickler payload cases were not run; no complete Python/OS matrix is claimed.
  • 4,680 generated short-key cases and 3,544 old-valid single paths / 3,721 old-valid nested paths passed, plus independent public API, persisted-Delta and input-preservation checks.
  • Pyright and blocking Flake8 checks pass; independent agent review found no remaining implementation issue after the string-subclass fix.

Existing include_paths assembly also has a both-quote limitation on unchanged source and remains outside this prototype; I am not claiming every path consumer is fixed. #640 handles byte keys, while #351/#430 concern earlier quoting cases; this report covers ordinary string keys containing both quote types.

AI assistance: OpenAI Codex investigated and implemented this user-authorized prototype, ran the checks and obtained an independent agent review. No separate human review or personal production incident is claimed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions