Repository navigation
Conversation
get_messages deserialized every Redis list entry without per-entry error handling, so a single empty string or corrupt JSON entry would raise JSONDecodeError / ValueError and abort the whole read. CosmosHistoryProvider already skips such entries with a warning. Wrap each entry in try/except (ValueError, json.JSONDecodeError), log a warning, and continue, matching the Cosmos behavior.
There was a problem hiding this comment.
🟡 Changes recommended
Valid JSON entries with malformed message shapes can raise uncaught exceptions that abort the entire history read, preventing later valid messages from being returned.
1 open issue
What changed in this PR
This PR makes the Python Redis history provider more resilient by skipping entries that raise JSON decoding or value errors instead of aborting the entire history read.
Changes:
- Add per-entry error handling and warning logs to
get_messages. - Add a regression test confirming valid messages remain readable after empty or invalid JSON entries.
| File | Description |
|---|---|
python/packages/redis/tests/test_providers.py |
Tests recovery from empty and invalid JSON history entries. |
python/packages/redis/agent_framework_redis/_history_provider.py |
Catches deserialization value errors and logs skipped entries. |
Review effort: Balanced
| try: | ||
| messages.append(Message.from_dict(self._deserialize_json(serialized))) | ||
| except (ValueError, json.JSONDecodeError) as exc: | ||
| logger.warning("Skipping malformed Redis history entry: %s", exc) |
Valid JSON that deserializes to a non-dict (null, []) or a dict missing
required fields ({}) previously raised AttributeError/TypeError from
Message.from_dict, which was not caught and aborted the entire read.
Guard the decoded payload with an isinstance-dict check and broaden the
except clause to (ValueError, TypeError, AttributeError, JSONDecodeError).
|
Thanks for the contribution, lisry-1. The code review is clean, but this changes malformed-history handling from fail-fast to skip-and-warn, so the PR needs the required Related Issue answer before approval. Please link an agreed issue with a closing keyword, or document a maintainer-approved exception. Missing AI Assistance alone is not a blocker. |
|
Thanks Eduard van Valkenburg (@eavanvalkenburg). I've opened #9974 to track this bug and linked it in the PR's Related Issue section ( |
|
lisry-1 The new warning logs |
…ormed history Message.from_dict embeds the payload's type value in mismatch errors, so logging str(exc) copies untrusted history data into application logs. Log the exception class name instead and add a regression asserting the sensitive value is absent from the warning.
|
Eduard van Valkenburg (@eavanvalkenburg) thanks for catching that — you're right, I've pushed a fix:
|
| if not isinstance(payload, dict): | ||
| raise ValueError(f"Expected a JSON object, got {type(payload).__name__}") | ||
| messages.append(Message.from_dict(payload)) | ||
| except (ValueError, TypeError, AttributeError, json.JSONDecodeError) as exc: |
There was a problem hiding this comment.
lisry-1 Skipping a malformed middle record can duplicate the valid history on the next save — For stored history [A, malformed, C], get_messages() now returns [A, C]. If the caller then saves the full transcript [A, B, C, D], the current contiguous-overlap check can append all four messages and persist [A, C, A, B, C, D]. Please make save-time deduplication account for skipped records and add this replay regression.
| if not isinstance(payload, dict): | ||
| raise ValueError(f"Expected a JSON object, got {type(payload).__name__}") | ||
| messages.append(Message.from_dict(payload)) | ||
| except (ValueError, TypeError, AttributeError, json.JSONDecodeError) as exc: |
There was a problem hiding this comment.
Skipping one record here can break function-call/result pairing: dropping a malformed tool-result record leaves its assistant call without a terminal result, while dropping a malformed call record can leave an orphaned tool result. Those lossy histories can survive serialization and make the next provider request invalid. Could we recover or reject complete tool exchanges atomically rather than individual records, and add corrupted call/result regressions consistent with the function-calling loop contract?
| payload = self._deserialize_json(serialized) | ||
| if not isinstance(payload, dict): | ||
| raise ValueError(f"Expected a JSON object, got {type(payload).__name__}") | ||
| messages.append(Message.from_dict(payload)) |
There was a problem hiding this comment.
lisry-1, non-string roles still pass this recovery boundary. For example, {"type":"chat_message","role":null,"contents":[]} satisfies the dictionary check and Message.from_dict() accepts role=None, so get_messages() returns an invalid message that provider serializers later forward and reject. Please validate that the role is a supported string before accepting the entry, and add null and non-string role regressions.
|
lisry-1 Maintainer direction on #9974 is to support recovery only when it is atomic and replay-safe. The current per-entry skip can orphan function-call/result pairs and can make full-transcript replay append duplicate history. Please rework this PR so recovery preserves complete exchanges and save-time deduplication; fail closed when a malformed record cannot be associated safely. Add corrupt-pair, unclassifiable-record, and replay regressions, keep diagnostics payload-free, and restore |

Summary
RedisHistoryProvider.get_messagesdeserialized every Redis list entry without per-entry error handling. A single empty string or corrupt JSON entry would raiseJSONDecodeError/ValueErrorand abort the entire read, even when the remaining entries were valid.CosmosHistoryProvider.get_messagesalready handles this case by skipping malformed entries with a warning (see_history_provider.pyinazure-cosmos).Change
Wrap each entry in
try/except (ValueError, json.JSONDecodeError), log a warning, and continue, matching the Cosmos behavior. Added a regression test that feeds["", "not-json", <valid>]and asserts the valid message is still returned.