fix: deep-copy document metadata in PythonCodeSplitter - #12424
Open
sainikhiljuluri wants to merge 1 commit into
Open
fix: deep-copy document metadata in PythonCodeSplitter#12424sainikhiljuluri wants to merge 1 commit into
sainikhiljuluri wants to merge 1 commit into
Conversation
PythonCodeSplitter copied the input document's metadata shallowly, so nested values such as a list under meta["tags"] were shared between every chunk and with the input document. Editing one chunk's metadata changed all the others and the caller's document. This is the same defect fixed for the Markdown, CSV and hierarchical splitters in deepset-ai#12249; PythonCodeSplitter was not covered there. The secondary line-based split of oversized units had it too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sainikhiljuluri
requested review from
julian-risch
and removed request for
a team
August 21, 2026 06:10
|
@sainikhiljuluri is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
Hi @sainikhiljuluri, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the |
HaystackBot
marked this pull request as draft
August 21, 2026 07:31
HaystackBot
marked this pull request as ready for review
August 21, 2026 08:01
Contributor
|
Thanks for signing the CLA, @sainikhiljuluri! 🎉 This PR is now ready for review again and the reviewer has been re-assigned. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PythonCodeSplittercopies the input document's metadata shallowly, so nested values are shared between every chunk and with the caller's document:Root cause
Two shallow copies:
_build_chunk_metadoesmeta.update({k: v for k, v in parent_doc.meta.items() ...}), so every chunk holds references into the input document's metadata._secondary_splitdoesmeta = dict(base_meta), so the pieces of an oversized unit share the same nested objects with each other.Fix
Deep-copy at both sites, matching the other splitters.
This is the same defect as #12248, fixed for the Markdown, CSV and hierarchical splitters by #12249.
PythonCodeSplitterwas not covered there. I checked the remaining splitters —DocumentSplitter,RecursiveDocumentSplitter,CSVDocumentCleanerandDocumentCleanerare already safe, so this should close out the family.Tests
Two regression tests in
TestFileNamePropagation, one per fixed site, written in the same shape as the one #12249 added totest_markdown_header_splitter.py.Reverting the fix fails both, on the leaked mutation itself:
test/components/preprocessors/is green: 369 passed, 10 skipped (69 -> 71 in this file).ruff checkandruff format --checkclean.Risk
Behavioural change is limited to chunks no longer aliasing the caller's metadata. Deep-copying runs once per chunk on metadata that was already being copied, so the added cost is proportional to metadata size, not document size.
AI assistance was used to write this change.