Skip to content

fix: PythonCodeSplitter emits invalid code for docstring-only bodies - #12331

Draft
AmirF194 wants to merge 1 commit into
deepset-ai:mainfrom
AmirF194:fix/python-code-splitter-docstring-only-body
Draft

fix: PythonCodeSplitter emits invalid code for docstring-only bodies#12331
AmirF194 wants to merge 1 commit into
deepset-ai:mainfrom
AmirF194:fix/python-code-splitter-docstring-only-body

Conversation

@AmirF194

Copy link
Copy Markdown

Related Issues

Proposed Changes:

_strip_docstring only refuses to strip a docstring when it shares a line with the def/class header or extends past the caller's slice. It never checks whether the docstring is the unit's only body statement. When it is, the before/after slices it composes leave just the header with no body, which is invalid Python.

This adds a third guard, len(body) == 1, to the existing bail-out check, so a docstring-only unit is left untouched (docstring kept in the chunk content, nothing added to meta["docstrings"]) instead of being reduced to an empty body.

How did you test it?

  • New parametrized test, TestDocstringStripping::test_strip_docstrings_skips_body_that_is_only_a_docstring (docstring-only class, function, and method): fails on main with a SyntaxError from ast.parse, passes on this branch.
  • Full test/components/preprocessors/test_python_code_splitter.py suite: 72 passed.
  • ruff check / ruff format --check and check_imports.py clean on the changed files.
  • Not checked: whether the same class of bug exists elsewhere in the splitter for units this fix does not touch.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title.
  • I have documented my code.
  • I have added a release note file.
  • I have run pre-commit hooks and fixed any issue.

@AmirF194
AmirF194 requested a review from a team as a code owner August 13, 2026 07:04
@AmirF194
AmirF194 requested review from julian-risch and removed request for a team August 13, 2026 07:04
@vercel

vercel Bot commented Aug 13, 2026

Copy link
Copy Markdown

@AmirF194 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @AmirF194, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@AmirF194

Copy link
Copy Markdown
Author

Flagging for whoever reviews this: two more PRs opened after this one address the same issue, and between the three there's a real completeness difference worth being upfront about.

#12346 and #12356 both touch _strip_docstring for the same bug. This PR's guard (len(body) == 1) covers the literal repro in #12330 (a unit whose full AST body is just the docstring), but #12356's body[1].lineno > unit_end check additionally catches a class whose header slice holds only the docstring even though the class has methods elsewhere in the file, since body there is the full class body, not the slice. That case isn't in the original repro, but it's the same underlying bug and #12356 handles it. #12346 takes a different approach (insert a pass placeholder rather than skip the strip) and also covers that case.

This PR was opened first, but #12356 is the more complete fix as far as I can tell. Happy to stand down if you'd rather move forward with one of the other two.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @AmirF194, just a friendly reminder: this PR is still in draft because the Contributor License Agreement (CLA) hasn't been signed yet. We'd love to review your contribution! Please sign the CLA via the link in the license/cla check, and this PR will automatically be marked ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-pending PR is in draft until the contributor signs the CLA topic:tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PythonCodeSplitter with strip_docstrings=True emits invalid Python for docstring-only function/method/class bodies

3 participants