fix: PythonCodeSplitter emits invalid code for docstring-only bodies - #12331
fix: PythonCodeSplitter emits invalid code for docstring-only bodies#12331AmirF194 wants to merge 1 commit into
Conversation
|
@AmirF194 is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
|
|
Hi @AmirF194, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the |
|
Flagging for whoever reviews this: two more PRs opened after this one address the same issue, and between the three there's a real completeness difference worth being upfront about. #12346 and #12356 both touch This PR was opened first, but #12356 is the more complete fix as far as I can tell. Happy to stand down if you'd rather move forward with one of the other two. |
|
Hi @AmirF194, just a friendly reminder: this PR is still in draft because the Contributor License Agreement (CLA) hasn't been signed yet. We'd love to review your contribution! Please sign the CLA via the link in the |
Related Issues
Proposed Changes:
_strip_docstringonly refuses to strip a docstring when it shares a line with thedef/classheader or extends past the caller's slice. It never checks whether the docstring is the unit's only body statement. When it is, thebefore/afterslices it composes leave just the header with no body, which is invalid Python.This adds a third guard,
len(body) == 1, to the existing bail-out check, so a docstring-only unit is left untouched (docstring kept in the chunk content, nothing added tometa["docstrings"]) instead of being reduced to an empty body.How did you test it?
TestDocstringStripping::test_strip_docstrings_skips_body_that_is_only_a_docstring(docstring-only class, function, and method): fails on main with aSyntaxErrorfromast.parse, passes on this branch.test/components/preprocessors/test_python_code_splitter.pysuite: 72 passed.ruff check/ruff format --checkandcheck_imports.pyclean on the changed files.Checklist