Skip to content

fix: preserve zero scores in document joiner#11355

Open
wolfkill wants to merge 1 commit into
deepset-ai:mainfrom
wolfkill:fix/document-joiner-zero-score
Open

fix: preserve zero scores in document joiner#11355
wolfkill wants to merge 1 commit into
deepset-ai:mainfrom
wolfkill:fix/document-joiner-zero-score

Conversation

@wolfkill
Copy link
Copy Markdown

Summary

  • fix DocumentJoiner duplicate selection in concatenate mode so score=0.0 is compared as a real score
  • align merge-mode score handling with the same explicit None check
  • add a regression test and release note

Why

DocumentJoiner._concatenate used a truthiness check for scores. Since 0.0 is falsy in Python, a duplicate document with score=0.0 was treated like an unscored document and could lose to a worse negative score.

Testing

  • hatch -e test run pytest test/components/joiners/test_document_joiner.py::TestDocumentJoiner::test_run_with_concatenate_join_mode_keeps_duplicate_document_with_zero_score -q
  • hatch -e test run pytest test/components/joiners/test_document_joiner.py -q
  • env -u ALL_PROXY -u HTTPS_PROXY -u HTTP_PROXY -u all_proxy -u https_proxy -u http_proxy hatch -e test run pytest test/components/joiners -q
  • hatch run fmt-check haystack/components/joiners/document_joiner.py test/components/joiners/test_document_joiner.py
  • hatch run test:types haystack/components/joiners/document_joiner.py
  • hatch -e test run python -m compileall -q haystack/components/joiners/document_joiner.py test/components/joiners/test_document_joiner.py
  • git diff --check

Closes #11352

@wolfkill wolfkill requested a review from a team as a code owner May 21, 2026 02:15
@wolfkill wolfkill requested review from anakin87 and removed request for a team May 21, 2026 02:15
@vercel
Copy link
Copy Markdown

vercel Bot commented May 21, 2026

@wolfkill is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@anakin87
Copy link
Copy Markdown
Member

This is a duplicate of #11353 and the CLA is not signed.
For these reasons, I won't review it for the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DocumentJoiner concatenate mode incorrectly drops documents with score=0.0 during deduplication

3 participants