Skip to content

fix(database): decode small bytes artifacts in get_artifacts - #537

Open
harshajv7981 wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
harshajv7981:fix/small-bytes-artifacts
Open

harshajv7981 wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
harshajv7981:fix/small-bytes-artifacts

Conversation

@harshajv7981

Copy link
Copy Markdown

Fixes #536

Problem

store_artifacts encodes small bytes artifacts as {"__bytes__": <base64>}, but get_artifacts decodes them with a plain json.loads, so they come back as dicts instead of bytes.

Change

Pass the existing _artifact_deserializer as object_hook in get_artifacts. It is the counterpart of _artifact_serializer and was otherwise unused. Strings, large on-disk artifacts and the checkpoint format are unchanged.

Verification

  • New regression test tests/test_artifacts.py::TestDatabaseArtifacts::test_store_small_bytes_artifacts fails before this change and passes after.
  • Artifact, database, checkpoint, prompt sampler and coevolution tests: 97 passed, no new failures.

I found this with patchwork, an automated code-audit tool I run, and verified the reproduction test in an isolated sandbox. If tool-assisted reports aren't welcome here, let me know and I won't send more.

store_artifacts encodes bytes as {"__bytes__": <base64>} but get_artifacts
decoded with plain json.loads, so small bytes artifacts came back as dicts.
Use the existing _artifact_deserializer as the object_hook.

Fixes algorithmicsuperintelligence#536
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

get_artifacts returns small bytes artifacts as {'__bytes__': ...} dicts

2 participants