Skip to content

Adding signed ingestion#58

Merged
kdeden merged 6 commits intomainfrom
feat/signed-ingestion
Mar 3, 2026
Merged

Adding signed ingestion#58
kdeden merged 6 commits intomainfrom
feat/signed-ingestion

Conversation

@bgmello
Copy link
Contributor

@bgmello bgmello commented Feb 11, 2026

This PR adds support for a new signed URL file upload workflow as an alternative to the existing direct ingest endpoint.

This is an optional flag that speeds up uploads and makes them more reliable, specifically for Aleph Pro. But OpenAleph may choose to implement this as well.

Instead of a single multipart POST to /collections/{id}/ingest, the new flow works in three steps:

  • request a signed upload URL;
  • PUT the file content to that URL;
  • than create the document record via /collections/{id}/document.

This is opt-in via a --signed-url flag on the CLI crawldir command, and a signed_url=True parameter on the AlephAPI.ingest_upload() Python method.

The old ingest path remains the default so nothing breaks for existing users.

New unit and integration tests were added and all pass.

@bgmello bgmello closed this Feb 11, 2026
@bgmello bgmello reopened this Feb 18, 2026
@kdeden kdeden merged commit 03434fe into main Mar 3, 2026
3 checks passed
@kdeden kdeden deleted the feat/signed-ingestion branch March 3, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants