Skip to content

ci: run Python package test suites on pull requests - #1439

Closed
vivekvar-dl wants to merge 1 commit into
supermemoryai:mainfrom
vivekvar-dl:ci/python-package-tests
Closed

ci: run Python package test suites on pull requests#1439
vivekvar-dl wants to merge 1 commit into
supermemoryai:mainfrom
vivekvar-dl:ci/python-package-tests

Conversation

@vivekvar-dl

Copy link
Copy Markdown

Fixes #1417

Problem

The Python packages ship real test suites (~108 test functions across four packages) and nothing runs them: ci.yml is TypeScript-only, and the publish-*-python.yml workflows build and publish to PyPI without ever invoking pytest. As the issue notes, #1235 shipped exactly this way — a broken import that the existing suite would have caught at the offending commit.

Change

One path-filtered workflow, python-tests.yml, following the issue's proposal: it triggers on PRs (and pushes to main) touching any of the three packages whose suites currently pass, and runs each suite as a matrix job with fail-fast: false.

Per-package install recipes match how each suite is designed to run:

  • pipecat / cartesiapip install --no-deps -e . + pytest. Their tests stub the heavy runtime dependencies (pipecat-ai, loguru, pydantic), so installing them would only add minutes of CI time without testing anything extra.
  • agent-framework — full install, with agent-framework-core pinned to 1.0.0rc3: releases after the rc series no longer export BaseContextProvider, which breaks the package's imports entirely. The pin is commented in the workflow and should be removed once the package supports current releases (that incompatibility deserves its own fix, per the issue's closing note).

Known limitation, stated rather than silent: any change to one of the three packages runs all three suites. They complete in seconds, so per-package path filtering wasn't worth the extra machinery.

openai-sdk-python is excluded — its suite cannot currently be collected against recent supermemory releases (#1235). A comment in the workflow marks it for inclusion once that's fixed.

Verification

Each matrix job was reproduced locally in a fresh Python 3.12 venv using the exact install commands from the workflow, against current main:

  • pipecat: 1 passed
  • cartesia: 1 passed
  • agent-framework: 54 passed

This PR also makes the regression tests added in #1438 enforceable — today they would merge without ever being executed by CI.

The four Python packages ship ~108 test functions and none run in CI —
ci.yml is TypeScript-only and the publish workflows build without
testing, so packages are released on whatever the author ran locally
(supermemoryai#1235 shipped exactly this way).

Add a path-filtered workflow that runs pytest for the three packages
whose suites currently pass: pipecat and cartesia install with
--no-deps since their tests stub the heavy runtime dependencies, and
agent-framework pins agent-framework-core==1.0.0rc3 because later
releases dropped BaseContextProvider and break the package's imports.

openai-sdk-python is excluded until its import against recent
supermemory releases is fixed (supermemoryai#1235).

Fixes supermemoryai#1417
@vivekvar-dl

Copy link
Copy Markdown
Author

Closing this one — I hadn't spotted that #1418 already proposed the same three-package matrix a few days earlier, and #1431 now covers the four packages from the maintainer side. No point in a third workflow PR for the same issue.

One data point from this PR that may be useful to whichever lands: each matrix job here was reproduced in a fresh Python 3.12 venv against current main — pipecat and cartesia pass with --no-deps installs (their suites stub the heavy deps), and agent-framework needs agent-framework-core==1.0.0rc3 pinned, since later releases dropped BaseContextProvider and break the package's imports entirely. Without the pin the suite fails at collection.

@vivekvar-dl vivekvar-dl closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK test suites never run in CI

1 participant