SEP-1194: Implementation: Apply QA architecture to codebase by cortiz-percona · Pull Request #830 · percona/SEP

cortiz-percona · 2026-05-27T20:04:42Z

Summary

Implements the approved QA architecture (SEP-1191) in the repository: a layered testing model selected by markers, canonical external-service stubs, reference examples per lane, and CI integration for each lane.

Backend layout — adds tests/unit/ and tests/contract/ alongside the existing tests/app/ tree, plus tests/_stubs/ as the single source of truth for Casdoor / Nomad / PMM stubs. The lane markers (unit, integration, contract) are applied automatically by per-directory conftest.py via pytest_collection_modifyitems — no manual decoration required. Integration tests stay in tests/app/ (which mirrors app/ one-to-one); the directory is not renamed — the marker is the layer.
Reference examples — one reference test per layer documenting the canonical pattern:
- tests/unit/test_example_string_utils.py — pure logic, no I/O
- tests/app/test_example_api_route.py — TestClient against the mounted apps
- tests/contract/test_example_openapi_contract.py — Schemathesis against a mounted app's openapi() (skips cleanly until the dependency lands — see below)
- frontend/packages/e2e/tests/example.spec.ts — Playwright + Vite preview + API mocking
Canonical stubs — tests/_stubs/{casdoor,nomad,pmm}.py expose patch_* helpers that point at the real client surfaces (CasdoorSDK.introspect_token/get_user, NomadExecutor.dispatch_job, PMMRemoteAPI.get_nodes/get_services). Milestone M4 consolidates the per-plugin stubs here.
Makefile / CI — adds make test-unit, make test-integration, make test-contract; splits the lanes into separate jobs in .github/workflows/python.yaml; adds .github/workflows/audit.yaml (reusable pip-audit + pnpm audit) wired into ci.yml.
Guidelines — refreshes .github/instructions/tests.instructions.md and adds the operational playbook in docs/testing-guidelines.md + the architecture in docs/qa-architecture.md.

Note on the contract lane: schemathesis is not yet declared in pyproject.toml. The contract reference test pytest.importorskips it, so make test-contract and its CI job run and skip cleanly rather than failing — the lane is scaffolded now and becomes enforcing at milestone M6 when the dependency is added.

Tested

make test-unit — passes
make test-integration — passes (auto-integration marker applied to tests/app/)
make test-contract — runs and skips cleanly (schemathesis not installed; tolerated exit code 5)
make lint (ruff + djlint)
pnpm --filter @sep/e2e test:e2e — example.spec.ts passes against the Vite preview server with API mocking
CI workflows (python.yaml, audit.yaml) executed on the branch

Checklist

New/modified functions have type hints and rST docstrings
New tests added for new features or bug fixes
All tests pass locally
Pre-commit hooks pass (make run-pre-commit)
Database migrations generated if models changed (make makemigrations) — N/A (no model changes)
User-facing changes documented (README, inline help, UI text)
Configuration changes documented with examples — N/A
Changelog fragment added under changelog.d/ — N/A (internal-only tooling/test change)

Copilot

Pull request overview

This PR implements SEP’s approved QA/testing architecture by splitting the Python test suite into layered lanes (unit / integration / contract), centralizing reusable test factories/stubs, and wiring the new lanes into local tooling (Makefile) and CI (GitHub Actions). It primarily reorganizes and expands the test harness around the existing FastAPI multi-app layout (inventory, tasks, sep) so each lane can be run independently.

Changes:

Introduces tests/unit/, tests/integration/, and tests/contract/ with per-lane auto-marking via conftest.py, plus reference example tests for each lane.
Migrates existing tests/imports to the new structure (notably switching tests.app.factories → tests.factories) and adds canonical external-service stubs under tests/_stubs/.
Adds Make targets (test-unit, test-integration, test-contract) and updates CI to run these lanes separately; adds a reusable audit workflow and CI wiring.

Reviewed changes

Copilot reviewed 66 out of 262 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/unit/test_example_string_utils.py	Adds a reference unit-layer test covering pure string helpers.
tests/unit/conftest.py	Auto-applies the `unit` marker to tests under `tests/unit/`.
tests/unit/init.py	Marks the unit tests directory as a package (license header only).
tests/integration/test_example_api_route.py	Adds a reference integration test asserting each mounted app exposes `openapi.json`.
tests/integration/test_celery_signals.py	Adds integration tests for Celery correlation-ID/log-context signal helpers.
tests/integration/tasks/test_routes.py	Updates factories import path after test suite restructuring.
tests/integration/tasks/test_pmm_regression.py	Updates factories/import paths after restructuring.
tests/integration/tasks/test_models.py	Updates factories import path after restructuring.
tests/integration/tasks/test_deps.py	Updates factories import path after restructuring.
tests/integration/tasks/test_crud.py	Updates factories import path after restructuring.
tests/integration/tasks/test_config.py	Adds tests for `app.tasks.config` defaults/settings.
tests/integration/tasks/test_celery.py	Updates factories import path after restructuring.
tests/integration/tasks/periodic/test_routes.py	Updates factories import path after restructuring.
tests/integration/tasks/periodic/conftest.py	Adds periodic-task fixtures (SQLite + celery-beat tables + TestClient).
tests/integration/tasks/periodic/init.py	Marks periodic tests as a package (license header only).
tests/integration/tasks/execution/test_models.py	Updates factories import path after restructuring.
tests/integration/tasks/execution/test_exceptions.py	Adds tests for tasks execution exception types.
tests/integration/tasks/execution/executors/nomad/init.py	Marks Nomad executor test package (license header only).
tests/integration/tasks/execution/executors/celery/test_models.py	Updates factories import path after restructuring.
tests/integration/tasks/execution/executors/celery/init.py	Marks Celery executor test package (license header only).
tests/integration/tasks/execution/executors/init.py	Marks executors test package (license header only).
tests/integration/tasks/execution/init.py	Marks execution tests as a package (license header only).
tests/integration/tasks/db/test_engine.py	Adds tests for tasks DB engine/sessionmaker helper.
tests/integration/tasks/db/init.py	Marks tasks db tests as a package (license header only).
tests/integration/tasks/connectivity/test_service.py	Updates factories import path after restructuring.
tests/integration/tasks/connectivity/test_routes.py	Updates factories import path after restructuring.
tests/integration/tasks/connectivity/init.py	Marks connectivity tests as a package (license header only).
tests/integration/tasks/conftest.py	Updates factories import path after restructuring.
tests/integration/tasks/anonymizer/test_entities.py	Adds encode/decode round-trip test for anonymizer entity selection.
tests/integration/tasks/anonymizer/init.py	Marks anonymizer tests as a package (license header only).
tests/integration/tasks/init.py	Marks tasks integration tests as a package (license header only).
tests/integration/sep/utils/test_static.py	Adds tests for authenticated static-file serving behavior.
tests/integration/sep/utils/init.py	Marks sep utils tests as a package (license header only).
tests/integration/sep/test_scheduled_tasks_template.py	Adds template-render regression tests for scheduled tasks partial.
tests/integration/sep/test_main.py	Updates factories import path after restructuring.
tests/integration/sep/test_deps.py	Updates factories import path after restructuring.
tests/integration/sep/test_css_coverage.py	Adds guard test ensuring plugin CSS classes have sidebar icon rules.
tests/integration/sep/sync/test_models.py	Updates factories import path after restructuring.
tests/integration/sep/sync/test_exceptions.py	Adds tests for sync exception message/attribute behavior.
tests/integration/sep/sync/syncers/test_pmm.py	Updates factories import path after restructuring.
tests/integration/sep/sync/syncers/mysql/test_syncer.py	Updates factories import path after restructuring.
tests/integration/sep/sync/syncers/mysql/init.py	Marks mysql syncer tests as a package (license header only).
tests/integration/sep/sync/syncers/init.py	Marks syncers tests as a package (license header only).
tests/integration/sep/sync/conftest.py	Adds shared in-memory async DB session fixture for sync tests.
tests/integration/sep/sync/init.py	Marks sep sync tests as a package (license header only).
tests/integration/sep/snippets/test_forms.py	Adds tests for snippet HTML element models & description/info-icon rendering.
tests/integration/sep/snippets/test_crud.py	Adds CRUD tests for snippet manager `get_or_create` override behavior.
tests/integration/sep/snippets/models/init.py	Marks snippets models tests as a package (license header only).
tests/integration/sep/snippets/init.py	Marks snippets tests as a package (license header only).
tests/integration/sep/routes/test_stop_task.py	Updates factories import path after restructuring.
tests/integration/sep/routes/test_periodic_tasks.py	Updates factories import path after restructuring.
tests/integration/sep/routes/test_execution_events.py	Adds tests for SEP execution-events proxy route behavior.
tests/integration/sep/routes/test_download_files.py	Updates factories import path after restructuring.
tests/integration/sep/routes/conftest.py	Updates factories import path after restructuring.
tests/integration/sep/routes/init.py	Marks sep routes tests as a package (license header only).
tests/integration/sep/plugins/tasks/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/tasks/test_api_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/tasks/init.py	Marks tasks plugin tests as a package (license header only).
tests/integration/sep/plugins/snippets/init.py	Marks snippets plugin tests as a package (license header only).
tests/integration/sep/plugins/report/init.py	Marks report plugin tests as a package (license header only).
tests/integration/sep/plugins/mysql_backups/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/mysql_backups/restore/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/mysql_backups/restore/test_models.py	Adds validator/security-focused tests for restore `backup_source` input.
tests/integration/sep/plugins/mysql_backups/restore/init.py	Marks mysql_backups restore tests as a package (license header only).
tests/integration/sep/plugins/mysql_backups/init.py	Marks mysql_backups plugin tests as a package (license header only).
tests/integration/sep/plugins/inventory/test_sync.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/inventory/test_schema.py	Adds tests for the inventory plugin schema object serialization/content.
tests/integration/sep/plugins/inventory/test_routes.py	Updates factories + conftest import paths after restructuring.
tests/integration/sep/plugins/inventory/test_api_routes.py	Updates in-doc references and conftest import paths after restructuring.
tests/integration/sep/plugins/inventory/init.py	Marks inventory plugin tests as a package (license header only).
tests/integration/sep/plugins/framework/test_deprecation.py	Adds tests for deprecated Jinja2 route class header/log behavior.
tests/integration/sep/plugins/framework/init.py	Marks framework plugin tests as a package (license header only).
tests/integration/sep/plugins/dipper/test_routes.py	Adds tests asserting dipper routes use deprecation route class contract.
tests/integration/sep/plugins/dipper/test_pcs_collect_pmm_mysql.py	Adds script-level tests for `pcs-collect-pmm-mysql.py` help/frontmatter invariants.
tests/integration/sep/plugins/dipper/test_deps.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/dipper/conftest.py	Updates shared conftest import path after restructuring.
tests/integration/sep/plugins/dipper/init.py	Marks dipper plugin tests as a package (license header only).
tests/integration/sep/plugins/conftest.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/checksums/test_schema.py	Adds tests for checksums plugin schema capability exposure.
tests/integration/sep/plugins/checksums/test_routes.py	Adds tests for checksums route behavior and dep-chain coverage.
tests/integration/sep/plugins/checksums/conftest.py	Updates shared conftest import path after restructuring.
tests/integration/sep/plugins/checksums/init.py	Marks checksums plugin tests as a package (license header only).
tests/integration/sep/plugins/backup_pg/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/backup_pg/conftest.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/backup_pg/init.py	Marks backup_pg plugin tests as a package (license header only).
tests/integration/sep/plugins/backup_mongo/test_routes.py	Adds tests for backup_mongo route dep-chain and task fan-out behavior.
tests/integration/sep/plugins/backup_mongo/test_deps.py	Adds tests for backup_mongo dep payload construction & 404 swallowing behavior.
tests/integration/sep/plugins/backup_mongo/restore/test_routes.py	Adds tests for backup_mongo restore create/update dep-chain behavior.
tests/integration/sep/plugins/backup_mongo/restore/conftest.py	Adds fixtures for restore tests (RestoreCreate inputs).
tests/integration/sep/plugins/backup_mongo/restore/init.py	Marks backup_mongo restore tests as a package (license header only).
tests/integration/sep/plugins/backup_mongo/conftest.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/backup_mongo/init.py	Marks backup_mongo plugin tests as a package (license header only).
tests/integration/sep/plugins/atw/init.py	Marks atw plugin tests as a package (license header only).
tests/integration/sep/plugins/archives/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/archives/test_deps.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/archives/init.py	Marks archives plugin tests as a package (license header only).
tests/integration/sep/plugins/alters/test_routes.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/alters/test_deps.py	Updates factories import path after restructuring.
tests/integration/sep/plugins/alters/init.py	Marks alters plugin tests as a package (license header only).
tests/integration/sep/plugins/alerts/test_crud.py	Adds CRUD tests for alerts backup persistence ordering/shape.
tests/integration/sep/plugins/alerts/test_config.py	Adds tests for alerts plugin PMM config factory/back-compat behavior.
tests/integration/sep/plugins/alerts/conftest.py	Adds in-memory session fixture for alerts plugin tests.
tests/integration/sep/plugins/alerts/init.py	Marks alerts plugin tests as a package (license header only).
tests/integration/sep/plugins/alert_troubleshooting/test_schema.py	Adds tests for alert_troubleshooting plugin schema + endpoint.
tests/integration/sep/plugins/alert_troubleshooting/conftest.py	Adds fixtures for alert_troubleshooting plugin tests (DB + snippets).
tests/integration/sep/plugins/alert_troubleshooting/init.py	Marks alert_troubleshooting tests as a package (license header only).
tests/integration/sep/plugins/init.py	Marks sep plugin tests as a package (license header only).
tests/integration/sep/migrations/init.py	Marks sep migrations tests package (license header only).
tests/integration/sep/middleware/init.py	Marks sep middleware tests package (license header only).
tests/integration/sep/db/init.py	Marks sep db tests package (license header only).
tests/integration/sep/conftest.py	Adds shared SEP integration fixtures (sessions, clients, RemoteAPI mocks).
tests/integration/sep/clients/init.py	Marks sep clients tests package (license header only).
tests/integration/sep/api/test_host_resolution.py	Adds tests for executor host resolution helpers.
tests/integration/sep/api/routes/test_task_stats.py	Adds tests for task-stats proxy route incl. 502 behavior + auth enforcement.
tests/integration/sep/api/routes/init.py	Marks sep api routes tests package (license header only).
tests/integration/sep/api/init.py	Marks sep api tests package (license header only).
tests/integration/sep/init.py	Marks sep integration tests as a package (license header only).
tests/integration/inventory/test_models.py	Adds tests for inventory model validators/enums.
tests/integration/inventory/test_main.py	Adds tests ensuring inventory lifespan is set at module import time.
tests/integration/inventory/test_config.py	Adds tests for inventory settings defaults.
tests/integration/inventory/routes/test_summary.py	Adds tests for inventory summary route response shape/counts.
tests/integration/inventory/routes/test_services.py	Updates factories import path after restructuring.
tests/integration/inventory/routes/test_schemas.py	Updates factories import path after restructuring.
tests/integration/inventory/routes/test_nodes.py	Updates factories import path after restructuring.
tests/integration/inventory/routes/init.py	Marks inventory routes tests package (license header only).
tests/integration/inventory/conftest.py	Updates factories import path after restructuring.
tests/integration/inventory/init.py	Marks inventory integration tests as a package (license header only).
tests/integration/core/utils/test_strings.py	Adds (migrated) tests for core string utils including slugify and b64 helpers.
tests/integration/core/utils/test_serialization.py	Adds (migrated) tests for JSON serialization helper behavior.
tests/integration/core/utils/test_imports.py	Adds (migrated) tests for dynamic import/validation helpers.
tests/integration/core/utils/test_fields.py	Adds (migrated) tests for core field validators/path resolution.
tests/integration/core/utils/test_dict.py	Adds (migrated) tests for dict utilities (sorting/filtering).
tests/integration/core/utils/test_date_time.py	Adds (migrated) tests for UTC datetime conversion helper.
tests/integration/core/utils/test_async_run.py	Adds (migrated) tests for `async_run` executor behavior.
tests/integration/core/utils/init.py	Marks core utils tests package (license header only).
tests/integration/core/test_security.py	Adds (migrated) tests for crypto serializers round-trip behavior.
tests/integration/core/test_models.py	Adds (migrated) tests for base model behaviors and pagination model validation.
tests/integration/core/test_log.py	Adds tests for contextual logging filter/context-var helpers.
tests/integration/core/test_exceptions.py	Adds tests for project HTTP exception helpers and details/headers behavior.
tests/integration/core/requests/test_correlation_id.py	Adds tests ensuring correlation ID is propagated in RemoteAPI requests.
tests/integration/core/requests/init.py	Marks core requests tests package (license header only).
tests/integration/core/middleware/init.py	Marks core middleware tests package (license header only).
tests/integration/core/db/test_sql_types.py	Adds tests for SQLAlchemy type decorators (AutoJSON, MaybeCompressedText).
tests/integration/core/db/test_config.py	Adds tests for DB options URL building and repr masking.
tests/integration/core/db/init.py	Marks core db tests package (license header only).
tests/integration/core/auth/providers/test_casdoor.py	Adds tests for CasdoorSDK repr masking and api-key encoding.
tests/integration/core/auth/providers/init.py	Marks auth providers tests package (license header only).
tests/integration/core/auth/init.py	Marks auth tests package (license header only).
tests/integration/core/alerts/test_config.py	Adds tests for alert provider configuration parsing/validation.
tests/integration/core/alerts/providers/test_pagerduty.py	Adds tests for PagerDuty provider payload and validation behaviors.
tests/integration/core/alerts/providers/init.py	Marks alerts providers tests package (license header only).
tests/integration/core/alerts/init.py	Marks core alerts tests package (license header only).
tests/integration/core/init.py	Marks core integration tests as a package (license header only).
tests/integration/conftest.py	Auto-applies the `integration` marker to tests under `tests/integration/`.
tests/integration/api/routes/test_users.py	Updates factories import path after restructuring.
tests/integration/api/routes/test_config.py	Adds tests for `/api/config/alerts` availability flag and auth behavior.
tests/integration/api/routes/init.py	Marks api routes tests package (license header only).
tests/integration/api/init.py	Marks api tests package (license header only).
tests/integration/init.py	Marks integration tests as a package (license header only).
tests/factories.py	Adds consolidated Polyfactory factories for test data generation.
tests/contract/test_example_openapi_contract.py	Adds reference Schemathesis-based contract test (skips if not installed).
tests/contract/conftest.py	Auto-applies the `contract` marker to tests under `tests/contract/`.
tests/contract/init.py	Marks contract tests as a package (license header only).
tests/conftest.py	Updates factories import path after restructuring.
tests/_stubs/pmm.py	Adds canonical PMM stub helpers for future consolidation.
tests/_stubs/nomad.py	Adds canonical Nomad stub helpers for future consolidation.
tests/_stubs/casdoor.py	Adds canonical Casdoor stub helpers for future consolidation.
tests/_stubs/init.py	Marks stubs directory as a package (license header only).
pyproject.toml	Registers new pytest markers (`unit`, `integration`, `contract`).
Makefile	Adds lane-specific test targets and updates `.PHONY` list.
.github/workflows/python.yaml	Splits CI test job into unit/integration lanes; adds contract lane job.
.github/workflows/ci.yml	Wires in reusable audit workflow and requires it for overall CI success.
.github/workflows/audit.yaml	Adds reusable workflow for `pip-audit` and `pnpm audit`.
.github/instructions/tests.instructions.md	Updates test layout/factory guidance to new directory structure.
.github/instructions/architecture.instructions.md	Updates test-path examples to new integration directory layout.

cortiz-percona · 2026-05-27T20:35:50Z

@copilot resolve the merge conflicts in this pull request

… CVE dependency fixes

Copilot · 2026-05-27T20:37:58Z

@copilot resolve the merge conflicts in this pull request

Resolved. Merged origin/main into this branch (commit 3941692) — brings in the SEP-1113 schema changes, the brace-expansion CVE-2026-45149 fix, and the ws CVE-2026-45736 fix. No conflicts were encountered during the merge.

cortiz-percona · 2026-05-28T15:33:55Z

@copilot resolve the merge conflicts in this pull request

… PostgreSQL config files

Copilot · 2026-05-28T15:39:07Z

@copilot resolve the merge conflicts in this pull request

Resolved (commit d254d3b). Merged origin/main bringing in SEP-1116 (PII anonymization) and SEP-1026 (PostgreSQL config files). The one real conflict — tests/app/sep/plugins/mysql_backups/test_api_routes.py modified in main but deleted in this branch — was resolved by keeping it deleted, since our branch migrated those tests to tests/integration/ and the pii_anonymization capability is already covered by tests/integration/sep/plugins/framework/test_schema.py.

…conditionally

yyyyyyyan

I don't think this is the right approach. It's an enormous diff for something that could be done with simple pytest markers.

The rename of tests/app/ → tests/integration/ produced an enormous diff with no functional benefit. The auto-marker approach (pytest markers applied via conftest.py hooks) works regardless of the directory name, so the rename was pure churn. This commit: - Restores all test files to tests/app/ (original location) - Removes tests/integration/ directory - Adds pytest_collection_modifyitems to tests/app/conftest.py to auto-apply @pytest.mark.integration to all tests there - Keeps tests/unit/, tests/contract/, tests/_stubs/ unchanged

cortiz-percona · 2026-06-02T23:09:27Z

I don't think this is the right approach. It's an enormous diff for something that could be done with simple pytest markers.

Fixed. Reverted the tests/app/ --> tests/integration/ rename and applied @pytest.mark.integration via pytest_collection_modifyitems in tests/app/conftest.py instead

Copilot

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 8 comments.

example, add QA docs - _stubs/pmm,nomad: point at real surfaces (PMMRemoteAPI.get_nodes/ get_services, NomadExecutor.dispatch_job); dispatch_job is sync so use MagicMock - contract conftest: filter by fspath so full-suite runs don't over-mark - instructions: revert tests/integration -> tests/app after the rename revert - restore tests/app/test_example_api_route.py (lost in the rename revert) - docs: add qa-architecture.md + testing-guidelines.md (AC3); signal the existing Vitest layer as canonical (AC2)

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 12 comments.

Replace the POSIX substring path guard ("tests/X/" in str(item.fspath)) in the unit/integration/contract auto-markers with a pathlib check (Path(__file__).parent in item.path.parents).

cortiz-percona · 2026-06-19T21:57:02Z

Local Testing

Ran the lanes and exercised the fixed stubs against real targets. Results:

Check	Command	Result
Unit lane	`pytest -m unit tests/unit/`	✅ 10 passed
Contract lane	`pytest -m contract tests/contract/`	✅ skips cleanly (exit 5 tolerated — `schemathesis` not installed, deferred to M6)
Integration reference	`pytest -m integration tests/app/test_example_api_route.py`	✅ 3 passed (`TestClient` against the three mounted apps' `openapi.json`)
Lint	`ruff check` + `ruff format --check` on changed files	✅ all checks pass, 11 files already formatted

Stub fixes — verified patching real surfaces

The canonical stubs previously pointed at non-existent paths (app.core.pmm.client.PMMClient, app.tasks…nomad.client.NomadClient) and would have raised at patch time. A smoke test in the integration context (full app loaded) confirms they now patch real targets without raising:

patch_pmm_metrics → PMMRemoteAPI.get_nodes / get_services ✅
patch_nomad_dispatch → NomadExecutor.dispatch_job (sync → MagicMock) ✅
patch_casdoor_sdk → CasdoorSDK.introspect_token / get_user ✅

Note: the stubs are integration-layer (they mock external services for integration tests), so they resolve their targets with the app loaded — which is their intended usage.

Marker scoping

Confirmed the contract auto-marker no longer leaks: pytest -m contract across the whole tests/ tree collects 0 (4269 deselected), so it doesn't over-mark the integration lane.

Not run locally

make test-integration full suite (~4k tests) and pnpm --filter @sep/e2e test:e2e (Playwright/Chromium) run in CI — python.yaml and frontend.yaml cover them on the branch.

cortiz-percona · 2026-07-03T21:55:46Z

Hi team, take a look when you have a moment. Want to make sure it doesn't stall. Happy to help it forward once there's feedback.

Setup test architecture

656b0b2

cortiz-percona self-assigned this May 27, 2026

Remove outdated documentation

2ee688c

cortiz-percona marked this pull request as ready for review May 27, 2026 20:16

Copilot AI review requested due to automatic review settings May 27, 2026 20:16

cortiz-percona requested review from a team, marcuscruz-percona, maxbube, nachodd, peter-o-addo and yyyyyyyan as code owners May 27, 2026 20:16

Copilot started reviewing on behalf of cortiz-percona May 27, 2026 20:16 View session

cortiz-percona marked this pull request as draft May 27, 2026 20:16

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread tests/unit/conftest.py

Comment thread tests/integration/conftest.py Outdated

Comment thread tests/contract/conftest.py

Resolve conflicts

37eb485

Copilot started work on behalf of cortiz-percona May 27, 2026 20:36 View session

Merge origin/main into SEP-1194: bring in SEP-1113 schema changes and…

3941692

… CVE dependency fixes

Copilot finished work on behalf of cortiz-percona May 27, 2026 20:38

github-actions Bot added python frontend labels May 28, 2026

Copilot started work on behalf of cortiz-percona May 28, 2026 15:34 View session

Merge origin/main into SEP-1194: SEP-1116 PII anonymization, SEP-1026…

d254d3b

… PostgreSQL config files

Copilot finished work on behalf of cortiz-percona May 28, 2026 15:39

Copilot started work on behalf of cortiz-percona May 28, 2026 15:56 View session

Fix path-separator issue in contract conftest by marking all items un…

8b2b036

…conditionally

Copilot finished work on behalf of cortiz-percona May 28, 2026 15:58

cortiz-percona added qa passed Tests for this PR are completed and successful. and removed qa in progress Someone is currently testing this PR - do not merge it labels May 29, 2026

yyyyyyyan added do not merge There are reasons to not merge the code under discussion The changes are still being discussed labels May 29, 2026

yyyyyyyan reviewed May 30, 2026

View reviewed changes

cortiz-percona added 2 commits June 2, 2026 17:52

Merge branch 'main' into SEP-1194

35f94b5

cortiz-percona requested a review from yyyyyyyan June 4, 2026 21:27

Merge branch 'main' into SEP-1194

214bd9b

cortiz-percona requested a review from Copilot June 8, 2026 20:38

Copilot started reviewing on behalf of cortiz-percona June 8, 2026 20:38 View session

Copilot AI reviewed Jun 8, 2026

View reviewed changes

Refactor Makefile tests unit and contract

fdca890

cortiz-percona added qa passed Tests for this PR are completed and successful. do not merge There are reasons to not merge the code and removed do not merge There are reasons to not merge the code qa passed Tests for this PR are completed and successful. labels Jun 8, 2026

cortiz-percona requested a review from a team as a code owner June 19, 2026 20:28

Merge branch 'main' into SEP-1194

c8749cd

cortiz-percona requested a review from Copilot June 19, 2026 20:48

Copilot started reviewing on behalf of cortiz-percona June 19, 2026 20:48 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

SEP-1194: make conftest auto-markers cross-platform

e9e0661

Replace the POSIX substring path guard ("tests/X/" in str(item.fspath)) in the unit/integration/contract auto-markers with a pathlib check (Path(__file__).parent in item.path.parents).

Copilot started work on behalf of cortiz-percona June 19, 2026 21:43 View session

fix: update test docstrings to imperative mood

283d104

Copilot finished work on behalf of cortiz-percona June 19, 2026 21:46

Uh oh!

Conversation

cortiz-percona commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tested

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cortiz-percona commented May 27, 2026

Uh oh!

Copilot AI commented May 27, 2026

Uh oh!

cortiz-percona commented May 28, 2026

Uh oh!

Copilot AI commented May 28, 2026

Uh oh!

yyyyyyyan left a comment

Choose a reason for hiding this comment

Uh oh!

cortiz-percona commented Jun 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cortiz-percona commented Jun 19, 2026

Local Testing

Stub fixes — verified patching real surfaces

Marker scoping

Not run locally

Uh oh!

cortiz-percona commented Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cortiz-percona commented May 27, 2026 •

edited

Loading