poetry-compatibility.md

Poetry compatibility and production backtests

socket-patch supports hosted and vendored Python patches in every poetry.lock generation Poetry has written, and agent mode (in-place patching of the project's virtualenv) on every Poetry release. The tests use real Poetry releases bootstrapped with uv, real PyPI artifacts, and the public Socket patch service. Successful rewriting alone is not an installation result: the backtest reinstalls from the rewritten lock and compares the installed bytes with the published patch.

This supplements the hosted and vendored production suites and mirrors the uv matrix. See the ecosystem matrix for other package managers.

Formats and rewrite behavior

Lock generation (writer)	Hosted (scan --mode hosted)	Vendored (scan --mode vendored)
[metadata.hashes], no lock-version (Poetry 0.12)	Refused (redirect_poetry_lock_unsupported): the installer ignores [package.source] type = "url" and installs the registry artifact, so a rewrite would attest a patch that never lands.	[package.source] type = "file" + reference = "" (read unconditionally by 0.12) and the wheel's SHA-256 in [metadata.hashes].
lock-version = "1.0" (Poetry 1.0)	type = "url" + reference = ""; the URL carries #sha256=<hex>& because Poetry 1.0 appends #egg=<name> unconditionally (see the pip caveat below). pip verifies the fragment; the patched hash is also written to [metadata.files] (consistency) and to the package's own files (what Poetry ≥ 1.2 verifies when it consumes a 1.0 lock; 1.0 ignores the extra key).	type = "file" + reference = ""; [metadata.files] entry replaced.
lock-version = "1.1" (Poetry 1.1, 1.2)	type = "url"; the patched hash is written to BOTH [metadata.files] (what Poetry 1.1 verifies) and the package's own files (what Poetry 1.2 verifies — it drops URL hashes from [metadata.files]).	type = "file"; [metadata.files] entry replaced.
lock-version = "2.0" / "2.1" / any "2.<n>" (Poetry 1.3+)	type = "url"; files = [{file, hash}] replaced with the single patched wheel.	type = "file"; files replaced. LF 2.x locks keep Poetry's own multi-line files formatting; CRLF locks and legacy formats go through the shared toml_edit rewriter, which writes a single-line inline array. Both are valid TOML and byte-stable under poetry check --lock.

Both modes retain the package version, dependencies, groups, markers, extras and the pyproject content-hash; no pyproject edit is required. A repeated scan leaves the lock unchanged. Vendored rollback restores the recorded original fragments. Hosted rollback resolves upstream registry entries; it has no replay ledger or guarantee of the original byte layout. Both preserve unrelated entries. Refused before any write: a [[package]] listed at several versions (marker fork), a user-authored [package.source] on another origin (an earlier Socket URL for the same wheel is superseded in place, e.g. after a grant-token rotation), an unsupported lock-version, a malformed [metadata.files] / [metadata.hashes] value, and a wheel whose filename does not match the locked package.

A dependency the pyproject pins to a named source (six = {version = "1.16.0", source = "pypi"}, in the main, legacy dev or any group table) leaves no trace in the lock. Poetry 1.3 – 2.2 refuse to install a lock that then sources it from a URL or file (Repository "pypi" does not exist.; 1.3 and 1.4 report a solver failure); 2.3+ install it (measured, #1282, #1447), but only from a lock in format 2.1 or later. Every Poetry from 1.3 on re-solves an older lock (lock-version 1.0 / 1.1, written by Poetry 1.0 – 1.2) on install instead of trusting it, and that solve refuses the constraint the same way, so only Poetry 1.0 – 1.2 install such a lock (measured on 1.3.2, 2.0.1, 2.2.1, 2.3.4, 2.4.3 and 2.5.1, #1503). So both modes refuse that package before any write (redirect_poetry_lock_unsupported hosted, pypi_poetry_source_constraint vendored) when the lock names a 1.3 – 2.2 writer (its @generated by Poetry X.Y stamp, or, with no versioned stamp, lock-version = "2.0", which only 1.3 – 1.8 write; 1.3 stamps no version) or when its lock-version is 1.0 or 1.1; the detail says which. Over a 2.3+ lock (or vendored, a 0.x one) they rewire it and warn (redirect_poetry_source_constraint / pypi_poetry_source_constraint) that a 1.3 – 2.2 installer would refuse the lock. The header names the writer, which is usually but not always the installer.

Installer boundaries (measured)

Poetry	Hosted	Vendored	Verifies the lock hash on install	Replaces an already-installed same-version package
0.12	refused (URL sources ignored)	supported	no	no
1.0	supported (#sha256=…& fragment; pip ≤ 22.2 or ≥ 23.1)	supported	hosted: yes (pip fragment); vendored: no	no
1.1 – 1.3	supported	supported	hosted: yes; vendored: no	no
1.4 – 1.8	supported	supported	yes / yes	yes
2.0 – 2.4	supported	supported	yes / yes	yes

Two consequences for Poetry releases before 1.4:

• A warm virtualenv keeps the upstream package after the lock is rewritten: poetry install compares installed packages by name and version only and prints "No dependencies to install or update". Recreate the virtualenv (or pip uninstall the package) before installing, or upgrade Poetry. Fresh installs pick up the patched wheel on every release. The CLI flags this as redirect_poetry_stale_install_risk (hosted) and pypi_poetry_integrity_unverified (vendored), keyed on the lock's writer: formats 0 / 1.0 / 1.1 are only written by pre-1.4 releases, and a lock 2.0 whose header lacks a @generated by Poetry X.Y.Z version was written by 1.3 (1.4+ stamp their version). A 2.0 lock stamped 1.4–1.8 is not flagged.
• Poetry 1.0 installs URL sources through pip, which reads the hash from the URL fragment. Poetry appends #egg=<name> to every URL, so the rewrite ends the fragment with & to keep sha256=<hex> intact. Measured across pip releases: pip 20.3 – 22.2 and 23.1+ install and verify the patched wheel with that spelling; pip 22.3 and 23.0 (the pip Python 3.8's ensurepip seeds) treat the trailing &#egg=… as part of the digest and refuse the install (fail-closed, nothing installed) — no fragment spelling satisfies them, so upgrade or downgrade pip inside the virtualenv. Without the &, every pip ≥ 22.3 refuses. The hosted advisory names this window for lock 1.0.
• Local (vendored) wheel hashes are not verified; the committed wheel bytes are the protection — review them. Hosted URL hashes are verified on every release from 1.0 on by the default installer (Poetry's deprecated pip backend, experimental.new-installer = false, verifies nothing).

Hosted mode also checks the installed package's bytes, independently of the lock's writer version. redirect_pypi_stale_install means readable files in a discovered interpreter still differ from the patched hashes. This read-only check repeats on re-scans and excludes the package from same-run VEX, even with --vex-no-verify or a healthy copy in another interpreter. If no patches remain to attest, --vex fails with no_applicable_patches. Reinstall in the affected interpreter, then verify with socket-patch vex. A lock-only checkout has no installed bytes to judge and keeps the existing lock-based attestation behavior.

Other measured details:

• poetry lock --no-update (1.1–1.8) and bare poetry lock (2.x) keep the patch source. Bare poetry lock on 0.12 / 1.0 (no --no-update), poetry lock --regenerate (2.x), poetry update (with or without --lock) and poetry update <patched-package> — even when the version does not change — drop the source and restore the registry hashes on every release. metadata.content-hash is unchanged by that, so poetry check --lock / poetry lock --check cannot detect the loss: re-run socket-patch scan --mode … after any of them, or gate CI on socket-patch vex.
• poetry lock --no-update on Poetry 1.1 / 1.2 keeps the hosted source but rewrites the entry in its own lock-1.1 shape, dropping the package-level files the rewrite added for Poetry ≥ 1.2's hash check; a lock relocked by 1.1 and then installed by 1.2+ installs the hosted wheel unverified. Re-run socket-patch scan --mode hosted after relocking on those releases: the re-scan restores the source and integrity fields. In v5, hosted rollback reconstructs upstream metadata and may refuse a drifted lock; it does not replay pre-scan fragments.
• Poetry 0.12 and 1.0 resolve a relative type = "file" path against the shell's working directory, not the project root; run poetry install from the project root on those releases.
• Poetry ≤ 1.1 stores its HTTP cache under the user cache directory regardless of POETRY_CACHE_DIR, behind one lockfile that wedges parallel runs and stays wedged after a SIGKILL; the harness gives each legacy case its own HOME.

Mode notes

• Agent mode patches the interpreter the crawler finds, following Poetry's EnvManager.get(): VIRTUAL_ENV (or a non-base conda CONDA_PREFIX), unless poetry env use recorded an env for the project in <virtualenvs.path>/envs.toml, which Poetry prefers over an activated venv (a record in any of the roots a {data-dir} path can expand to counts); then an existing ./.venv unless virtualenvs.in-project = false; then the virtualenv Poetry placed under its virtualenvs.path: the envs.toml env (<name>-<hash>-py<minor>) when there is one, else every <name>-<hash>-py<X.Y> env. That answer is final. Poetry never uses ./venv, so a stray one is never patched, and a fresh checkout with no env yet patches nothing (its packages come from poetry.lock) instead of falling back to the OS Python. Poetry's placement is reproduced without running Poetry, from POETRY_*, the project's poetry.toml, the user config.toml and the platform default cache dir. The env name follows poetry-core's precedence: [project] name (2.x), then [tool.poetry] name, then non-package-mode for a nameless package-mode = false project. On Windows the cwd hash drops the \\?\ prefix that path canonicalization adds, as Python's realpath does. So a bare socket-patch scan --mode agent / rollback in a default-configured Poetry checkout works; poetry run socket-patch …, VIRTUAL_ENV=$(poetry env info -p) and --global-prefix <site-packages> keep working. virtualenvs.create = false (containers) means Poetry installed into the system interpreter (unless an active venv or a usable ./.venv came first), which the project-marker global fallback covers. Patched bytes survive poetry install, poetry sync and poetry install --sync on every release (same version → no reinstall).
• Vendored mode works lock-only for locks that list a pure-Python (-none-any.whl) wheel for the package — lock 2.x files, lock 1.0/1.1 [metadata.files]: the inventory carries that sha256 and the fetcher resolves the file through PyPI's JSON API by digest, verifying the bytes again, exactly like uv's lock-only path. Poetry 0.12's bare [metadata.hashes] names no wheel, and a package that ships only platform wheels has no platform-independent choice: those still need the package installed in the discovered virtualenv (vendor_fetch_unverifiable + package_not_installed). Fresh clones of the committed .socket/vendor/ tree and rewired lock install from the committed wheel with no CLI at all (the matrix's fresh-clone leg).
• Hosted mode works lock-only (redirected: 1 with no virtualenv).

Running the matrix

cargo build -p socket-patch-cli
cp target/debug/socket-patch /tmp/socket-patch-under-test   # rebuilds must not swap it mid-run
python3 scripts/backtest-poetry.py \
  --cli /tmp/socket-patch-under-test \
  --cli-revision "$(git rev-parse --short HEAD)" \
  --output /tmp/socket-patch-poetry-backtest \
  --modes hosted vendored agent agent-oot \
  --shapes direct populated crlf pep621
python3 scripts/backtest-poetry.py --render-doc-table /tmp/socket-patch-poetry-backtest/summary.json

The harness bootstraps every release in VERSIONS with uv (Python 3.8.20 for 0.12–1.1, 3.12.13 for 1.2+; Poetry 1.2.2 needs cleo==1.0.0a5), generates a native lock for a one-dependency project (urllib3 = "1.26.18", whose public free-tier patch needs no token), and for each mode checks: exactly one patch applied, pyproject untouched, idempotent re-scan, poetry install into an emptied virtualenv installs bytes matching the patch record's afterHash, poetry check --lock, a fresh clone of the committed state installs the patch, socket-patch vex attests it, a corrupted hash is rejected where the installer verifies, Poetry's own relock keeps the source, and rollback restores every byte and clears the ledgers. Shapes: direct (the committed native fixture), populated (legacy locks with real upstream hashes filled in — today's PyPI JSON API leaves old Poetry's [metadata.files] empty), crlf, and pep621 (2.x [project] tables with package-mode = false). Mode agent-oot (Poetry's default out-of-tree virtualenv via poetry run) is informational.

A case is re-run from a fresh directory only when every failed required check is explained by a terminal transport failure in its own operation (Poetry or pip giving up on PyPI, a CLI request error, patch API 5xx, or exhausted 429 retry). CLI errors reported in JSON with exit code zero also qualify. Recovered retry warnings, informational commands, and expected tamper failures do not trigger retries. Independent file invariants and earlier functional failures remain failures even if a later operation raises a transport exception. There are at most three attempts in total. Failed attempt logs stay under attempts/<case>/<n>/, and the final row lists them in transportRetries. A failing case prints each failed check's recorded detail to the job log.

Rust coverage of the rewriters: cargo test -p socket-patch-core --lib utils::poetry_lock vendor::pypi_poetry and cargo test -p socket-patch-core --test poetry_hosted. The committed native locks under crates/socket-patch-core/tests/fixtures/poetry/<version>/ are the harness's original/ inputs (same pyproject, same content-hash).

Full run results belong with the source revision and toolchain versions in CI artifacts or a local output directory. See the testing guide.