Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .agents/skills/third-party-package-patches/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
name: third-party-package-patches
description: Create or update GraalPy third-party package compatibility patches under graalpython/lib-graalpython/patches, including PyPI source preparation, rebasing existing patches, metadata.toml updates, license checks, version-range validation, and verify_patches.py validation.
---

# Third-Party Package Patches

Use this skill when creating or updating compatibility patches for packages installed by pip on GraalPy.

## Key Files
- Source preparation: `scripts/get_pypi_source.py`
- Patch metadata: `graalpython/lib-graalpython/patches/metadata.toml`
- Patch directory: `graalpython/lib-graalpython/patches/`
- Metadata verifier: `mx.graalpython/verify_patches.py`

## Workflow
1. Identify the package name and target version. Use the normalized package key used by PyPI/pip: lowercase with runs of `-`, `_`, and `.` normalized to `-`.

2. Prepare source with the repo helper:
```bash
python scripts/get_pypi_source.py package==version
```
The script prints `Prepared source at: ...`. Use that directory as the working tree. It is already a temporary git repository with an initial commit, and it has already been processed by `graalpython/lib-graalpython/modules/autopatch_capi.py`.

3. Inspect `graalpython/lib-graalpython/patches/metadata.toml` for existing `[[package.rules]]` entries.
- If a matching patch exists for the requested version, apply it first.
- If only a nearby version has a patch, try applying that patch and rebase it carefully onto the prepared source.
- Honor `subdir` when present: pip applies sdist patches from that subdirectory. Apply and generate the patch from the same directory layout that the metadata rule will use.
- Honor `dist-type` when choosing whether the patch is for `sdist`, `wheel`, or both.

4. Apply existing patches using the same semantics as pip where practical:
```bash
patch -f -d /tmp/package-version-... -p1 -i /path/to/graalpython/lib-graalpython/patches/existing.patch
```
If the rule has `subdir = 'src'`, use `-d /tmp/package-version-.../src`. Resolve rejects by editing source files in the temporary repository, then remove any `.rej`/`.orig` files after checking them. Search for conflict markers and rejects before staging.

5. Make the GraalPy compatibility changes in the prepared source. Keep the patch minimal and package-focused.

6. Stage the desired changes in the temporary source repository:
```bash
git add -A
git diff --cached
```
Review the staged diff. Do not include generated caches, build outputs, rejected patch files, or unrelated churn.

7. Create or refresh the patch file only from the staged git diff:
```bash
git diff --cached > /path/to/graalpython/lib-graalpython/patches/package-version.patch
```
Guardrail: do not hand-edit patch files. If the patch output is wrong, fix the temporary source tree, adjust staging, and regenerate with `git diff --cached`.

8. Update `metadata.toml` if needed.
- Add a new `[[package.rules]]` entry when no suitable one exists.
- Keep existing precedent for rule ordering, patch names, comments, `install-priority`, `dist-type`, and `subdir`.
- Every rule with `patch = ...` needs `license = ...`.
- When adding a new patch entry, confirm the license from PyPI metadata, preferably the JSON API for the exact release or current package metadata. Use an SPDX identifier accepted by `mx.graalpython/verify_patches.py`.
- If upstream publishes no suitable PyPI source artifact, add `[[package.add-sources]]` with the exact version and release tarball URL, then rerun `scripts/get_pypi_source.py`.

9. Choose the version range deliberately.
- Prefer one patch over many when the same patch applies cleanly and the underlying package layout/API is stable.
- Test every version covered by a widened range, including lower and upper boundary releases.
- Small, robust patches may use an open-ended range when they are unlikely to break in future versions.
- If newer versions no longer need a patch, add a no-patch rule with a note rather than leaving users pointed at stale patched versions.

10. Validate patch application for the covered versions.
- For each version in the rule range that you claim to support, prepare a fresh source tree with `scripts/get_pypi_source.py`.
- Apply the generated patch with `patch -f -p1` from the root or `subdir` exactly as the metadata rule requires.
- Check for nonzero exit status, `.rej` files, `.orig` files, unexpected unstaged files, and conflict markers.

11. Run the repository verifier before finishing:
```bash
python mx.graalpython/verify_patches.py graalpython/lib-graalpython/patches
```

12. If you were asked to build or test the patched package, you need to rebuild GraalPy with `mx python-jvm` to pick up the changes. Create a venv with `mx python -m venv venv_name` and use it for building and testing.

## Metadata Reference
Rule keys accepted by the verifier are:
- `version`
- `patch`
- `license`
- `subdir`
- `dist-type`
- `install-priority`
- `note`

Allowed `dist-type` values are `wheel` and `sdist`.

Allowed license identifiers are maintained in `mx.graalpython/verify_patches.py`. If PyPI metadata is ambiguous, inspect the source distribution license files and report the ambiguity instead of guessing.

## Reporting
When done, report:
- package and version(s) tested
- patch file created or updated
- metadata rule added or changed
- PyPI license value used
- `verify_patches.py` result
196 changes: 196 additions & 0 deletions graalpython/lib-graalpython/patches/librt-0.10.0.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
diff --git a/CPy.h b/CPy.h
index 89ef4d0..a763914 100644
--- a/CPy.h
+++ b/CPy.h
@@ -333,6 +333,7 @@ static inline bool CPyTagged_IsLe(CPyTagged left, CPyTagged right) {
}

static inline int64_t CPyLong_AsInt64(PyObject *o) {
+#ifndef GRAALPY_VERSION_NUM
if (likely(PyLong_Check(o))) {
PyLongObject *lobj = (PyLongObject *)o;
#if CPY_3_12_FEATURES
@@ -349,11 +350,13 @@ static inline int64_t CPyLong_AsInt64(PyObject *o) {
}
#endif
}
+#endif
// Slow path
return CPyLong_AsInt64_(o);
}

static inline int32_t CPyLong_AsInt32(PyObject *o) {
+#ifndef GRAALPY_VERSION_NUM
if (likely(PyLong_Check(o))) {
#if CPY_3_12_FEATURES
PyLongObject *lobj = (PyLongObject *)o;
@@ -375,11 +378,13 @@ static inline int32_t CPyLong_AsInt32(PyObject *o) {
}
#endif
}
+#endif
// Slow path
return CPyLong_AsInt32_(o);
}

static inline int16_t CPyLong_AsInt16(PyObject *o) {
+#ifndef GRAALPY_VERSION_NUM
if (likely(PyLong_Check(o))) {
#if CPY_3_12_FEATURES
PyLongObject *lobj = (PyLongObject *)o;
@@ -405,11 +410,13 @@ static inline int16_t CPyLong_AsInt16(PyObject *o) {
}
#endif
}
+#endif
// Slow path
return CPyLong_AsInt16_(o);
}

static inline uint8_t CPyLong_AsUInt8(PyObject *o) {
+#ifndef GRAALPY_VERSION_NUM
if (likely(PyLong_Check(o))) {
#if CPY_3_12_FEATURES
PyLongObject *lobj = (PyLongObject *)o;
@@ -435,6 +442,7 @@ static inline uint8_t CPyLong_AsUInt8(PyObject *o) {
}
#endif
}
+#endif
// Slow path
return CPyLong_AsUInt8_(o);
}
diff --git a/mypyc_util.h b/mypyc_util.h
index 6309ed5..2851067 100644
--- a/mypyc_util.h
+++ b/mypyc_util.h
@@ -82,21 +82,17 @@

static inline void CPy_INCREF_NO_IMM(PyObject *op)
{
- Py_REFCNT(op)++;
+ Py_INCREF(op);
}

static inline void CPy_DECREF_NO_IMM(PyObject *op)
{
- if ((Py_SET_REFCNT(op, Py_REFCNT(op) - 1), Py_REFCNT(op)) == 0) {
- _Py_Dealloc(op);
- }
+ Py_DECREF(op);
}

static inline void CPy_XDECREF_NO_IMM(PyObject *op)
{
- if (op != NULL && (Py_SET_REFCNT(op, Py_REFCNT(op) - 1), Py_REFCNT(op)) == 0) {
- _Py_Dealloc(op);
- }
+ Py_XDECREF(op);
}

#define CPy_INCREF_NO_IMM(op) CPy_INCREF_NO_IMM((PyObject *)(op))
diff --git a/pythonsupport.c b/pythonsupport.c
index 0a99f0a..1a442bb 100644
--- a/pythonsupport.c
+++ b/pythonsupport.c
@@ -108,7 +108,46 @@ init_subclass(PyTypeObject *type, PyObject *kwds)
return 0;
}

-#if CPY_3_12_FEATURES
+#ifdef GRAALPY_VERSION_NUM
+
+Py_ssize_t
+CPyLong_AsSsize_tAndOverflow_(PyObject *vv, int *overflow)
+{
+ Py_ssize_t res;
+
+ *overflow = 0;
+
+ res = PyLong_AsSsize_t(vv);
+ if (res == -1 && PyErr_Occurred()) {
+ PyObject *zero;
+ int is_negative;
+
+ if (!PyErr_ExceptionMatches(PyExc_OverflowError)) {
+ return -1;
+ }
+ PyErr_Clear();
+
+ zero = PyLong_FromLong(0);
+ if (zero == NULL) {
+ return -1;
+ }
+ is_negative = PyObject_RichCompareBool(vv, zero, Py_LT);
+ Py_DECREF(zero);
+ if (is_negative < 0) {
+ return -1;
+ }
+ *overflow = is_negative ? -1 : 1;
+ return -1;
+ }
+
+ if ((size_t)res > CPY_TAGGED_MAX && (res >= 0 || res < CPY_TAGGED_MIN)) {
+ *overflow = res < 0 ? -1 : 1;
+ return -1;
+ }
+ return res;
+}
+
+#elif CPY_3_12_FEATURES

// Slow path of CPyLong_AsSsize_tAndOverflow (non-inlined)
Py_ssize_t
diff --git a/pythonsupport.h b/pythonsupport.h
index 6f38a9b..0e89511 100644
--- a/pythonsupport.h
+++ b/pythonsupport.h
@@ -40,7 +40,15 @@ int init_subclass(PyTypeObject *type, PyObject *kwds);
Py_ssize_t
CPyLong_AsSsize_tAndOverflow_(PyObject *vv, int *overflow);

-#if CPY_3_12_FEATURES
+#ifdef GRAALPY_VERSION_NUM
+
+static inline Py_ssize_t
+CPyLong_AsSsize_tAndOverflow(PyObject *vv, int *overflow)
+{
+ return CPyLong_AsSsize_tAndOverflow_(vv, overflow);
+}
+
+#elif CPY_3_12_FEATURES

static inline Py_ssize_t
CPyLong_AsSsize_tAndOverflow(PyObject *vv, int *overflow)
@@ -117,6 +125,7 @@ CPyLong_AsSsize_tAndOverflow(PyObject *vv, int *overflow)
#endif

// Adapted from listobject.c in Python 3.7.0
+#if 0 // GraalPy change
static int
list_resize(PyListObject *self, Py_ssize_t newsize)
{
@@ -162,6 +171,7 @@ list_resize(PyListObject *self, Py_ssize_t newsize)
self->allocated = new_allocated;
return 0;
}
+#endif

// Changed to use PyList_SetSlice instead of the internal list_ass_slice
static PyObject *
@@ -182,6 +192,7 @@ list_pop_impl(PyListObject *self, Py_ssize_t index)
return NULL;
}
v = PySequence_Fast_ITEMS((PyObject*)self)[index];
+#if 0 // GraalPy change
if (index == Py_SIZE(self) - 1) {
status = list_resize(self, Py_SIZE(self) - 1);
if (status >= 0)
@@ -189,6 +200,7 @@ list_pop_impl(PyListObject *self, Py_ssize_t index)
else
return NULL;
}
+#endif
Py_INCREF(v);
status = PyList_SetSlice((PyObject *)self, index, index+1, (PyObject *)NULL);
if (status < 0) {
Loading
Loading