Fix ci

[Erotemic]

Bumping CI versions with xcookie to see if that avoids recent CI errors. This will drop 3.8 support, which is overdue anyway.

I think we may need to re-add the arm entries:

        - python-version: '3.13'
          install-extras: tests
          os: ubuntu-24.04-arm

The wheel path detection logic changed, but I think that's ok. Would be nice if xcookie could keep the comments in the CI yaml. I was having issues getting ruamel.yaml doing that. I may manually update if I can solve the CI issues.

[TTsangSC]

Some observations:

• The common failure points on the jobs seem to consistently be tests/test_child_procs.py::test_running_multiproc_script, i.e. the tests created in FIX: kernprof to always run code in sys.modules['__main__']'s namespace #423 to guard against BUG: kernprof executed code inconsistent with sys.modules['__main__'], causing pickling issues #422. Which is strange – after all, the PR which introduced the tests also implemented the fix. The problem seems to be that somehow pip install isn't respecting the -f flag, pulling the wheel from PyPI instead of ./wheelhouse.

  • Given that the wheelhouse and a wheel of the matching version and architecture exist, IDK why this would happen... can't seem to reproduce this behavior on local with (vanilla) pipeither.
  • Unfortunately the equivalent uv command in the passing pipeline didn't produce enough output to indicate whence it pulled the line_profiler wheel, but I have to guess the wheelhouse and not PyPI judging from how the tests did pass.

• The 3.9 [...] with tests-strict,runtime-strict jobs additionally fail the ref-counting in these four tests in tests/test_line_profiler.py:

  • test_double_decoration
  • test_function_decorator
  • test_async_gen_decorator[...]

  The cause is how the tests-strict extra pins coverage to 6.5 (see requirements/tests.txt); in my testing, anything below 7.3 consistently breaks these tests between Python 3.9 and 3.11. This bug seems consistent with what is described in CTracer fails to deallocate bound methods references coveragepy/coveragepy#1283 (reported on Python 3.8, coverage 6.1), where a couple refs are leaked when coverage is used; this however begs the question of why this adverse interaction didn't manifest when we had equivalent setups on Python 3.8? (Same discrepancy on local – 6.5 works with 3.8 but not any newer Pythons.)

• Looking at the diffs, not only is the ARM-Linux stuff gone (regressing FIX: building Linux-ARM64 wheels #402), one of the test jobs on regular Ubuntu (3.13 on ubuntu-latest, arch=auto with tests) is also caught in the crossfire:

           - python-version: '3.13'
             install-extras: tests-strict,runtime-strict,optional-strict
             os: windows-11-arm
             arch: auto
  -        - python-version: '3.13'
  -          install-extras: tests
  -          os: ubuntu-latest
  -          arch: auto
  -        - python-version: '3.13'
  -          install-extras: tests
  -          os: ubuntu-24.04-arm
  -          arch: auto
           - python-version: '3.13'
             install-extras: tests
             os: macOS-latest
             arch: auto

• Is there any reason why we're removing cp39-win_arm64 from cibw_skip?

  -        - '*-win32 cp3{9,10}-win_arm64 cp313-musllinux_i686'
  +        - '*-win32 cp3{10}-win_arm64 cp313-musllinux_i686'

  As described in FIX: building on Windows-ARM64 #391, we skipped these Python versions because we can't test them on GitHub's windows-11-arm runners. As far as I can tell, this constraint is still true:

  test script

  import json
  import subprocess  # I don't have requests
  
  
  # Format:
  # [{"version": ...,
  #   "stable": ...,
  #   "release_url": ...,
  #   "files": [{"filename": ...,
  #               "arch": ...,
  #               "platform": ...,
  #               "platform_version": ...,  # Optional
  #               "download_url": ...},
  #             ...]},
  #  ...]
  manifest = 'https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json'
  proc = subprocess.run(['curl', manifest], text=True, capture_output=True)
  versions = json.loads(proc.stdout)
  
  
  def is_available(python_version: str) -> bool:
      return any((version['version'].startswith(python_version)
                  and any((file['arch'].startswith('arm64')
                           and file['platform'] == 'win32')
                          for file in version['files']))
                 for version in versions)
  
  
  assert is_available('3.11')
  assert not is_available('3.10')
  assert not is_available('3.9')

  Not to mention: even if the intent was to specifically un-skip building 3.9 wheel for ARM-Windows, this form still causes issues as braces without multiple options therein isn't correctly parsed by cibuildwheel.

[Erotemic]

@TTsangSC I've bumped the minimum versions of coverage / pytest-cov to prevent the failures, and I'm working on adding things back in so I can squash all these patches into a reasonable diff that will fix your PRs.

Thanks for the analysis, it was very helpful.

EDIT: Also noting, I want to add back in the removal of qemu given that we are running the CI's on arm naively.

[Erotemic]

I'm thinking we just EOL Python 3.9 while we are at it, and just keep up with the non-EOL python versions. I'm also going to switch over to github trusted publishing to reduce long-lived secrets.