GHA: clean up & extend testing; update to latest LLVM - #231
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2291b0dc5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
e2291b0 to
771cd38
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 771cd38a0c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Re-triaged from run 30147111711 (#231), now that the --daemonize and local-action-path fixes let the jobs actually execute: x86_64 functionalities, i686 commands + Python API, and aarch64 commands + Python API each ran to completion and their real FAIL/ERROR sets are recorded here. i686 and aarch64 functionalities both crashed lldb itself (Fatal Python error: Segmentation fault, in TestThreadJump and TestReturnValue.test_with_python respectively) before producing a usable failure list; those two classes are excluded outright so the rest of the suite can run, and the crashes themselves still need root-causing separately. Not yet covered: x86_64/i686 misc (job cancelled/not captured) and x86_64 commands (log expired before it could be fetched) -- still using the empty placeholder for those categories' contributions. Android is untouched here: its failures (missing libc++_static.a for the NDK sysroot at api level 28) are a build-environment problem, not per-test failures worth cataloging into an exclusion list -- that's part of the still-pending Android CI fix-up.
771cd38 to
2b84261
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b84261671
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| path: ${{ github.workspace }}/SourceCache/ds2 | ||
|
|
||
| - name: Install test dependencies | ||
| run: python -m pip install --quiet packaging psutil |
There was a problem hiding this comment.
On a clean Windows runner, these jobs install only the Python packages before invoking lldb-dotest, whose API tests use make to build their inferiors. Although build-lldb-windows installs the Chocolatey make package, that happens in a separate job and its PATH and installation do not carry into test-windows, so the new Windows test entries fail when their first test build tries to launch make; install it in this job as well.
Useful? React with 👍 / 👎.
Re-triaged from run 30147111711 (#231), now that the --daemonize and local-action-path fixes let the jobs actually execute: x86_64 functionalities, i686 commands + Python API, and aarch64 commands + Python API each ran to completion and their real FAIL/ERROR sets are recorded here. i686 and aarch64 functionalities both crashed lldb itself (Fatal Python error: Segmentation fault, in TestThreadJump and TestReturnValue.test_with_python respectively) before producing a usable failure list; those two classes are excluded outright so the rest of the suite can run, and the crashes themselves still need root-causing separately. Not yet covered: x86_64/i686 misc (job cancelled/not captured) and x86_64 commands (log expired before it could be fetched) -- still using the empty placeholder for those categories' contributions. Android is untouched here: its failures (missing libc++_static.a for the NDK sysroot at api level 28) are a build-environment problem, not per-test failures worth cataloging into an exclusion list -- that's part of the still-pending Android CI fix-up.
2b84261 to
fa775d8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa775d8802
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fa775d8 to
a098288
Compare
Re-triaged from run 30147111711 (#231), now that the --daemonize and local-action-path fixes let the jobs actually execute: x86_64 functionalities, i686 commands + Python API, and aarch64 commands + Python API each ran to completion and their real FAIL/ERROR sets are recorded here. i686 and aarch64 functionalities both crashed lldb itself (Fatal Python error: Segmentation fault, in TestThreadJump and TestReturnValue.test_with_python respectively) before producing a usable failure list; those two classes are excluded outright so the rest of the suite can run, and the crashes themselves still need root-causing separately. Not yet covered: x86_64/i686 misc (job cancelled/not captured) and x86_64 commands (log expired before it could be fetched) -- still using the empty placeholder for those categories' contributions. Android is untouched here: its failures (missing libc++_static.a for the NDK sysroot at api level 28) are a build-environment problem, not per-test failures worth cataloging into an exclusion list -- that's part of the still-pending Android CI fix-up.
a098288 to
4ab6d1f
Compare
Re-triaged from run 30147111711 (#231), now that the --daemonize and local-action-path fixes let the jobs actually execute: x86_64 functionalities, i686 commands + Python API, and aarch64 commands + Python API each ran to completion and their real FAIL/ERROR sets are recorded here. i686 and aarch64 functionalities both crashed lldb itself (Fatal Python error: Segmentation fault, in TestThreadJump and TestReturnValue.test_with_python respectively) before producing a usable failure list; those two classes are excluded outright so the rest of the suite can run, and the crashes themselves still need root-causing separately. Not yet covered: x86_64/i686 misc (job cancelled/not captured) and x86_64 commands (log expired before it could be fetched) -- still using the empty placeholder for those categories' contributions. Android is untouched here: its failures (missing libc++_static.a for the NDK sysroot at api level 28) are a build-environment problem, not per-test failures worth cataloging into an exclusion list -- that's part of the still-pending Android CI fix-up.
2afdf9c to
4047965
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4047965ac3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -1,131 +1 @@ | |||
| skip | |||
There was a problem hiding this comment.
The Linux and Android exclusion lists now contain only the skip section header, so the tools shards run the previously excluded TestNonStop and TestGdbRemoteForkNonStop cases. Those cases remain incompatible with ds2 because DebugSessionImplBase::onNonStopMode in Sources/GDBRemote/DebugSessionImpl.cpp:260-262 still returns kErrorUnsupported whenever non-stop mode is enabled; no corresponding implementation was added in this change. Retain the feature-driven exclusions (with any names updated for LLVM 23) or these test matrices will report known failures.
Useful? React with 👍 / 👎.
d89d88d to
0606e6f
Compare
Re-triaged from run 30147111711 (#231), now that the --daemonize and local-action-path fixes let the jobs actually execute: x86_64 functionalities, i686 commands + Python API, and aarch64 commands + Python API each ran to completion and their real FAIL/ERROR sets are recorded here. i686 and aarch64 functionalities both crashed lldb itself (Fatal Python error: Segmentation fault, in TestThreadJump and TestReturnValue.test_with_python respectively) before producing a usable failure list; those two classes are excluded outright so the rest of the suite can run, and the crashes themselves still need root-causing separately. Not yet covered: x86_64/i686 misc (job cancelled/not captured) and x86_64 commands (log expired before it could be fetched) -- still using the empty placeholder for those categories' contributions. Android is untouched here: its failures (missing libc++_static.a for the NDK sysroot at api level 28) are a build-environment problem, not per-test failures worth cataloging into an exclusion list -- that's part of the still-pending Android CI fix-up.
0606e6f to
49d9079
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49d9079d16
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -0,0 +1 @@ | |||
| skip | |||
There was a problem hiding this comment.
When the new Windows shards reach tests that exchange inferior stdin/stdout, such as the process-I/O functionality tests, this empty exclusion set allows them to run even though DebugSessionImplBase::spawnProcess relies on redirectInputToTerminal and the output-delegate methods, while Headers/DebugServer2/Host/Windows/ProcessSpawner.h:163-167 still returns false for all three. Those failures are ignored, and CREATE_NO_WINDOW now also leaves the inferior without a console, so LLDB receives neither output packets nor a usable input channel and the affected tests fail or hang. Implement Windows terminal/delegate redirection or exclude these tests for all three Windows architectures before enabling the shards.
Useful? React with 👍 / 👎.
Move CI off the stale main-branch pin to the release/23.x branch tip, matching what ds2's Windows platform-mode work has been validated against locally.
dotest.py dropped --arch for --triple as of release/23.x, so Test Linux and Test Android need the same migration or argument parsing fails outright. Android's TRIPLE has to stay unversioned (x86_64-none-linux-android): Android.rules derives ARCH_DIR by stripping "-none" from TRIPLE to find the NDK's per-arch library directory, and a versioned TRIPLE breaks that lookup. That unversioned --target lands after the --compiler wrapper's own versioned default in the actual compile invocation, and the last -target flag wins, silently dropping the API level. NDK r29's libc++ then unconditionally calls pthread_cond_clockwait, which bionic only declares from API 30 onward, breaking any test inferior that includes <condition_variable>. CFLAGS_EXTRAS can't fix this: many test Makefiles set their own, which wins over an inherited environment value under GNU Make's precedence rules. Wrapping the compiler to append the real target as its own last argument fixes this unconditionally; verified locally against NDK r29. The wrapper has to live 5 directories deep under a fake NDK root (toolchains/llvm/prebuilt/linux-x86_64/bin/...), with the real sysroot symlinked in: Android.rules derives NDK_ROOT purely from --compiler's own path nesting, and a flat wrapper directory broke that derivation, making every test inferior fail to build with "No unified toolchain sysroot found". Verified end-to-end locally.
Add build-lldb-windows (mirroring build-lldb-linux's per-arch runner matrix, native x64 and arm64 builds) and test-windows, driving ds2's platform mode with lldb-dotest the same way test-linux/test-android already do, across the same misc/commands/Python API/functionalities split. These invocations use --triple rather than --arch, matching the migration test-linux/test-android just picked up: dotest.py dropped --arch in favor of --triple as of the pinned release/23.x LLVM_REF. test-linux.yml and test-android.yml already exist as standalone files (the platform-specific split from a few commits back), so this adds a new test-windows.yml alongside them rather than reintroducing a single shared test.yml, and reuses the existing resolve-test-subdirs composite action for its own "Resolve test subdirectories" step instead of duplicating that logic a third time. No per-arch *-status jobs here: branch protection only requires the overall workflow to pass, and test-linux.yml/test-android.yml already dropped theirs earlier in the stack for the same reason. i686 reuses the x64 lldb-dotest build cross-compiling to i686-pc-windows-msvc, the same way Test Linux i686 reuses the x86_64 build -- this path is unverified and may need adjustment. Pin the lldb build's Python3_EXECUTABLE to the interpreter that test-windows later invokes lldb-dotest with. windows-latest carries several hostedtoolcache Python versions side by side, and CMake's find_package(Python3) picks the newest one it can see rather than the one the plain `python` command on PATH resolves to; without pinning, the compiled _lldb extension's ABI tag doesn't match the interpreter running the tests, and dotest.py fails to import lldb before a single test runs. windows-11-arm only has one Python version installed, so arm64 happened to avoid this by luck. The exclusion lists are empty placeholders for now; populating them with real failures is follow-up work, not blocking this wiring.
CreateProcessW was called without CREATE_NO_WINDOW, so any child launched without inherited or explicitly redirected stdio handles (the common case: debuggee launches never wire up a console today) got a brand-new console window auto-allocated by Windows. For short-lived processes -- every test-suite launch, for instance -- that window opens and closes before anything could read it, so it was pure UI noise, not a working way to observe output.
49d9079 to
2805fd5
Compare
Without an explicit timeout, a genuinely hung test run burns up to GitHub's 360-minute job default before anything notices. Set each job's timeout to roughly 2x its slowest observed category duration: Linux x86_64/i686 at 20 minutes (3-8 observed), Linux aarch64 at 15 (2-5.5 observed), Android x86_64 at 30 (5-14 observed), and Android arm64-v8a sanity at 15 (~7 observed).
1 participant
This pull request introduces significant improvements to the CI workflow for building and testing LLDB and ds2 across Linux, Android, and Windows platforms. The main changes include refactoring the test invocation into a reusable GitHub Action, modernizing and extending Windows build and test coverage (including ARM64), and updating exclusion lists for Android architectures.
CI Workflow Improvements:
.github/actions/run-lldb-tests) to standardize how LLDB tests are run against ds2 across platforms. This encapsulates the logic for starting ds2, configuring test parameters, and applying exclusion lists..github/workflows/build.ymlto use the new action, simplifying the workflow and ensuring consistency.--tripleargument instead of--archfor compatibility with upstream LLDB changes.Windows Support Enhancements:
General Workflow Updates:
release/23.x.android-aarch64andandroid-x86_64), reflecting improved test support or upstream changes. [1] [2]These changes collectively modernize the CI pipeline, improve maintainability, and expand cross-platform test coverage for ds2 and LLDB.