diff --git a/.github/workflows/result-server-tests.yml b/.github/workflows/result-server-tests.yml index a64b9e0..b5b82a5 100644 --- a/.github/workflows/result-server-tests.yml +++ b/.github/workflows/result-server-tests.yml @@ -19,7 +19,9 @@ on: - "scripts/tests/test_send_estimate_artifacts.sh" - "scripts/tests/test_send_results_profile_data.sh" - "scripts/test_estimate_submit.sh" + - "programs/**/build.sh" - "programs/**/list.csv" + - "programs/**/run.sh" - "programs/genesis/estimate.sh" - "programs/genesis/run.sh" - "programs/qws/estimate.sh" @@ -48,7 +50,9 @@ on: - "scripts/tests/test_send_estimate_artifacts.sh" - "scripts/tests/test_send_results_profile_data.sh" - "scripts/test_estimate_submit.sh" + - "programs/**/build.sh" - "programs/**/list.csv" + - "programs/**/run.sh" - "programs/genesis/estimate.sh" - "programs/genesis/run.sh" - "programs/qws/estimate.sh" diff --git a/docs/ci.md b/docs/ci.md index 4602c0c..df0d01b 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -254,7 +254,8 @@ app support matrix、partial support、app entrypoint不足、`list.csv` 内の | `result_server/**/*` / `result_server/**/*` | `Result Server Tests` | Skipped by `.gitlab-ci.yml` rules / `.gitlab-ci.yml` rulesでskip | Portal regressions should be caught by lightweight Python tests / portal回帰はlightweight Python testで捕捉する | | Public site config or portal metadata `config/system.csv`, `config/queue.csv`, `config/system_info.csv` / 公開site configまたはportal表示メタデータ`config/system.csv`、`config/queue.csv`、`config/system_info.csv` | `Result Server Tests`, including site config preflight / site config preflightを含む`Result Server Tests` | `config/system.csv` and `config/queue.csv` run by `.gitlab-ci.yml`; `config/system_info.csv` is skipped / `config/system.csv`と`config/queue.csv`は`.gitlab-ci.yml`で実行、`config/system_info.csv`はskip | Public systems listed in `system_info.csv` must also exist in `system.csv` and reference a queue defined in `queue.csv` / `system_info.csv`に載せる公開systemは`system.csv`にも存在し、`queue.csv`定義済みqueueを参照する必要がある | | Portal upload or profile-data helper `scripts/bk_functions.sh`, `scripts/result.sh`, `scripts/result_server/**` / portal uploadまたはprofile-data helper `scripts/bk_functions.sh`、`scripts/result.sh`、`scripts/result_server/**` | `Result Server Tests` when covered by its path filter; `Shellcheck` for `.sh` changes / path filter対象なら`Result Server Tests`; `.sh`変更は`Shellcheck` | GitHub pull requests do not start GitLab by default; if a direct/manual GitLab pipeline is started, `scripts/**/*` is treated as benchmark-affecting and runs / GitHub pull requestでは既定でGitLabは起動しない。直接/手動GitLab pipelineを起動した場合、`scripts/**/*` はbenchmark影響ありとして実行される | These helpers shape result JSON / upload behavior. Use lightweight tests first, then start `GitLab Manual CI` when benchmark-side behavior needs validation / これらのhelperはResult JSONやupload挙動へ影響する。まずlightweight testで確認し、benchmark側挙動の検証が必要な場合は`GitLab Manual CI`を起動する | -| Benchmark app, BenchPark bridge, or other shared scripts / benchmark app、BenchPark bridge、またはその他の共通script | `Shellcheck` for `.sh` changes; otherwise normal GitHub review checks / `.sh`変更は`Shellcheck`; それ以外は通常のGitHub review check | Run through `GitLab Manual CI` when maintainer starts it / maintainerが`GitLab Manual CI`を起動した場合に実行 | Use `code`, `system`, `benchpark`, or `park_only` filters when broad validation is unnecessary / 広範な検証が不要なら`code`、`system`、`benchpark`、`park_only`を指定する | +| Benchmark app entrypoints `programs/**/build.sh`, `programs/**/run.sh`, or app matrix `programs/**/list.csv` / benchmark app entrypoint `programs/**/build.sh`、`programs/**/run.sh`、またはapp matrix `programs/**/list.csv` | `Result Server Tests` for portal app-support visibility; `Shellcheck` for `.sh` changes / portal app-support表示向けに`Result Server Tests`; `.sh`変更は`Shellcheck` | Run through `GitLab Manual CI` when maintainer starts it / maintainerが`GitLab Manual CI`を起動した場合に実行 | App support matrix reads list/build/run for `/results/usage`; these checks provide visibility, not a readiness gate / app support matrixは`/results/usage`用にlist/build/runを読む。これはvisibilityでありreadiness gateではない | +| Other benchmark app files, BenchPark bridge, or shared scripts / その他のbenchmark appファイル、BenchPark bridge、または共通script | `Shellcheck` for `.sh` changes when covered; otherwise normal GitHub review checks / 対象`.sh`変更は`Shellcheck`; それ以外は通常のGitHub review check | Run through `GitLab Manual CI` when maintainer starts it / maintainerが`GitLab Manual CI`を起動した場合に実行 | Use `code`, `system`, `benchpark`, or `park_only` filters when broad validation is unnecessary / 広範な検証が不要なら`code`、`system`、`benchpark`、`park_only`を指定する | | GitHub workflow/action `.github/**/*` / GitHub workflow/action `.github/**/*` | Workflow-specific checks when paths match / path一致時にworkflowごとのcheck | Skipped by `.gitlab-ci.yml` rules / `.gitlab-ci.yml` rulesでskip | GitHub workflow/action changes affect API-calling or sync control logic. Validate them on the GitHub side; they are pushed to GitLab with `ci.skip` during protected-branch sync / GitHub workflow/action変更はAPI呼び出しやsync制御に影響する。GitHub側で確認する。protected-branch syncでは`ci.skip`付きでGitLabへpushされる | | `.gitlab-ci.yml` / `.gitlab-ci.yml` | Normal GitHub review checks only / 通常のGitHub review checkのみ | Run through `GitLab Manual CI` when a maintainer needs to validate GitLab pipeline behavior / GitLab pipeline挙動の検証が必要な場合にmaintainerが`GitLab Manual CI`で実行 | This file defines GitLab benchmark pipeline behavior / このファイルはGitLab benchmark pipeline挙動を定義する | @@ -271,7 +272,8 @@ Use these examples when deciding whether to split a pull request or start GitLab | `config/system_info.csv` only / `config/system_info.csv`のみ | `Result Server Tests` should verify public site config consistency / 公開site config整合性を`Result Server Tests`で確認 | No benchmark run because this file is portal display metadata / portal表示metadataなのでbenchmark不要 | | `config/system.csv` or `config/queue.csv` for a public system / 公開system向けの`config/system.csv`または`config/queue.csv` | `Result Server Tests` should run the site config preflight / `Result Server Tests`でsite config preflightを実行 | Start `GitLab Manual CI` too when benchmark execution behavior needs validation / benchmark実行挙動の検証が必要なら`GitLab Manual CI`も起動 | | `scripts/bk_functions.sh`, `scripts/result.sh`, or `scripts/result_server/**` only / `scripts/bk_functions.sh`、`scripts/result.sh`、または`scripts/result_server/**`のみ | `Result Server Tests` should run when the path filter matches; `Shellcheck` should run for `.sh` changes / path filter対象なら`Result Server Tests`が動く; `.sh`変更では`Shellcheck`が動く | Protected-branch sync uses `ci.skip`; direct/manual GitLab pipelines run because `.gitlab-ci.yml` treats `scripts/**/*` as benchmark-affecting / protected branch syncは`ci.skip`を使う。直接/手動GitLab pipelineでは`.gitlab-ci.yml`が`scripts/**/*`をbenchmark影響ありとして扱うため実行される | -| `programs/qws/**/*`, `scripts/job_functions.sh`, or `scripts/test_submit.sh` / `programs/qws/**/*`、`scripts/job_functions.sh`、または`scripts/test_submit.sh` | `Shellcheck` should run for `.sh` changes / `.sh`変更では`Shellcheck`が動く | Start `GitLab Manual CI` when benchmark validation is needed, preferably with explicit `code` and `system` filters / benchmark検証が必要なら`code`と`system`を明示して`GitLab Manual CI`を起動 | +| `programs/**/build.sh`, `programs/**/run.sh`, or `programs/**/list.csv` / `programs/**/build.sh`、`programs/**/run.sh`、または`programs/**/list.csv` | `Result Server Tests` should run for app-support visibility; `Shellcheck` should run for `.sh` changes / app-support visibility向けに`Result Server Tests`が動く; `.sh`変更では`Shellcheck`が動く | Start `GitLab Manual CI` when benchmark validation is needed, preferably with explicit `code` and `system` filters / benchmark検証が必要なら`code`と`system`を明示して`GitLab Manual CI`を起動 | +| `scripts/job_functions.sh` or `scripts/test_submit.sh` / `scripts/job_functions.sh`または`scripts/test_submit.sh` | `Shellcheck` should run for `.sh` changes / `.sh`変更では`Shellcheck`が動く | Start `GitLab Manual CI` when scheduler generation or submission behavior needs validation / scheduler生成や投入挙動の検証が必要なら`GitLab Manual CI`を起動 | | `benchpark-bridge/scripts/**/*.sh` / `benchpark-bridge/scripts/**/*.sh` | `Shellcheck` should run / `Shellcheck`が動く | Start `GitLab Manual CI` with `benchpark=true` or `park_only=true` when BenchPark execution behavior needs validation / BenchPark実行挙動の検証が必要なら`benchpark=true`または`park_only=true`で`GitLab Manual CI`を起動 | | `.github/workflows/sync-to-gitlab.yml` or `.github/actions/prepare-gitlab-repo/action.yml` / `.github/workflows/sync-to-gitlab.yml`または`.github/actions/prepare-gitlab-repo/action.yml` | Validate on the GitHub Actions side / GitHub Actions側で確認 | Skipped by `.gitlab-ci.yml` rules when changed alone; protected-branch sync pushes it with `ci.skip` / 単独変更なら`.gitlab-ci.yml` rulesでskip。protected-branch syncでは`ci.skip`付きでpushされる | | `.gitlab-ci.yml` / `.gitlab-ci.yml` | Review the GitLab rule diff carefully / GitLab rule差分を慎重にreview | Start `GitLab Manual CI` if rule behavior itself needs validation / rule挙動そのものの検証が必要なら`GitLab Manual CI`を起動 | diff --git a/docs/cx/BENCHKIT_GAP_ANALYSIS.md b/docs/cx/BENCHKIT_GAP_ANALYSIS.md index b9592c1..42a1481 100644 --- a/docs/cx/BENCHKIT_GAP_ANALYSIS.md +++ b/docs/cx/BENCHKIT_GAP_ANALYSIS.md @@ -373,7 +373,7 @@ Once the estimation specification is clarified, many other design decisions beco CI 関連の残作業は、「仕組みを新規に置く」段階から「対象範囲を運用に耐える形へ広げ、古くならないようにする」段階へ移っている。 短期的な実装・確認は次の状態まで進んでいる。 -1. `result-server-tests.yml` の path filter は、`result_server/**/*`、`scripts/bk_functions.sh`、`scripts/result.sh`、`scripts/result_server/**`、profile-data shell tests、`config/system.csv`、`config/queue.csv`、`config/system_info.csv`、`requirements-result-server.txt` を対象にする形へ更新済みである。 +1. `result-server-tests.yml` の path filter は、`result_server/**/*`、`scripts/bk_functions.sh`、`scripts/result.sh`、`scripts/result_server/**`、profile-data shell tests、`programs/**/list.csv`、`programs/**/build.sh`、`programs/**/run.sh`、`config/system.csv`、`config/queue.csv`、`config/system_info.csv`、`requirements-result-server.txt` を対象にする形へ更新済みである。 2. `.gitlab-ci.yml` の heavy benchmark skip rules と `docs/ci.md` の説明は、root Markdown、`docs/**/*`、`result_server/**/*`、`requirements-result-server.txt`、`config/system_info.csv` の skip 対象について同期済みである。一方、`scripts/bk_functions.sh`、`scripts/result.sh`、`scripts/result_server/**` は GitHub Actions の lightweight verification 対象でもあるが、直接または手動で GitLab pipeline を起動した場合は `.gitlab-ci.yml` の `scripts/**/*` rule により benchmark-affecting として実行される。 3. 手動 GitLab CI は、`qws` / `MiyabiG` の最小実行で GitLab pipeline 起動から推定まで確認済みである。Pipeline API variables は JSON payload で渡す。 4. protected branch sync は、`ci.skip` により GitLab mirror 更新時に GitLab CI が自動起動しないことを運用上確認済みである。