[Makisuo]

What

Every CI read moves from raw traces to ci_runs / ci_jobs: the CI page (totals, series, workflows, jobs, red streaks, runs list), the pull request sheet's CI runs and the ci_runs MCP tool. Attempt and pull request come back as numbers, so the string parsing in the service goes.

Numbers

Same synthetic dataset as the previous PR. One page load of the CI tab (today's 11 reads, the previous period's totals approximated by the current one):

org / window	raw traces	rollup
Maple-sized, 7d (the default)	1.3 GB, 83 ms	9.6 MB, 7 ms
Maple-sized, 30d	5.4 GB, 321 ms	26 MB, 17 ms
10x org, 30d	9.8 GB, 798 ms	117 MB, 75 ms

Times are isolated server time on the critical path of the three waves; prod adds cold reads and up to seven concurrent queries on top, which is where the 4 to 15 s tails came from. Merging the page's reads into fewer scans (GROUPING SETS) read less but lengthened the critical path, so the reads stay separate.

The shipped builders, run through apps/api/scripts/bench-suites/ci.ts on the same data: every 30-day read is 2 to 12 ms and under 6 MB.

Parity

All 54 overlapping raw-vs-rollup cases (3 orgs x 7d/30d x 9 queries) return identical counts, sums, latest results, run lists, PR filters, streaks and failed jobs. Only sampled quantile() values drift, as they do between two raw runs.

Self-hosted ClickHouse

A cluster that has not applied migration 0047 has no CI tables. The MCP tool answers with an explicit "apply the schema from its ClickHouse settings page" (McpUnavailableError); the CI page shows the classified warehouse config error, as funnels do for product_events.

Verification

• ci-runs.clickhouse.e2e.test.ts now also runs every CI builder on seeded runs and jobs: totals, failures vs cancellations, series, per-workflow latest result, jobs that ran (zero-length and skipped excluded) with reruns and queue time, newest-first paging after a cursor, the PR filter, the red streak and failed jobs. A mutation in the expected order fails it.
• The SQL-text pins in ci-runs.test.ts moved to that e2e.
• query-engine (1658), backend pr-review (234), ai MCP (760) tests; SQL catalog baseline updated; scoped typecheck and effect lint.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

Stack: #1718 (tables) → #1719 (reads) → #1720 (one-year range)

Stack: #1718 (tables) → #1719 (reads) → #1720 (one-year range)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • CI run, job, and pipeline analytics now use dedicated CI data, improving consistency across run totals, trends, filtering, and failed-job results.
  • Run attempts display as numbers and are shown as at least 1. Pull requests without an associated number are no longer shown as pull request 0.
  • When CI data tables are missing, the CI tool now reports that the self-hosted ClickHouse schema may need attention.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 1036cc0c-271c-41f3-9eb6-58a696b975ee

📥 Commits

Reviewing files that changed from the base of the PR and between 4e74953 and fa3ad2b.

📒 Files selected for processing: 16

• .github/workflows/ci.yml
• apps/ai/src/mcp/__evals__/integration-world.ts
• apps/ai/src/mcp/tools/ci-runs.ts
• apps/api/scripts/bench-suites/ci.ts
• packages/backend/src/services/integrations/ci-traces/otlp.ts
• packages/backend/src/services/pr-review/CiAnalyticsService.test.ts
• packages/backend/src/services/pr-review/CiAnalyticsService.ts
• packages/backend/src/services/warehouse/ci-runs.clickhouse.e2e.test.ts
• packages/backend/src/services/warehouse/missing-table.ts
• packages/domain/src/mcp-outputs/ci.ts
• packages/domain/src/tinybird/datasources.ts
• packages/domain/src/tinybird/materializations.ts
• packages/query-engine/src/__sql_baseline__/catalog.sql
• packages/query-engine/src/ch/index.ts
• packages/query-engine/src/ch/queries/ci-runs.test.ts
• packages/query-engine/src/ch/queries/ci-runs.ts

💤 Files with no reviewable changes: 1

• packages/query-engine/src/ch/index.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

📝 Walkthrough

CI analytics queries now read normalized ci_runs and ci_jobs tables. Numeric attempt and pull-request values flow through the domain and service layers. ClickHouse end-to-end tests and a CI workload benchmark cover the queries. MCP reports a schema-specific unavailable error when CI tables are missing.

Changes: CI analytics table migration

Layer / File(s)	Summary
Shared CI data contracts packages/domain/src/tinybird/datasources.ts, packages/domain/src/tinybird/materializations.ts, packages/backend/src/services/integrations/ci-traces/otlp.ts, packages/domain/src/mcp-outputs/ci.ts	The CI service name is exported from the domain package and used by materializations and OTLP conversion. The MCP run attempt schema now uses a number.
Queries over normalized CI tables packages/query-engine/src/ch/queries/ci-runs.ts, packages/query-engine/src/__sql_baseline__/catalog.sql, packages/query-engine/src/ch/index.ts, packages/query-engine/src/ch/queries/ci-runs.test.ts	Run, job, and aggregate queries now read CI fields from ci_runs and ci_jobs. The SQL baselines and exports reflect the query changes. SQL-compilation tests were removed while the ciRunsSpanMs and ciRedStreak tests remain.
CI service and MCP adaptation packages/backend/src/services/warehouse/missing-table.ts, packages/backend/src/services/pr-review/CiAnalyticsService.ts, packages/backend/src/services/pr-review/CiAnalyticsService.test.ts, apps/ai/src/mcp/tools/ci-runs.ts, apps/ai/src/mcp/__evals__/integration-world.ts	Backend serialization and MCP fixtures use numeric attempt and pull-request values. MCP maps missing CI tables to a ci_runs unavailable error. Service tests and evaluation fixtures use normalized table columns.
ClickHouse query validation packages/backend/src/services/warehouse/ci-runs.clickhouse.e2e.test.ts, .github/workflows/ci.yml, apps/api/scripts/bench-suites/ci.ts	ClickHouse end-to-end tests cover run, job, and aggregate query results, plus migration backfills. The CI workflow runs this test, and a ci-workload benchmark suite is added.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Refactor

Merge Risk: ⚪ Minimal · up to fa3ad

No merge-blocking issue was identified in the CI analytics switch; it is ready for normal checks.

Pre-merge checks: 5

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: CI analytics now read from the ci_runs and ci_jobs projections instead of raw traces.
Docstring Coverage		Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 13 files. (2 skipped: 2 …
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 13 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[maple-review-bot]

Maple review

🟢 Confidence 10/10 · safe to merge
Every CI read moves onto ci_runs/ci_jobs with OrgId intact, columns verified against migration 0047, and the numeric attempt/pull request conversion is pinned by the ClickHouse e2e and the servi…
_{no findings · tests covered · risk medium}

Moves every CI read (analytics page, PR sheet, ci_runs MCP tool) from raw traces to the ci_runs / ci_jobs projections, and lets attempt and pull request arrive as numbers. Parity holds and the missing-table case is handled; safe to merge once the 0047 tables are live.

• ci-runs.ts builders read CiRuns/CiJobs columns instead of traces attributes
• ciJobStatsQuery counts reruns as Attempt > 1 rather than run_attempt != '1'
• CiRunRow.attempt becomes a Number; the MCP tool and service drop their string parsing
• The ci_runs tool maps a missing table to McpUnavailableError with remediation

Before merge

• Warehouse · Deploy the schema before the code that reads it · packages/domain/src/tinybird/datasources.ts
• Manual · Deploy only after migration 0047 is applied: the CI page and the ci_runs tool now fail without ci_runs/ci_jobs
• Manual · Self-hosted ClickHouse orgs must apply the schema or the CI tab shows the warehouse config error

Production impact

Telemetry this change adds and removes (16)

• ➖ attribute cicd.pipeline.task.name (~33k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:389
• ➖ attribute cicd.pipeline.task.run.result (~33k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:389
• ➖ attribute maple.cicd.failure.reason (~97/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:389
• ➖ attribute cicd.pipeline.task.run.url.full (~33k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:389
• ➖ attribute maple.cicd.step.number (~29k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:389
• ➖ attribute cicd.pipeline.name (~36k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:409
• ➖ attribute maple.cicd.run_attempt (~36k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:414
• ➖ attribute maple.cicd.queue_duration_ms (~6.7k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:414
• ➖ attribute cicd.pipeline.result (~779/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:438
• ➖ attribute cicd.pipeline.run.id (~54k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute cicd.pipeline.run.url.full (~779/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute vcs.ref.head.name (~49k/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute vcs.ref.head.revision (~13.2M/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute maple.cicd.trigger_event (~779/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute maple.cicd.actor (~779/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491
• ➖ attribute vcs.change.id (~670/day today) · packages/query-engine/src/__sql_baseline__/catalog.sql:491

What was checked

• ciScope, ciRunsQuery and ciFailedJobsQuery all still filter OrgId (queries/ci-runs.ts:57,98,165)
• ci_jobs carries StatusCode and Attempt columns, so the Error filter and rerun count read real data (materializations.ts:1297,1300)
• attempt/pullRequest are uint16/uint32, so ClickHouse's JSON output is numeric and matches the new Schema.Number

_{fa3ad2b · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.}

[Makisuo]

Merge order for the deploy items above: merge #1718 first and let its prd deploy create and fill ci_runs / ci_jobs (Tinybird replays the views over the retained traces), then merge this one. Self-hosted clusters: until 0047 is applied the MCP tool answers with McpUnavailableError (apply the schema), and the CI tab shows the classified warehouse config error.

[Makisuo]

Real-data check (Maple's own CI, 2026-10-14..22)

Exported the org's 333k github-actions spans read-only from prod into a local ClickHouse 25.8, built ci_runs / ci_jobs with migration 0047's SQL (6,486 runs and 35,420 jobs, matching prod), and ran the 7-day page reads both ways (bench:queries, 7 runs, 2 threads):

query	raw traces	rollup
run totals	19 ms / 107 MB	3 ms / 0.12 MB
run series	19 ms / 107 MB	3 ms / 0.12 MB
workflow stats	26 ms / 168 MB	4 ms / 0.14 MB
job stats	43 ms / 162 MB	7 ms / 1.0 MB
queue totals	31 ms / 101 MB	4 ms / 0.78 MB
runs list	25 ms / 179 MB	3 ms / 1.5 MB

This local raw table holds only CI spans; prod's also holds every other service's, so prod reads more (205 MB for one queue query, 16 s cold for a 30-day scan).

Real rows compress to 91 B per run and 52 B per job: 2.3 MB for nine days of the org's CI.

Parity on the real data: all 9 reads return identical rows. The one visible difference, queue p95 (381 s vs 400 s), is quantile() sampling over 21k values: quantileExact(0.95) is 391,000 ms on both sides.