Skip to content

chore(sql): remove the superseded manual update scripts 01-22 - #7703

Open
aglinxinyuan wants to merge 2 commits into
apache:mainfrom
aglinxinyuan:chore/remove-superseded-sql-updates
Open

chore(sql): remove the superseded manual update scripts 01-22#7703
aglinxinyuan wants to merge 2 commits into
apache:mainfrom
aglinxinyuan:chore/remove-superseded-sql-updates

Conversation

@aglinxinyuan

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Deletes sql/updates/01.sql through 22.sql, which sit outside the applied migration chain — nothing in the repository can execute them. Pure deletion, no behaviour change: −989 lines.

The chain is defined solely by sql/changelog.xml, whose live changesets are 23-37. Its only mention of a lower number is sql/updates/1.sql, inside an <!-- example changeSet --> comment — and no such file exists, since the real ones are zero-padded. Neither bootstrap path needs the deleted files:

fresh DB      -> sql/texera_ddl.sql (one-time bootstrap, already the post-migration shape)
existing DB   -> already past changeset 23, so 01-22 ran long ago

Every runner was checked against that:

Runner Why 01-22 are unreachable
sql/docker-compose.yml liquibase runs --changeLogFile=changelog.xml; the updates/*.sql loop only sed-normalizes files into /tmp, and its glob still matches 23-37
bin/local-dev/main.sh parse_changelog_changesets skips XML comments and applies only referenced changesets
bin/single-node/docker-compose.yml mounts sql/ into /docker-entrypoint-initdb.d; the postgres entrypoint does not recurse into updates/
.github/workflows/automatic-email-notif-on-ddl-change.yml diffs with --diff-filter=A (added only), so deletions are inert

Reviewer note — one judgement call worth flagging. These are the historical manual-upgrade scripts operators were emailed about. An operator still running a pre-changeset-23 database would lose the only in-tree copy; it stays recoverable from git history, but is no longer browsable in the tree. Happy to add a short sql/updates/README.md breadcrumb pointing at the history if that is preferred — I kept this PR a pure deletion.

Any related issues, documentation, discussions?

Closes #7702

How was this PR tested?

Existing tests only — this PR removes files that no runner executes.

From the repo root:

  • bash bin/local-dev/tests/test_local_dev_sh.sh — 114 passed, including the two that guard this exact contract:

    • changelog parser: 15 changesets, commented example skipped
    • changelog: every referenced sql/updates file exists

    Its 3 failures (worktree pointer, listen_pid_for_port, install cmd: java via brew) are pre-existing on this Windows machine — confirmed identical with the deletion stashed, and caused by the absence of lsof/ss/brew rather than by this change.

Verification, re-runnable by a reviewer:

grep -oE 'sql/updates/[0-9]+\.sql' sql/changelog.xml | sort -V   # 1(commented) + 23..37 only
for n in $(seq -w 1 22); do git grep -l "$n\.sql" -- . ':!sql/updates'; done   # no hits

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

Copilot AI lite review requested due to automatic review settings August 17, 2026 02:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the ddl-change Changes to the TexeraDB DDL label Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@codecov-commenter

codecov-commenter commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.93%. Comparing base (1c4662b) to head (621a2ef).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7703      +/-   ##
============================================
- Coverage     90.93%   90.93%   -0.01%     
+ Complexity     4450     4449       -1     
============================================
  Files          1174     1174              
  Lines         47139    47139              
  Branches       5284     5284              
============================================
- Hits          42867    42865       -2     
- Misses         2579     2580       +1     
- Partials       1693     1694       +1     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø)
agent-service 98.62% <ø> (ø) Carriedforward from 1c4662b
amber 87.38% <ø> (-0.02%) ⬇️
computing-unit-managing-service 73.67% <ø> (ø)
config-service 86.73% <ø> (ø)
file-service 68.90% <ø> (ø)
frontend 92.43% <ø> (ø) Carriedforward from 1c4662b
notebook-migration-service 83.74% <ø> (ø)
pyamber 97.57% <ø> (ø) Carriedforward from 1c4662b
workflow-compiling-service 77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

✅ No material benchmark regressions detected

🟢 2 better · 🔴 0 worse · ⚪ 13 noise (<±5%) · 0 without baseline

Compared against main 1c4662b benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
bs=10 sw=10 sl=64 395 0.241 24,956/33,851/33,851 us ⚪ within ±5% / 🔴 +109.4%
🟢 bs=100 sw=10 sl=64 803 0.49 123,571/146,241/146,241 us 🟢 -6.0% / 🔴 +38.5%
bs=1000 sw=10 sl=64 909 0.555 1,098,300/1,140,381/1,140,381 us ⚪ within ±5% / 🔴 -14.0%
Baseline details

Latest main 1c4662b from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 395 tuples/sec 405 tuples/sec 784.16 tuples/sec -2.5% -49.6%
bs=10 sw=10 sl=64 MB/s 0.241 MB/s 0.247 MB/s 0.479 MB/s -2.4% -49.6%
bs=10 sw=10 sl=64 p50 24,956 us 25,177 us 12,626 us -0.9% +97.6%
bs=10 sw=10 sl=64 p95 33,851 us 34,510 us 16,169 us -1.9% +109.4%
bs=10 sw=10 sl=64 p99 33,851 us 34,510 us 18,986 us -1.9% +78.3%
bs=100 sw=10 sl=64 throughput 803 tuples/sec 823 tuples/sec 1,023 tuples/sec -2.4% -21.5%
bs=100 sw=10 sl=64 MB/s 0.49 MB/s 0.502 MB/s 0.625 MB/s -2.4% -21.6%
bs=100 sw=10 sl=64 p50 123,571 us 119,569 us 99,185 us +3.3% +24.6%
bs=100 sw=10 sl=64 p95 146,241 us 155,506 us 105,616 us -6.0% +38.5%
bs=100 sw=10 sl=64 p99 146,241 us 155,506 us 113,681 us -6.0% +28.6%
bs=1000 sw=10 sl=64 throughput 909 tuples/sec 918 tuples/sec 1,057 tuples/sec -1.0% -14.0%
bs=1000 sw=10 sl=64 MB/s 0.555 MB/s 0.56 MB/s 0.645 MB/s -0.9% -13.9%
bs=1000 sw=10 sl=64 p50 1,098,300 us 1,087,306 us 965,435 us +1.0% +13.8%
bs=1000 sw=10 sl=64 p95 1,140,381 us 1,117,128 us 1,012,135 us +2.1% +12.7%
bs=1000 sw=10 sl=64 p99 1,140,381 us 1,117,128 us 1,042,088 us +2.1% +9.4%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,506.57,200,128000,395,0.241,24955.74,33851.36,33851.36
1,100,10,64,20,2489.55,2000,1280000,803,0.490,123571.15,146240.82,146240.82
2,1000,10,64,20,21999.26,20000,12800000,909,0.555,1098299.95,1140380.65,1140380.65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ddl-change Changes to the TexeraDB DDL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the superseded manual SQL update scripts 01-22

3 participants