Skip to content

perf(sidebar): time and fix the sidebar, Open Quickly and autocomplete on 50,000 tables - #3830

Merged
datlechin merged 2 commits into
mainfrom
perf/large-catalog-measurements
Oct 24, 2026
Merged

datlechin merged 2 commits into
mainfrom
perf/large-catalog-measurements

Conversation

@datlechin

@datlechin datlechin commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Summary

  • scripts/benchmark-large-catalog.sh times, on the Measure the sidebar, Open Quickly and autocomplete on 100 schemas and 50,000 tables #3661 fixtures (PostgreSQL and Oracle with 100 schemas of 500 tables, MySQL with one database of 50,000), the sidebar's first listing after connect, one filter keystroke and opening a schema, Open Quickly's panel and first result, completion after FROM and after schema. (MySQL: database.), and resident memory. 10 runs each, budgets in scripts/benchmark/large-catalog/baseline.json (1.25x the first run's median and p95). A step also fails when the main thread stalls for more than 100 ms during it.
  • The app times these itself with TABLEPRO_INTERACTION_TRACE=1: InteractionTracer (Interaction intervals under Points of Interest, one stderr line each) and MainThreadStallMonitor (each main run loop turn of 50 ms or more). New CodeSuggestionDelegate.completionWindowDidShow(textView:itemCount:).
  • Sidebar: a wake from a shared service rebuilds the outline only when what this connection draws moved (DatabaseTreeRenderInputs); filter buckets, listing matches and table rows are memoized across refreshes; SidebarSearch ranks in one pass; the listing's 50,000-row mapping runs off the main actor.
  • Sidebar search on a one-row-per-schema tree (Oracle, Dameng, SAP HANA, Trino) draws a schema it opens from the all-schema listing instead of reading it, and reads no schema until that listing lands: a one-character search queued 19 schema reads ahead of it on the one metadata connection, and the first keystroke took 8.1 s.
  • Oracle schema rows are selectable, so the keyboard can reach and open them.
  • Oracle reads that keep every row fetch 1,000 rows per round trip: the 50,000-table listing went from 1,001 fetches to 51.
  • Open Quickly builds its table rows off the main actor (each open rebuilt 50,000 rows on it), rebuilds them only when the rows it reads change, and stops a superseded ranking early.
  • Autocomplete ranks off the main actor, keeps the pool for the prefix it answered, and asks again when a deleted character leaves a prefix that pool cannot answer. A pool cut at 400 no longer hides the best match. schema. finds an unquoted schema in any case and keeps a quoted one as written. database. lists that database's tables on an engine without schemas.
  • AI chat ranks the schema off the main actor when a message is sent. The footer @ Tables menu lists the first 100 tables; @ in the message finds any other.
  • MCP list_indexes reads a whole schema's indexes in one query where the driver supports it.
  • Menu validation probes a driver's statement hooks once per driver.
  • Fixture script: MySQL over a socket, sessions start in the first schema.

Before is the first run (2026-10-22, now the baseline), after is this branch. Release build, 10 runs, Mac16,11 (M4 Pro, 24 GB, macOS 27.0.1), PostgreSQL 17 local, Oracle Free 23 and MySQL 8.4 in containers, load 4 to 8.

Fixture Step Before ms (median / p95) Before worst stall After ms (median / p95) After worst stall Budget
PostgreSQL sidebar-listing 445 / 506 271 336 / 449 251 560 / 635
sidebar-filter 124 / 380 192 89 / 472 70 155 / 480
sidebar-expand 133 / 243 58 79 / 112 0 170 / 305
quick-open-panel 170 / 261 51 62 / 98 91 215 / 330
quick-open-result 236 / 369 0 174 / 183 0 295 / 465
complete-from 85 / 92 0 69 / 72 0 110 / 120
complete-schema 122 / 140 0 75 / 83 0 155 / 175
Oracle sidebar-listing 1,445 / 2,185 964 1,311 / 2,077 787 1,810 / 2,735
sidebar-filter 138 / 668 195 103 / 1,046 82 175 / 840
sidebar-expand not measurable 282 / 338 0 355 / 425
quick-open-panel 173 / 203 60 77 / 93 86 220 / 255
quick-open-result 172 / 208 0 156 / 168 0 215 / 260
complete-from 84 / 91 0 68 / 75 0 105 / 115
complete-schema 499 / 540 0 442 / 508 0 625 / 675
MySQL sidebar-listing 574 / 658 222 476 / 544 217 720 / 825
sidebar-filter 134 / 196 140 81 / 84 71 170 / 245
sidebar-expand 286 / 317 112 244 / 251 96 360 / 400
quick-open-panel 156 / 186 0 73 / 91 78 195 / 235
quick-open-result 142 / 150 0 126 / 138 0 180 / 190
complete-from 103 / 110 0 77 / 87 0 130 / 140
complete-schema completed nothing 95 / 116 0 120 / 145

Resident memory with the table list loaded: PostgreSQL 182 MB, Oracle 162 MB, MySQL 492 MB (before: 119, 194, 772).

Still over:

  • sidebar-listing stalls 200 to 310 ms on every fixture, the same on a two-table PostgreSQL database: it is the connection window's first SwiftUI build (editor, results pane, sidebar rows, toolbar), not the catalog. It needs its own design.
  • Oracle sidebar-filter p95 is the first keystroke of the session, which waits for the all-schema listing; Oracle's own time for that query on this fixture was 0.55 to 1.2 s (V$SQL). The other nine runs were 80 to 109 ms.

Before / After

Oracle, typing LC_S050 in the sidebar. Before: type-select skips schema rows and lands on a table. After: the schema row is selected.

Before After
Light Oracle before, light Oracle after, light
Dark Oracle before, dark Oracle after, dark

AI chat footer @ > Tables on a schema of 500 tables. Before: all 500. After: the first 100 and a hint to type @.

Before After
Light AI Tables menu before, light AI Tables menu after, light
Dark AI Tables menu before, dark AI Tables menu after, dark

Test plan

  • New: DatabaseTreeReconcileTests, DatabaseTreeFilterMemoTests, DatabaseTreeHierarchicalSearchTests, InteractionTraceRecorderTests, DatabaseObjectToolSupportTests, AIChatTableMentionsTests, AIChatViewModelSchemaRankingTests, MCPListIndexesTests, OracleRowLimitsTests, and cases in QuickSwitcherListingDemandTests, SQLSchemaProviderUnqualifiedScopeTests, SQLCompletionProviderCatalogTests, SQLContextAnalyzerQuotedReferenceTests, DatabaseTreeNodeTests, SuggestionLiveCursorTests, SidebarSearchTests, SidebarNameFilterTests.
  • Changed on purpose: AllSchemaTablesDemandTests.resetForgetsEarlierRequests (a reset asks again only for a listing that is not current).
  • Local: 1,027 hosted and package-in-plan tests across the sidebar, Quick Switcher, autocomplete, AI chat and editor suites; swift test for TableProAutocomplete (475) and TableProOracle (316); lint clean.
  • Keys in the benchmark came from a library loaded into the measured copy that posts CGEvent-built events into its own queue: this session has no Accessibility grant, and XCTest's Automation Mode asks for the owner's password.

Refs #3661

@datlechin
datlechin enabled auto-merge (squash) October 24, 2026 22:08
@datlechin
datlechin merged commit 5533f2e into main Oct 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant