Skip to content

Mobile queries benchmark - #51

Merged
dewabisma merged 5 commits into
mainfrom
beast/mobile-queries-benchmark
Sep 23, 2026
Merged

dewabisma merged 5 commits into
mainfrom
beast/mobile-queries-benchmark

Conversation

@dewabisma

Copy link
Copy Markdown
Collaborator

Added benchmark for mobile queries

@dewabisma
dewabisma requested a review from n13 September 23, 2026 03:15

@n13 n13 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT 5.6 Sol

REQUEST_CHANGES: The mobile suite does not benchmark the current app's history-query plan, and it can report a successful run without executing any benchmark query.

  1. [P1] Mirror the query shape the mobile app actually sends (src/lib/graphql-benchmark/mobile-queries.ts:163, src/lib/graphql-benchmark/mobile-registry.ts:97). The new AccountEvents*/scheduled documents use one account_event selection with account_id: {_in: $accounts}, order only by timestamp/id, and the registry always supplies one account. Current quantus-apps/main@11e035a3 deliberately emits one aliased _eq selection per wallet account and includes account_id plus the direction column in order_by; its builder explains that _in does not use the intended composite index. These shapes produce materially different database plans, and the current suite also misses multi-account fan-out, so its timings cannot characterize the mobile app or guard its performance. Please generate the same alias-per-account documents/order and exercise representative account counts, ideally with a drift check against the SDK query builders.

  2. [P1] Fail when mobile bootstrap cannot reach or query the endpoint (src/lib/graphql-benchmark/mobile-bootstrap.ts:14, scripts/graphql-bench.ts:58). Every bootstrap exception is discarded, while skipped rows are excluded from hasFailure. Reproduction: GRAPHQL_BENCH_URL=http://127.0.0.1:1/graphql npx tsx scripts/graphql-bench.ts --suite mobile --samples 1 prints an empty context, marks every operation SKIPPED, and exits 0. This makes an unavailable endpoint or incompatible schema look like a successful benchmark, and the new per-query timeout does not cover bootstrap. Preserve optional-data skips, but propagate/log bootstrap request failures and return nonzero when no meaningful suite ran; apply the caller's abort/timeout during bootstrap as well.

Validation at exact head 4274f298ec2b9054186b707228e7fd54a2d1bedd:

  • npm run build (Vite + tsc): passed; Bun was unavailable locally, so dependencies were installed transiently with npm.
  • git diff --check and Prettier over all changed files: passed.
  • Live --suite mobile --samples 1 against the default endpoint: completed and confirmed the documents are schema-valid; four data-dependent operations were skipped.
  • Invalid-endpoint reproduction above: exited 0 with all operations skipped.
  • ESLint over changed files: failed on import/export sorting in index.ts, mobile-bootstrap.ts, and mobile-registry.ts.
  • GitHub's exact-head build and dependency-cooldown checks are green. Jest could not collect tests in the local npm environment because the repository's TS transform setup was not applied.

The two correctness issues above are blocking.

@dewabisma
dewabisma requested a review from n13 September 23, 2026 05:07

@n13 n13 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT 5.6 Sol

APPROVE: The two blocking findings from the review at 4274f298 are resolved at dade877e; no blocking findings remain.

  • The history benchmarks now reproduce quantus-apps/main@11e035a3's alias-per-account _eq query shape, composite ordering, keyset pagination, and multi-account fan-out.
  • Mobile bootstrap requests now use the benchmark timeout/abort signal, expose request failures, and make an unavailable endpoint fail the CLI instead of succeeding with an entirely skipped suite.
  • The latest commit fixes the TypeScript error in the new failure-helper test; both the local production build and GitHub's exact-head Build and Test check pass.

Validation:

  • npm run build, npx tsc --noEmit --pretty false, git diff --check, and changed-file Prettier: passed.
  • Live --suite mobile --samples 1 against sqm.quantus.com: completed with all runnable queries successful, including the new 2- and 8-account shapes.
  • Invalid-endpoint reproduction: reported the bootstrap failure, skipped dependent operations, and exited 1.
  • The repository's Jest configuration still cannot collect TypeScript tests because no TypeScript transform is configured. Changed-file ESLint also retains one non-blocking simple-import-sort/exports error in src/lib/graphql-benchmark/index.ts.

@dewabisma
dewabisma merged commit 5aad986 into main Sep 23, 2026
2 checks passed
@dewabisma
dewabisma deleted the beast/mobile-queries-benchmark branch September 23, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants