sqlite: cache column names in StatementSync::All() and Get() by araujogui · Pull Request #62793 · nodejs/node

araujogui · 2026-04-17T17:22:44Z

Cache columns names in StatementSync::All() and StatementSync::Get() too. Currently, it's cached only on StatementSync::Iterate()

This improves single-row query performance (~10%) while leaving multi-row throughput essentially unchanged.

nodejs-github-bot · 2026-04-17T17:22:50Z

Review requested:

@nodejs/sqlite

Copilot

Pull request overview

This PR extends StatementSync’s existing column-name caching (previously used by Iterate()) to also cover StatementSync::All() and StatementSync::Get(), aiming to improve single-row query performance by avoiding repeated V8 string creation for column keys.

Changes:

Update StatementExecutionHelper::All() / Get() to accept a StatementSync* so they can access the statement’s cached column-name machinery.
Use StatementSync::GetCachedColumnNames() in All() and Get() when returning row objects (non-array mode).
Update call sites (including SQLTagStore) to use the new helper signatures.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
`src/node_sqlite.h`	Adjusts `StatementExecutionHelper` function signatures to take `StatementSync*`.
`src/node_sqlite.cc`	Implements column-name caching usage in `All()`/`Get()` and updates helper call sites accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-17T17:26:29Z

+  sqlite3_stmt* stmt = sync_stmt->statement_;
  int num_cols = sqlite3_column_count(stmt);
  LocalVector<Value> rows(isolate);
  LocalVector<Value> row_values(isolate);
  LocalVector<Name> row_keys(isolate);

+  if (!return_arrays && num_cols > 0) {
+    if (!sync_stmt->GetCachedColumnNames(&row_keys)) {
+      return MaybeLocal<Value>();
+    }
+  }
+


In StatementExecutionHelper::All(), num_cols and the cached row_keys are computed before the first sqlite3_step(). SQLite can auto-reprepare a statement during sqlite3_step() (see SQLITE_STMTSTATUS_REPREPARE), which can change the column count/names; in that case ExtractRowValues() and Object::New(..., num_cols) may use stale num_cols/keys and can lead to a key/value length mismatch (potentially OOB reads in release builds). Consider initializing num_cols/column names only after the first SQLITE_ROW, and/or tracking the reprepare counter during the loop so you can refresh num_cols and rebuild row_keys if the statement was reprepared mid-iteration.

Hmm that makes sense but I feel it wasn't right before neither. Also this doesn't seem possible in a sync api

CC @nodejs/sqlite

codecov · 2026-04-17T18:43:30Z

Codecov Report

❌ Patch coverage is 55.55556% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.67%. Comparing base (5f02bdb) to head (92d55d8).

Files with missing lines	Patch %	Lines
src/node_sqlite.cc	55.55%	2 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #62793      +/-   ##
==========================================
- Coverage   89.69%   89.67%   -0.02%     
==========================================
  Files         706      706              
  Lines      218270   218250      -20     
  Branches    41781    41780       -1     
==========================================
- Hits       195772   195719      -53     
- Misses      14398    14450      +52     
+ Partials     8100     8081      -19

Files with missing lines	Coverage Δ
src/node_sqlite.h	`80.64% <ø> (ø)`
src/node_sqlite.cc	`80.36% <55.55%> (-0.38%)`	⬇️

... and 33 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

araujogui · 2026-04-17T20:10:15Z

Baseline:

sqlite/sqlite-prepare-select-all.js statement="SELECT 1" tableSeedSize=100000 n=100000: 2,335,548.211553957
sqlite/sqlite-prepare-select-all.js statement="SELECT * FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,154,100.8283881845
sqlite/sqlite-prepare-select-all.js statement="SELECT * FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 22,015.504380653096
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 2,110,582.1804559585
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 61,244.744295282806
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,830,653.8297314732
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 48,701.34714174577
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,561,539.287352508
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 39,499.35488804361
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column, blob_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,149,638.9219822034
qsqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column, blob_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 21,783.247235640414
sqlite/sqlite-prepare-select-all.js statement="SELECT text_8kb_column FROM foo_large LIMIT 1" tableSeedSize=100000 n=100000: 981,162.5040978251
sqlite/sqlite-prepare-select-all.js statement="SELECT text_8kb_column FROM foo_large LIMIT 100" tableSeedSize=100000 n=100000: 11,884.22583377536

New:

sqlite/sqlite-prepare-select-all.js statement="SELECT 1" tableSeedSize=100000 n=100000: 2,571,421.224510787
sqlite/sqlite-prepare-select-all.js statement="SELECT * FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,314,946.0050296686
sqlite/sqlite-prepare-select-all.js statement="SELECT * FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 22,273.34966685878
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 2,247,395.145419725
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 61,335.20297916023
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,990,908.1991632094
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 48,737.45422567943
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,812,698.4055051652
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 39,904.82897812861
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column, blob_column FROM foo LIMIT 1" tableSeedSize=100000 n=100000: 1,307,903.332864668
sqlite/sqlite-prepare-select-all.js statement="SELECT text_column, integer_column, real_column, blob_column FROM foo LIMIT 100" tableSeedSize=100000 n=100000: 22,266.04158920139
sqlite/sqlite-prepare-select-all.js statement="SELECT text_8kb_column FROM foo_large LIMIT 1" tableSeedSize=100000 n=100000: 986,878.2204611928
sqlite/sqlite-prepare-select-all.js statement="SELECT text_8kb_column FROM foo_large LIMIT 100" tableSeedSize=100000 n=100000: 11,740.314056697696

thisalihassan

LGTM

thisalihassan · 2026-04-17T20:06:29Z

  static v8::MaybeLocal<v8::Value> All(Environment* env,
-                                       DatabaseSync* db,
-                                       sqlite3_stmt* stmt,
+                                       StatementSync* stmt,


nit: inconsistency in variable naming we are using sync_stmt for StatementSync

thisalihassan · 2026-04-17T20:11:53Z

  static v8::MaybeLocal<v8::Value> Get(Environment* env,
-                                       DatabaseSync* db,
-                                       sqlite3_stmt* stmt,
+                                       StatementSync* stmt,


sqlite: cache column names in StatementSync::All() and Get()

92d55d8

Copilot AI review requested due to automatic review settings April 17, 2026 17:22

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem. labels Apr 17, 2026

Copilot started reviewing on behalf of araujogui April 17, 2026 17:23 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

thisalihassan reviewed Apr 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

sqlite: cache column names in StatementSync::All() and Get()#62793

sqlite: cache column names in StatementSync::All() and Get()#62793
araujogui wants to merge 1 commit intonodejs:mainfrom
araujogui:sqlite-cache-columns

araujogui commented Apr 17, 2026

Uh oh!

nodejs-github-bot commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

araujogui Apr 17, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 17, 2026 •

edited

Loading

Uh oh!

araujogui commented Apr 17, 2026

Uh oh!

thisalihassan left a comment

Uh oh!

thisalihassan Apr 17, 2026 •

edited

Loading

Uh oh!

thisalihassan Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

araujogui commented Apr 17, 2026

Uh oh!

nodejs-github-bot commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

araujogui Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

araujogui commented Apr 17, 2026

Uh oh!

thisalihassan left a comment

Choose a reason for hiding this comment

Uh oh!

thisalihassan Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thisalihassan Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

araujogui Apr 17, 2026 •

edited

Loading

codecov bot commented Apr 17, 2026 •

edited

Loading

thisalihassan Apr 17, 2026 •

edited

Loading