Skip to content

Stop boxing the row and iteration state in load!'s insert loop - #392

Merged
quinnj-bot[bot] merged 1 commit into
masterfrom
claude/load-row-loop
Nov 19, 2026
Merged

quinnj-bot[bot] merged 1 commit into
masterfrom
claude/load-row-loop

Conversation

@quinnj-bot

@quinnj-bot quinnj-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

SQLite.load! spent as much time on its own loop as SQLite spent inserting, for narrow tables. The insert loop ran inside the transaction closure and reassigned row and st there, so Julia boxed both. Every row then allocated, and every iterate, eachcolumn, and bind call dispatched at run time.

What changed

  • The loop moved into insertrows!. The closure passes it the first row and iteration state, so they are ordinary typed locals.

Scope

  • load! only. Its behavior, errors, and transaction handling don't change.
  • executemany goes through DBInterface and doesn't change.

Blast Radius

Every load! call. The loop body is the same code. The existing load! tests cover the known-schema, unknown-schema, empty, append, and conflict paths.

Verification

  • load! of 1M rows of one integer column into an in-memory database went from 379 ms to 199 ms, and its allocations from 153 MB to 15 MB. A mixed 10-column table went from 658 ms to 547 ms (2237 to 1851 million cycles). Each number is the median of 7 fresh-process runs per side, interleaved with master, on a shared 14-core Apple silicon machine (load average 10 to 50).
  • None of the other 34 read, write, and latency cases is slower at rank-sum p < 0.01.
  • Pkg.test() passes on Julia 1.12 and 1.10. Lowered code of the load! keyword body no longer contains Core.Box.

🤖 Generated with Claude Code

`load!` ran its insert loop inside the `DBInterface.transaction` closure
and reassigned `row` and `st` there. Julia stores captured variables that
a closure reassigns in a `Core.Box`, so every row was boxed and every
`iterate`, `eachcolumn`, and bind call in the loop dispatched at run time.
Loading 1M rows of one integer column took about 370 ms and allocated
150 MB.

The loop now lives in `insertrows!`, which the closure calls with the
first row and state as arguments, so they are ordinary typed locals.
Behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.31%. Comparing base (511b83a) to head (60cb6c2).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #392   +/-   ##
=======================================
  Coverage   97.30%   97.31%           
=======================================
  Files           5        5           
  Lines         854      857    +3     
=======================================
+ Hits          831      834    +3     
  Misses         23       23           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@quinnj-bot
quinnj-bot Bot merged commit b019812 into master Nov 19, 2026
9 checks passed
@quinnj-bot
quinnj-bot Bot deleted the claude/load-row-loop branch November 19, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant