Skip to content

Read fixed-width protocol fields without allocating - #55

Merged
quinnj-bot[bot] merged 3 commits into
mainfrom
claude/buffered-reads
Nov 19, 2026
Merged

quinnj-bot[bot] merged 3 commits into
mainfrom
claude/buffered-reads

Conversation

@quinnj-bot

@quinnj-bot quinnj-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

Reading a large result spends much of its client CPU in the garbage collector. One of the allocations behind it happens once per protocol message, whatever the query: every message header reads its 4-byte length through Base's generic read(io, Int32), which heap-allocates a Ref because unsafe_read on the connection is not inlined. A 1M-row result is 1M DataRow messages, so 1M of these. For a one-column int4 result read into a Vector{@NamedTuple{v::Int32}} it is 1 of the 6 allocations per row, and a profile of that read puts 266 of its 1,207 samples on it, most of them garbage collection the allocation triggers.

What changed

  • read(::BufferedConn, T) for Int16, UInt16, Int32 and UInt32 loads the value straight from the connection's buffer, refilling it first when fewer bytes are buffered. Every caller after startup (message headers, the process ID in NotificationResponse, and the ParameterDescription and RowDescription counts) uses it.
  • Base.unsafe_read(::BufferedConn, ...) is removed. The fixed-width reads were its only callers, and message bodies go through readbytes!; any other generic IO read on the connection, which the package does not make, falls back to Base's byte-at-a-time method.

Measurements

Seven rounds of fresh processes, main and this branch (and a build with all three open perf PRs) in alternating order, PostgreSQL 17.12 in Docker on the same Mac, Julia 1.12.6 with one thread. Each process times two reads of 1M rows per case after a full GC and keeps the faster; the table gives the median over rounds, range in brackets. "Mcycles" and "Minstr" are the reading thread's CPU cycles and instructions. The machine was busy (load average 10 to 72, median 36), so wall times are noisy; the instruction and allocation counts are not.

1M rows main wall ms branch wall ms main → branch Mcycles main → branch Minstr
int4, typed 112 [97-137] 100 [89-110] 308 → 289 1705 → 1547
int8, typed 141 [128-188] 125 [114-182] 341 → 321 1761 → 1605
bool, typed 83 [77-86] 79 [73-113] 255 → 231 1449 → 1288
int4, Postgres.cursor 513 [480-626] 494 [465-507] 508 → 482 2369 → 2207
int4, copy_to 87 [81-117] 86 [84-99] 137 → 116 646 → 486

Each million-message read allocates 15 MB less (one 16-byte Ref per message) and, where garbage collection does not dominate, runs 146-170 million fewer instructions (every narrow-table typed, cursor and copy_to read). The reading thread's cycles went down in 59 of the 65 case/mode cells. The six that went up are reads that keep a large result (long text, the 12-column table), where whether full collections land inside the timed read varies from round to round on both sides, and the collector runs on the reading thread. A 10-round recheck of those (load average 14-51) gives this branch 0.95x wall and 1.00x cycles for text_long typed, 0.95x and 1.05x for text_long untyped, 0.93x and 0.97x for the 12-column typed read, and 1.12x wall but 1.02x cycles for the 12-column Tables.columntable read, which has 1 collection in some reads and 7-8 in others on main and on this branch alike. For scale, two builds that run the same copy_to code differ by up to ±10% in wall time and ±8% in cycles in the same run.

Small queries are unchanged (6 interleaved rounds of 6,000 calls each): SELECT 1 236 vs 267 µs on main, a prepared one-row typed read 214 vs 214 µs, 49 vs 51 thousand cycles per call. They are bound by the round trip.

Verification

  • New fake-server test: a message whose header arrives one byte per transport read decodes, and a read past the end of the stream still throws EOFError.
  • A differential decode harness (every scalar, array, range and composite type, untyped and typed, checked against the server's binary *send output, seed 12, 40 values per type, PostgreSQL 17.12) gives identical results on main and this branch: 2,339 untyped decodes, 9,471 typed reads, and the same known mismatches.
  • Self-review: the method loads with unsafe_load from pointer(buf, pos) under GC.@preserve buf, after fill_buffer! has made at least sizeof(T) bytes available; fill_buffer! compacts and appends, so the partial bytes stay in order. Byte order is unchanged (callers still apply ntoh). Startup reads, which run on the raw transport before the BufferedConn exists, still use Base's method. Trim: the JuliaC --trim=unsafe-warn build of test/postgres_trim_queries.jl (Julia 1.12.6) reports 0 package verifier errors and the 30 known stdlib ones, as on main.

An earlier version of this branch also copied the rows of a result into shared 64 KiB buffers instead of one buffer per row. That cut allocations further, but untyped reads that keep a large text result then ran 50-100% more garbage collections in Julia 1.12 (more of them full) and got 10-18% slower (1M rows of 200-byte text, untyped and Tables.columntable), so it was dropped.

🤖 Generated with Claude Code

quinnj and others added 2 commits November 19, 2026 11:02
Every message header read its Int32 length through Base's generic
`read(io, Int32)`, which allocates a `Ref`, and every DataRow body was
read into its own `Vector{UInt8}`. Fixed-width reads now load straight
from the BufferedConn buffer, and the rows of one result are copied
into shared 64 KiB chunks. A chunk is never overwritten, so rows a
consumer keeps (a `Vector{Any}` target keeps the DataRow) stay valid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copying the rows of a result into shared chunks cut allocations, but in
untyped reads that keep a large text result Julia 1.12's collector then
ran 50-100% more collections, several of them full, and those reads got
slower. Each DataRow again gets its own buffer; only the allocation-free
fixed-width reads remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@quinnj-bot quinnj-bot Bot changed the title Read protocol messages without a heap allocation per message Read fixed-width protocol fields without allocating Nov 19, 2026
Its only callers were the fixed-width reads, which now load from the
buffer directly; message bodies go through readbytes!.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@quinnj-bot
quinnj-bot Bot marked this pull request as ready for review November 19, 2026 20:45
@quinnj-bot
quinnj-bot Bot merged commit cf4aa8d into main Nov 19, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant