Repository navigation
Read fixed-width protocol fields without allocating - #55
Merged
Merged
Conversation
Every message header read its Int32 length through Base's generic
`read(io, Int32)`, which allocates a `Ref`, and every DataRow body was
read into its own `Vector{UInt8}`. Fixed-width reads now load straight
from the BufferedConn buffer, and the rows of one result are copied
into shared 64 KiB chunks. A chunk is never overwritten, so rows a
consumer keeps (a `Vector{Any}` target keeps the DataRow) stay valid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copying the rows of a result into shared chunks cut allocations, but in untyped reads that keep a large text result Julia 1.12's collector then ran 50-100% more collections, several of them full, and those reads got slower. Each DataRow again gets its own buffer; only the allocation-free fixed-width reads remain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its only callers were the fixed-width reads, which now load from the buffer directly; message bodies go through readbytes!. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Reading a large result spends much of its client CPU in the garbage collector. One of the allocations behind it happens once per protocol message, whatever the query: every message header reads its 4-byte length through Base's generic
read(io, Int32), which heap-allocates aRefbecauseunsafe_readon the connection is not inlined. A 1M-row result is 1MDataRowmessages, so 1M of these. For a one-columnint4result read into aVector{@NamedTuple{v::Int32}}it is 1 of the 6 allocations per row, and a profile of that read puts 266 of its 1,207 samples on it, most of them garbage collection the allocation triggers.What changed
read(::BufferedConn, T)forInt16,UInt16,Int32andUInt32loads the value straight from the connection's buffer, refilling it first when fewer bytes are buffered. Every caller after startup (message headers, the process ID inNotificationResponse, and theParameterDescriptionandRowDescriptioncounts) uses it.Base.unsafe_read(::BufferedConn, ...)is removed. The fixed-width reads were its only callers, and message bodies go throughreadbytes!; any other genericIOread on the connection, which the package does not make, falls back to Base's byte-at-a-time method.Measurements
Seven rounds of fresh processes, main and this branch (and a build with all three open perf PRs) in alternating order, PostgreSQL 17.12 in Docker on the same Mac, Julia 1.12.6 with one thread. Each process times two reads of 1M rows per case after a full GC and keeps the faster; the table gives the median over rounds, range in brackets. "Mcycles" and "Minstr" are the reading thread's CPU cycles and instructions. The machine was busy (load average 10 to 72, median 36), so wall times are noisy; the instruction and allocation counts are not.
int4, typedint8, typedbool, typedint4,Postgres.cursorint4,copy_toEach million-message read allocates 15 MB less (one 16-byte
Refper message) and, where garbage collection does not dominate, runs 146-170 million fewer instructions (every narrow-table typed, cursor andcopy_toread). The reading thread's cycles went down in 59 of the 65 case/mode cells. The six that went up are reads that keep a large result (long text, the 12-column table), where whether full collections land inside the timed read varies from round to round on both sides, and the collector runs on the reading thread. A 10-round recheck of those (load average 14-51) gives this branch 0.95x wall and 1.00x cycles fortext_longtyped, 0.95x and 1.05x fortext_longuntyped, 0.93x and 0.97x for the 12-column typed read, and 1.12x wall but 1.02x cycles for the 12-columnTables.columntableread, which has 1 collection in some reads and 7-8 in others on main and on this branch alike. For scale, two builds that run the samecopy_tocode differ by up to ±10% in wall time and ±8% in cycles in the same run.Small queries are unchanged (6 interleaved rounds of 6,000 calls each):
SELECT 1236 vs 267 µs on main, a prepared one-row typed read 214 vs 214 µs, 49 vs 51 thousand cycles per call. They are bound by the round trip.Verification
EOFError.*sendoutput, seed 12, 40 values per type, PostgreSQL 17.12) gives identical results on main and this branch: 2,339 untyped decodes, 9,471 typed reads, and the same known mismatches.unsafe_loadfrompointer(buf, pos)underGC.@preserve buf, afterfill_buffer!has made at leastsizeof(T)bytes available;fill_buffer!compacts and appends, so the partial bytes stay in order. Byte order is unchanged (callers still applyntoh). Startup reads, which run on the raw transport before theBufferedConnexists, still use Base's method. Trim: the JuliaC--trim=unsafe-warnbuild oftest/postgres_trim_queries.jl(Julia 1.12.6) reports 0 package verifier errors and the 30 known stdlib ones, as on main.An earlier version of this branch also copied the rows of a result into shared 64 KiB buffers instead of one buffer per row. That cut allocations further, but untyped reads that keep a large text result then ran 50-100% more garbage collections in Julia 1.12 (more of them full) and got 10-18% slower (1M rows of 200-byte text, untyped and
Tables.columntable), so it was dropped.🤖 Generated with Claude Code