Repository navigation
Compare column types by identity when decoding untyped values - #57
Merged
Merged
Conversation
parse_value runs once per value and picked the decoder with `==` and `<:` on the column's Julia type, each a runtime call; a text value went through about twenty of them. The identity checks now use `===`, a pointer comparison, text columns return first, and the subtype checks only see types no identity check matched. type_info also stops looking up the default registry when the connection's registry has the OID. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Text columns now return before the subtype checks, so no test reached the branches for other registered types; cover them, and the int2[], float4[] and bool[] decoders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Untyped reads (
DBInterface.execute(conn, sql), thenTables.columntableor row iteration) spent most of their decode time choosing a decoder, not decoding.parse_valueruns once per value and compared the column's Julia type withT == Bool,T == Int16, ... andT <: Number. On aTypethat is not known at compile time each==is a runtimejl_types_equalcall and each<:a runtime subtype check. A text value reached its branch after 18 of them. A profile of a million shorttextvalues puts 1,064 of its 2,047 samples outside garbage collection in those comparisons.What changed
parse_valuecompares types with===, which compiles to a pointer comparison (==compiles to a call tojl_types_equal). Text columns return first. The identity checks forbytea, JSON and the array types now come before the two subtype checks, which only see types no identity check matched. The branch each registered type takes is unchanged.type_infolooks up the default registry only when the connection's registry lacks the OID; it used to build the default entry for every value.Typed reads into the field types the manual lists (integers, floats, strings, dates, UUIDs, ...) parse the column text without
parse_valueand are unchanged.Measurements
Seven rounds of fresh processes, each side (main, this branch, two other perf branches, psycopg) in alternating order, PostgreSQL 17.12 in Docker on the same Mac, Julia 1.12.6 with one thread, load average 8 to 59 (median 16). Each process times two reads of 1M rows per case after a full GC and keeps the faster; the table gives the median over rounds, range in brackets. "Mcycles" is the reading thread's CPU cycles.
Tables.columntablePostgres.cursortext, untypeddate, untypeduuid, untypedint4[5], untypedint8, untypedUntyped and cursor reads of the other types are faster or unchanged in cycles, except
timestamptzuntyped and columntable (below).numericandtimestamptzdecode through registered parsers, which return before these checks, andboolis the first check, so those barely change. Typed reads run the same code on both sides; their medians moved by -9% to +5%. The one large wall-time outlier,timestamptzuntyped (median 299 ms on main, 399 ms here, range 290-448), is a full garbage collection that landed inside the timed read in more rounds on this branch: its non-GC time (wall minus GC time) is 243 ms against 248 ms on main (columntable: 257 vs 258), its instructions and allocations are equal, and its cursor read, which keeps no result, is unchanged (627 vs 630 ms, cycles 0.99x).Small queries are unchanged (6 interleaved rounds of 6,000 calls each):
SELECT 1243 vs 267 µs on main and prepared one-row reads 216-224 vs 214 µs, all bound by the round trip; a 100-row untyped read of the 12-column table goes from 590 to 500 µs.Verification
*sendoutput, seed 12, 40 values per type, PostgreSQL 17.12) gives identical results on main and this branch: 2,339 untyped decodes, 9,471 typed reads, and the same known mismatches.int2[],float4[]andbool[]decoders. Once text returned first, no existing test reached them; they pass on main too.===and==agree unless someone registers a different type object equal to one of them (such asVector{T} where Int32 <: T <: Int32).NumericValue,IntervalTypeand the timestamp and array types with parsers return before these checks, as before. Trim: the JuliaC--trim=unsafe-warnbuild oftest/postgres_trim_queries.jl(Julia 1.12.6) reports 0 package verifier errors and the 30 known stdlib ones, as on main.DateTimebranch ofparse_value; whichever merges second needs a one-line conflict resolution.🤖 Generated with Claude Code