repr/strconv: accept full u32 range when parsing oid from text#37440
Merged
Conversation
petrosagg
reviewed
Jul 6, 2026
| /// contract in `mz_storage_types::sources::casts`). Changing its behavior would | ||
| /// break retraction symmetry: a row ingested under the old behavior as a cast | ||
| /// error could later be retracted as a value. Use [`parse_oid`] everywhere else. | ||
| pub fn parse_oid_legacy(s: &str) -> Result<u32, ParseError> { |
Contributor
There was a problem hiding this comment.
The argument for keeping around error behavior is so that if a source ingests an error that is later retracted the error goes away. However in practice users don't keep around errored sources, they recreate them and if the error persists they file a support request since they have data that can't be ingested. Combined with the fact that ingesting oid is pretty rare in and of itself I think we could just delete the legacy version
Contributor
Author
There was a problem hiding this comment.
Perfect, that makes this much simpler.
Contributor
Author
There was a problem hiding this comment.
QA LLM review noticed after merging that this was wrong after all: #37458
`parse_oid` parsed OIDs as `i32` and reinterpreted them as `u32`, which rejected valid OIDs in `2147483648..=4294967295`. For example `'4294967295'::oid` errored even though `4294967295::oid` succeeded, and `COPY` into such a column failed with "invalid input syntax for type oid". PostgreSQL accepts the union of the `u32` and `i32` ranges (negatives are reinterpreted), so parse the full `u32` range directly and fall back to `i32` for negative inputs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
petrosagg
approved these changes
Jul 6, 2026
def-
added a commit
to def-/materialize
that referenced
this pull request
Jul 6, 2026
Follow-up to MaterializeInc#37440 Sorry for only realizing so late that it's an issue after all! `parse_oid` was widened to the full `u32` range (MaterializeInc#37440), but the same helper backs `CastStringToOid` in the PostgreSQL source, whose eval behavior must stay stable across releases. Replication re-casts the old tuple on delete, so a value ingested pre-upgrade as a `CastError` would be retracted post-upgrade as a value. That leaves the error stuck and adds a phantom negative row. Keep the storage cast on a frozen `parse_oid_legacy` that accepts only the `i32` range. The SQL, COPY, and pgwire paths keep the widened `parse_oid`. Add a unit snapshot pinning the frozen behavior and a platform-check reproducing the cross-upgrade retraction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
def-
added a commit
to def-/materialize
that referenced
this pull request
Jul 6, 2026
Freezing CastStringToOid on the legacy i32 range fixed retraction symmetry for existing sources, but also made fresh PostgreSQL sources reject valid oid values above i32::MAX, undoing the source-side part of the widening from MaterializeInc#37440 for all future sources. Version the cast per export instead. Newly purified exports persist cast_oid_full_range in their statement DETAILS and plan the new CastStringToOidFullRange, which accepts the full u32 range via the widened parse_oid. Exports whose details predate the flag decode as false and keep the legacy i32-range cast, so their persisted rows keep retracting cleanly. The cast choice derives from the persisted details rather than the binary version, which keeps replanning stable across restarts and upgrades. Add unit coverage for the full-range variant (error snapshot and parity with the SQL cast) and a pg-cdc testdrive file exercising snapshot, replication, and retraction of above-i32::MAX oids in scalar and array columns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
parse_oidparsed OIDs asi32and reinterpreted them asu32, which rejected valid OIDs in2147483648..=4294967295. For example'4294967295'::oiderrored even though4294967295::oidsucceeded, andCOPYinto such a column failed with "invalid input syntax for type oid".PostgreSQL accepts the union of the
u32andi32ranges (negatives are reinterpreted), so parse the fullu32range directly and fall back toi32for negative inputs.Found via #34779