Repository navigation
decodebudget: share state across merged singular messages - #1921
Merged
Merged
Conversation
The decoder merges each repeated occurrence of a singular message field into the message it already holds, so that message's slices keep growing. The wire walk started every occurrence with fresh state, so a singular field repeated with one packed value each, e.g. ExponentialHistogramDataPoint.positive, was estimated at ~1/1800 of its quadratic decode allocation and passed the budget. Keep a child messageState per singular message field and decode each occurrence into it, charging the struct once. Switching oneof members discards the previous member, so a member that is set again starts over.
🦋 Changeset detectedLatest commit: 3064268 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
davidzhao
approved these changes
Nov 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1920. It fixes the case Codex reported on #1920: quadratic growth through merged singular messages. That case was not fixed by 0fd0ae4, which covered many packed runs within a single message instance.
Bug
protobuf-go merges each repeated occurrence of a singular message field into the message it already holds, so that message's slices keep growing. The wire walk began every occurrence with fresh state. A singular field repeated with one packed value each, such as
ExponentialHistogramDataPoint.positivewith onebucket_countsvalue per occurrence, was charged as if each slice started empty.On #1920's head, Codex's repro is 200 KB of input estimated at 3.7 MB. That passes a 512 MiB budget, yet decoding it allocates 6.55 GB.
Fix
fieldStatekeeps a childmessageState, and every occurrence decodes into that child. The struct is charged once, on the first occurrence.Tests
TestProtoRejectsQuadraticMergedPackedRunsis Codex's repro, verbatim: 40k mergedpositiveoccurrences.TestProtoRejectsQuadraticMergedMessagesuses 400k occurrences.mainand pass here, rejecting the input in under a millisecond.New calibration probes (estimate ÷ measured allocation):
positiveoccurrences, one bucket eachResourceLogs.resourcerepeated 64k timesAnyValueswitching betweenkvlist_valueandarray_value64k timesAll existing calibration cases are unchanged, at 1.02–1.08.