fix: Improve jwriter throughput with append-based buffer internals - #51
Merged
Conversation
The internal streamableBuffer now accumulates output in a plain byte slice via append operations instead of bytes.Buffer method calls, string escaping runs as a single-pass scan that appends whole clean segments directly into the buffer, numbers are formatted straight into the buffer with strconv append functions, and the streaming chunk-size check runs once per token instead of once per write. Public API and encoded output are unchanged; streaming chunk boundaries now always fall between tokens.
keelerm84
approved these changes
Aug 6, 2026
kinyoklion
marked this pull request as ready for review
August 6, 2026 16:35
…aming memory Buffer reallocation now at least doubles the capacity, restoring the amortized growth that bytes.Buffer provided; number and raw-value writes reserve space before appending, and in-memory string writes reserve the full encoded length up front. MarshalJSONWithWriter allocates its initial buffer at full size instead of growing a smaller one. Flush now reports a destination write failure even after the undeliverable data has been discarded, records it on the Writer, and detects short writes. Streaming string encoding flushes at chunk boundaries while scanning, so the buffer no longer grows with the length of an escape-heavy string. Grow panics on negative sizes as it did when bytes.Buffer backed it. NewWriter documents that the returned value must not be copied. New tests cover streamed-versus-in-memory output equality across chunk sizes, destination failures at varying points, per-token flushing, the streaming memory bound, and Grow.
…ves, coverage Multi-byte token writes (keywords and raw values, via Write) now reserve capacity like strings and numbers already did, so allocation growth stays amortized for documents dominated by booleans, nulls, or raw values; the single-byte delimiter writes deliberately do not reserve, since a capacity check there measurably slows encoding. Reserving with a size so large that the required capacity overflows now panics instead of silently allocating less than requested. Removed the unused internal Grow method. Documentation: the no-copy warning now also appears on the Writer type and NewStreamingWriter, which documents its panic on negative buffer sizes; buffer comments state precisely which paths reserve and which do not. New tests pin the previously untested invariants: reallocation counts stay logarithmic for every token kind, MarshalJSONWithWriter allocates exactly once for its buffer, a destination that reports a short write yields io.ErrShortWrite from Flush and Error, a failed destination receives no further writes and retains no undelivered data, and Grow panics on negative and overflowing sizes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #28.
SDK-2878
The
jwritercomparative benchmarks show the default implementation losing toencoding/jsonfor arrays of primitives on modern Go —ArrayOfBools~1.4x slower andArrayOfStrings~1.7x slower on Go 1.24 — contradicting the README's performance claim. Nothing regressed in this library:encoding/jsonadopted cached-encoder, append-based encoding across Go 1.19-1.24, whilejwriterroutes every output fragment throughbytes.Buffermethod calls with a chunk-size check on every write. This PR adopts the same append-based technique internally:streamableBufferholds a plain[]byte; writes are inlinable appends instead ofbytes.Buffermethod calls. Reallocation at least doubles the capacity (reserve), keeping growth amortized likebytes.Buffer; bulk writers reserve space before appending.\u00XXescapes are built from a hex-digit table. In-memory writers reserve the encoded length up front; streaming writers instead flush at chunk boundaries mid-scan, so the buffer stays near the chunk size even for long escape-heavy strings (previously such strings were buffered fragment by fragment; an escape-free string is still buffered whole, as before).Int/Float64format straight into the buffer withstrconv.AppendInt/AppendFloat, dropping thetempBytesstaging array.MarshalJSONWithWriterallocates its buffer at full initial size instead of growing a 64-byte one.The encoded output is byte-for-byte unchanged. Error handling is strictly better:
Flushnow reports a destination write failure even when a failed mid-stream flush already emptied the buffer (previously such failures could be visible only viaError(), or with some chunk sizes lost byFlushentirely), records the failure on theWriter, and detects short writes.Benchmarks
go1.24.3, linux/amd64, i9-10885H. Three interleaved before/after rounds (
-count 2each) so CPU frequency drift averages out; theencoding/jsoncomparatives act as a control and are statistically flat between the two runs. The 10k/100k-element cases were run with the same harness to check behavior at payload sizes shaped like whole-environment marshals (ld-relay PUT events).Against
encoding/jsonin the same run:WriteArrayOfStringsis now faster than the stdlib comparative (was 1.7x slower),WriteArrayOfBoolsis at parity (was 1.4x slower), andWriteObjectremains ~2x faster.Allocations:
B/opis equal or lower than base at small and medium sizes (WriteArrayOfStrings-12.6%, 10k elements -11.5%);allocs/opmatches base everywhere except +1 in the streaming benchmark and +1-2 at 10k/100k elements. At 100k elementsB/opis higher than base (14.0 MiB vs 8.0 MiB) because that payload happens to fill base's final power-of-two buffer to ~100% while the doubling ladder here takes one more step; sizes away from that boundary are at or below base.Behavior notes
TestStreamingWriterWritesToTargetInChunksasserts exact flush positions, so its expected boundaries are updated.NewStreamingWriterwith a negativebufferSizepanics, as it did whenbytes.Buffer.Growbacked it.NewWriter().Bytes()on an unused writer now returns an empty non-nil slice rather than nil (the output buffer is preallocated). For the same reason, theWritervalue returned byNewWritermust not be copied; this matchesbytes.Buffersemantics and is now documented.Testing
-race;golangci-lintclean.Error()andFlush()surface the error regardless of where the failure lands; per-token flush behavior at chunk size 1; a streaming memory bound for escape-heavy strings;Growsemantics (content preserved, capacity guaranteed, negative panics). The failure-matrix, per-token, andGrowtests were mutation-checked (deleting the flush check fromInt, no-op'ingreserve, and reverting theFlusherror fix each make the suite fail).Round-2 review changes
Write) now reserve capacity like strings and numbers already did, so allocation growth stays amortized even for documents dominated by booleans, nulls, or raw values. The single-byte delimiter writes deliberately do not reserve — benchmarks showed a capacity check on every delimiter costs ~8% geomean across the suite, while reserving only on token writes is statistically flat (p >= 0.29) and restores near-base reallocation ladders for every token kind.Grow/reserve panic (jwriter: buffer too large) when the required capacity overflows, instead of silently allocating less than requested; previouslyNewStreamingWriter(dest, math.MaxInt)succeeded where base panicked.Writertype andNewStreamingWriter(which also documents its panic on negative sizes); removed the internal deadGrowmethod.MarshalJSONWithWriter, short-write detection (io.ErrShortWrite), no-writes-after-failure and no-retained-data assertions on the failing-destination matrix, and panic assertions onGrow.