Skip to content

fix: Improve jwriter throughput with append-based buffer internals - #51

Merged
kinyoklion merged 3 commits into
v4from
rlamb/sdk-2878/append-based-jwriter
Aug 7, 2026
Merged

fix: Improve jwriter throughput with append-based buffer internals#51
kinyoklion merged 3 commits into
v4from
rlamb/sdk-2878/append-based-jwriter

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Aug 6, 2026

Copy link
Copy Markdown
Member

Resolves #28.

SDK-2878

The jwriter comparative benchmarks show the default implementation losing to encoding/json for arrays of primitives on modern Go — ArrayOfBools ~1.4x slower and ArrayOfStrings ~1.7x slower on Go 1.24 — contradicting the README's performance claim. Nothing regressed in this library: encoding/json adopted cached-encoder, append-based encoding across Go 1.19-1.24, while jwriter routes every output fragment through bytes.Buffer method calls with a chunk-size check on every write. This PR adopts the same append-based technique internally:

  • streamableBuffer holds a plain []byte; writes are inlinable appends instead of bytes.Buffer method calls. Reallocation at least doubles the capacity (reserve), keeping growth amortized like bytes.Buffer; bulk writers reserve space before appending.
  • String escaping is a single-pass scan that appends whole clean segments directly into the buffer. Bytes outside the ASCII range pass through without rune decoding, and \u00XX escapes are built from a hex-digit table. In-memory writers reserve the encoded length up front; streaming writers instead flush at chunk boundaries mid-scan, so the buffer stays near the chunk size even for long escape-heavy strings (previously such strings were buffered fragment by fragment; an escape-free string is still buffered whole, as before).
  • Int/Float64 format straight into the buffer with strconv.AppendInt/AppendFloat, dropping the tempBytes staging array.
  • The streaming chunk-size check runs once per token instead of once per write.
  • MarshalJSONWithWriter allocates its buffer at full initial size instead of growing a 64-byte one.

The encoded output is byte-for-byte unchanged. Error handling is strictly better: Flush now reports a destination write failure even when a failed mid-stream flush already emptied the buffer (previously such failures could be visible only via Error(), or with some chunk sizes lost by Flush entirely), records the failure on the Writer, and detects short writes.

Benchmarks

go1.24.3, linux/amd64, i9-10885H. Three interleaved before/after rounds (-count 2 each) so CPU frequency drift averages out; the encoding/json comparatives act as a control and are statistically flat between the two runs. The 10k/100k-element cases were run with the same harness to check behavior at payload sizes shaped like whole-environment marshals (ld-relay PUT events).

                                   │    before     │              after                  │
                                   │    sec/op     │    sec/op     vs base               │
WriteBoolean-16                       69.61n ± 21%   61.27n ± 32%        ~ (p=0.065 n=6)
WriteString-16                        83.60n ±  5%   63.50n ± 16%  -24.04% (p=0.002 n=6)
WriteArrayOfBools-16                  2.413µ ±  5%   1.943µ ± 18%  -19.52% (p=0.002 n=6)
WriteArrayOfStrings-16               10.040µ ±  8%   5.936µ ± 19%  -40.87% (p=0.002 n=6)
WriteObject-16                        273.3n ± 13%   182.9n ± 18%  -33.06% (p=0.002 n=6)
StreamingWriterArrayOfStrings-16     11.418µ ± 10%   8.192µ ± 14%  -28.26% (p=0.002 n=6)
ArrayOfStrings, 10k elements         1437.6µ ±  9%   770.1µ ±  8%  -46.43% (p=0.002 n=6)
ArrayOfStrings, 100k elements        14.325m ±  8%   9.588m ±  7%  -33.07% (p=0.002 n=6)

Against encoding/json in the same run: WriteArrayOfStrings is now faster than the stdlib comparative (was 1.7x slower), WriteArrayOfBools is at parity (was 1.4x slower), and WriteObject remains ~2x faster.

Allocations: B/op is equal or lower than base at small and medium sizes (WriteArrayOfStrings -12.6%, 10k elements -11.5%); allocs/op matches base everywhere except +1 in the streaming benchmark and +1-2 at 10k/100k elements. At 100k elements B/op is higher than base (14.0 MiB vs 8.0 MiB) because that payload happens to fill base's final power-of-two buffer to ~100% while the doubling ladder here takes one more step; sizes away from that boundary are at or below base.

Behavior notes

  • In streaming mode, chunk boundaries now fall between tokens (or at escape boundaries inside long strings) rather than at arbitrary fragment positions. Total output is byte-identical; a clean (escape-free) string segment may overshoot the configured chunk size by its own length, which the buffer always permitted. TestStreamingWriterWritesToTargetInChunks asserts exact flush positions, so its expected boundaries are updated.
  • NewStreamingWriter with a negative bufferSize panics, as it did when bytes.Buffer.Grow backed it.
  • NewWriter().Bytes() on an unused writer now returns an empty non-nil slice rather than nil (the output buffer is preallocated). For the same reason, the Writer value returned by NewWriter must not be copied; this matches bytes.Buffer semantics and is now documented.

Testing

  • Full test suite passes, including with -race; golangci-lint clean.
  • New tests: streamed output equals in-memory output across chunk sizes 0-1000; destination-failure matrix asserting both Error() and Flush() surface the error regardless of where the failure lands; per-token flush behavior at chunk size 1; a streaming memory bound for escape-heavy strings; Grow semantics (content preserved, capacity guaranteed, negative panics). The failure-matrix, per-token, and Grow tests were mutation-checked (deleting the flush check from Int, no-op'ing reserve, and reverting the Flush error fix each make the suite fail).

Round-2 review changes

  • Token-level writes (keywords/raw values via the buffer's Write) now reserve capacity like strings and numbers already did, so allocation growth stays amortized even for documents dominated by booleans, nulls, or raw values. The single-byte delimiter writes deliberately do not reserve — benchmarks showed a capacity check on every delimiter costs ~8% geomean across the suite, while reserving only on token writes is statistically flat (p >= 0.29) and restores near-base reallocation ladders for every token kind.
  • Grow/reserve panic (jwriter: buffer too large) when the required capacity overflows, instead of silently allocating less than requested; previously NewStreamingWriter(dest, math.MaxInt) succeeded where base panicked.
  • The no-copy warning is mirrored on the Writer type and NewStreamingWriter (which also documents its panic on negative sizes); removed the internal dead Grow method.
  • New pinning tests, each mutation-verified: reallocation-ladder bound per token kind, exact-allocation pin on MarshalJSONWithWriter, short-write detection (io.ErrShortWrite), no-writes-after-failure and no-retained-data assertions on the failing-destination matrix, and panic assertions on Grow.

The internal streamableBuffer now accumulates output in a plain byte
slice via append operations instead of bytes.Buffer method calls, string
escaping runs as a single-pass scan that appends whole clean segments
directly into the buffer, numbers are formatted straight into the buffer
with strconv append functions, and the streaming chunk-size check runs
once per token instead of once per write. Public API and encoded output
are unchanged; streaming chunk boundaries now always fall between tokens.
@kinyoklion
kinyoklion marked this pull request as ready for review August 6, 2026 16:35
@kinyoklion
kinyoklion requested a review from a team as a code owner August 6, 2026 16:35
…aming memory

Buffer reallocation now at least doubles the capacity, restoring the
amortized growth that bytes.Buffer provided; number and raw-value writes
reserve space before appending, and in-memory string writes reserve the
full encoded length up front. MarshalJSONWithWriter allocates its initial
buffer at full size instead of growing a smaller one. Flush now reports a
destination write failure even after the undeliverable data has been
discarded, records it on the Writer, and detects short writes. Streaming
string encoding flushes at chunk boundaries while scanning, so the buffer
no longer grows with the length of an escape-heavy string. Grow panics on
negative sizes as it did when bytes.Buffer backed it. NewWriter documents
that the returned value must not be copied.

New tests cover streamed-versus-in-memory output equality across chunk
sizes, destination failures at varying points, per-token flushing, the
streaming memory bound, and Grow.
…ves, coverage

Multi-byte token writes (keywords and raw values, via Write) now reserve
capacity like strings and numbers already did, so allocation growth stays
amortized for documents dominated by booleans, nulls, or raw values; the
single-byte delimiter writes deliberately do not reserve, since a capacity
check there measurably slows encoding. Reserving with a size so large that
the required capacity overflows now panics instead of silently allocating
less than requested. Removed the unused internal Grow method.

Documentation: the no-copy warning now also appears on the Writer type and
NewStreamingWriter, which documents its panic on negative buffer sizes;
buffer comments state precisely which paths reserve and which do not.

New tests pin the previously untested invariants: reallocation counts stay
logarithmic for every token kind, MarshalJSONWithWriter allocates exactly
once for its buffer, a destination that reports a short write yields
io.ErrShortWrite from Flush and Error, a failed destination receives no
further writes and retains no undelivered data, and Grow panics on
negative and overflowing sizes.
@kinyoklion
kinyoklion merged commit ae1830d into v4 Aug 7, 2026
13 checks passed
@kinyoklion
kinyoklion deleted the rlamb/sdk-2878/append-based-jwriter branch August 7, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jwriter performance

2 participants