Own the parallel stream driver - #2
Open
lemire wants to merge 12 commits into
Open
Conversation
The parallel decomposition was carried as an experimental header inside simdjson (PR #2788). It does not need to be there: it is a slicing rule plus a thread pool over the public iterate_many API. Moving it here makes it something a reader can lift into their own pipeline, and lets simdjson carry only the primitives that make it fast. src/parallel_stream.h is that driver. What simdjson keeps is the delimiter-based document skip, which the driver opts into by asking for stream_format::newline_delimited -- exactly the guarantee our slicing rule already relies on. CMake now pins released simdjson and detects newline_delimited rather than requiring it, so the benchmark still builds against a simdjson without the pending change; it just uses whitespace_delimited and forgoes the skip.
run_serial produces every engine=simdjson row, and it was still asking for whitespace_delimited, so it took none of the delimiter skip. Pison cannot parse without its record table, which is the same line structure, and we build and hand it that; withholding it from simdjson is not the neutral choice.
Runtime dispatch picks the widest supported kernel, which is what you want in production and not what you want when asking whether 512-bit code is the right choice at two threads per core.
Workers claimed the next free slice from a shared counter, so a worker's consecutive regions sat threads*slice apart -- tens of megabytes at high thread counts. Giving each worker one contiguous run instead removes the small-slice collapse and is never slower on any dataset we measure. Default slice size is now 64 KiB, the size whose worst case over six datasets is the least bad, and per-worker parser state drops accordingly. Add --assign to select either policy, and record the spread between the slowest and fastest repetition so measurement dispersion is reportable.
The pin 4ee79f7f is a commit on the PR #2803 branch, which is not reachable from master now that the pull request has been merged. A checkout with a cold CPM cache therefore cannot configure at all: CMake Error ... Failed to checkout tag: '4ee79f7f' Our own builds only worked because the commit was already sitting in a warm CPM_SOURCE_CACHE, so the breakage was invisible from here. 9b89b82d is the same change as merged to master, and it resolves from a plain clone. Verified on the reference machine (2x Xeon Gold 6548N) from an empty cache: the build succeeds, every engine-agreement check passes, and the six-dataset matrix reproduces the previous numbers within run-to-run variance. Instructions per byte are unchanged except on Best Buy, where the merged tree retires 3.3% fewer. Claude-Session: https://claude.ai/code/session_011wn2H8y2i839FRVw5Jugoi
The pin 9b89b82d was the commit that merged PR #2803, and it sat 19 commits behind master. The comment above it claimed "a released simdjson suffices", which is not true in the sense that matters: simdjson cuts releases from a 4.6.x branch, so neither stream_format::newline_delimited nor simdjson::slice_at has shipped in one. The newest tag reachable from master is v4.6.1 while the release line is out at v4.6.9, and neither v4.6.1 nor v4.6.9 contains either symbol. A release does still build -- the configure-time probe detects their absence and the driver falls back to its own memchr slicing -- but that is the path every release user takes, not a corner case, and it measures a different thing. Pinning master says so honestly. 3839ac68 is master as of 2026-08-26. It carries 19 commits since the old pin, including #2809, which puts state-based container locking on on-demand object iteration and so could plausibly move the numbers. Verified on the reference machine (2x Xeon Gold 6548N) from a cold pin: the configure resolves from a plain clone, the probe still finds newline_delimited, the build is clean, and all six datasets pass the agreement check with the published match counts unchanged -- twitter 300,270, bestbuy 459,332, google_map 1,716,752, walmart 288,391, wiki 15,603. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
Three loose ends from the driver's own commits. run_parallel defaulted static_partition to false in the header while parallel::options and --assign both default it to true. Nothing hit the disagreement, because main.cpp always passes the flag explicitly, but a caller who omitted it would have measured the other policy. --assign took any value that was not "static" as dynamic, so --assign statc silently changed what was measured. It is now rejected. --engine-only gates the scaling section only; the other sections still run every engine. The help text said "run only this engine", which overstates it. Say what it does and point at --sections scaling, which is what makes the isolation complete. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
The README still described the parallel path as simdjson's experimental parse_many_parallel from PR #2788, which is no longer what the benchmark runs, and still gave --slice-kb's default as 1024. Neither --assign, --engine-only nor --impl was documented at all. Add a section on the driver: the slicing rule, why it lives here rather than in simdjson, the delimiter it requires and why comma-delimited input cannot be sliced, the configure-time detection of newline_delimited and slice_at, and the two knobs with the measurements that set their defaults. State plainly that the defaults changed, so numbers collected before this branch are not comparable to numbers collected after it. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
main gained the OpenAlex authors dataset, a stream-format overhead study, and a serial batch that grows with the longest document. Four files conflicted. The one resolution that is not mechanical is in run_serial. main refactored it into run_serial_format, which takes the format as a parameter so the study can ask for comma_delimited and whitespace_delimited explicitly; this branch had changed run_serial to pass newline_delimited. run_serial_format now keeps the format it is given -- the study's comparison depends on that -- and the newline_delimited choice moves up into run_serial, where the assumption actually holds: that path parses our corpus, which is one document per line. The study calls run_serial_format directly and is unaffected. The rest combine both sides: the options block and usage text list --assign, --engine-only and --impl alongside the format section, and the README keeps main's format and OpenAlex prose alongside the parallel-driver section. OpenAlex is a corpus of bulky records, up to 1.37 MB, which is the first thing in this repository that the 64 KiB slice default does not suit. Correctness is unaffected -- verified on synthetic 1.36 MB documents down to 16 KiB slices, with no lost, duplicated or errored document at any thread count -- but a document much larger than a slice is rescanned once per overlapping slice, and throughput falls accordingly. The README says so and says what to do about it. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
Nine paragraphs and two tables to describe one slicing rule and two knobs was more than the section earns. Three paragraphs: what the driver is and why it lives here, the rule and what it requires of the input, and the knobs with the evidence for their defaults. The figures are also re-measured. The old ones came from the previous pin and from a machine with a core busy, which had moved nspl far enough to invert its sign: it appeared to prefer dynamic by 9%, and on an idle machine against master it prefers static by 13%. Walmart is now the only dataset that prefers dynamic, by 2%, which is inside the spread. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
The parallel-driver section quoted throughput from one machine, which dates the README against every rerun and invites the reader to treat one box's numbers as the result. Say what the knobs do and which way each effect runs; leave the figures to the runs that produce them. Claude-Session: https://claude.ai/code/session_01UdJYgjwyfPStkhb1cc6SnZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The benchmark's parallel path was
simdjson::experimental::parse_many_parallel,which exists only on the PR #2788 branch. That pinned the project to an
unmerged pull request.
We move it to this repository as
src/parallel_stream.h.Defaults
We drop slice size from 1024 KiB to 64 KiB.
Each worker now owns one contiguous range.
Instrumentation
--engine-onlyrestricts the scaling section to one engine.perf stat -acannot attribute a counter to an engine when several run in the same process,
so isolating one is the only way to ask what this engine waits on.
--implforces a simdjson kernel instead of runtime dispatch, for askingwhether 512-bit code is the right choice at two threads per core.
64 KiB is not a universal default
The six Pison datasets have tiny documents; OpenAlex has 232,330 author records averaging 27.6 KB with a largest
record of 1.37 MB, so documents routinely exceed a slice. This means we need larger slices in some cases.
That is not a correctness problem.
slice_atsnaps slice boundaries forward to the next delimiter, so a document larger than a slice is still parsed exactly once, by whichever slice starts at its first byte; it is a throughput problem.slice_atdoes two forwardmemchrs per slice, and when a slice lands inside a document longer than itself, both scans run to that document's end -- so a document of length L is rescanned by roughlyL/sliceslices, costing aboutL^2 / (2 * slice)bytes of scanning.For OpenAlex those brackets suggest something in the 1-2 MiB range rather than
64 KiB.