Conversation
bgentry
force-pushed
the
bg/unique-args-escape-paths
branch
from
September 25, 2026 00:30
24cabfb to
3a2c0a1
Compare
bgentry
force-pushed
the
bg/unique-args-escape-paths
branch
from
September 25, 2026 00:58
3a2c0a1 to
47f0cc1
Compare
bgentry
marked this pull request as ready for review
September 25, 2026 01:03
Fields marked `river:"unique"` are read and written as JSON paths, so a name like `user.id` can omit its value and deduplicate distinct jobs. Unnamed JSON tags also omit their fields instead of using the Go field name. Escape each path component, including leading colons, and fall back to the Go field name for unnamed tags. Keep the original field ordering so unaffected jobs retain their existing unique keys. Cover escaped components, literal and nested path collisions, and unnamed tags with explicit encoding fixtures and cross-driver insertion cases.
bgentry
force-pushed
the
bg/unique-args-escape-paths
branch
from
September 25, 2026 15:29
47f0cc1 to
b5ceebd
Compare
brandur
reviewed
Sep 25, 2026
brandur
left a comment
Contributor
There was a problem hiding this comment.
Wanna ask Claude for a quick benchmark just to make sure that there isn't a serious perf regression on the unique path here?
Comment on lines
+16
to
+17
| - Fixed `UniqueOpts.ByArgs` skipping distinct jobs when `river:"unique"` fields have JSON names containing path syntax (like `user.id` or `alice@example.com`), or an unnamed JSON tag like `json:",omitempty"`. Field names are now addressed literally, and unnamed tags use the Go field name. Unique keys change for affected jobs, so old and new clients may each insert a job with the same args during a rolling upgrade. [PR #1387](https://github.com/riverqueue/river/pull/1387). | ||
| - Fixed `UniqueOpts.ByArgs` skipping distinct jobs or failing inserts when all args participate in uniqueness and an object's keys contain JSON path syntax or are empty. These keys are now included literally, while unique keys for unaffected args remain unchanged. Affected jobs may be inserted again after upgrading or by old and new clients during a rolling upgrade. [PR #1387](https://github.com/riverqueue/river/pull/1387). |
Contributor
There was a problem hiding this comment.
There is a ton of identical verbiage in these two entries. Maybe there's two separate issues in a technical sense, but I don't think the technical correctness is worth the added verbosity. Do you want to condense them down to one?
Contributor
Author
There was a problem hiding this comment.
Thanks for catching, I cleaned this up a lot! 🙏
The all-args unique key builder interprets object keys as JSON paths. Keys containing path syntax can lose their values or collide with other keys, while an empty key causes an insertion error. Walk the object directly and rebuild it in key order with raw values. Preserve the previous key encoding and first-value handling for repeated keys so unaffected hashes remain stable. Continue rejecting scalar and nonempty array args instead of silently hashing them as empty objects. Cover literal keys and duplicate detection across drivers. Pin the key encoding with explicit compatibility fixtures, including Unicode, control characters, and raw nested values, and verify rejection of non-object args.
bgentry
force-pushed
the
bg/unique-args-escape-paths
branch
from
September 26, 2026 00:11
b5ceebd to
a8b2624
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes
UniqueOpts.ByArgsincorrectly skipping distinct jobs when their JSON field names contain dots or other characters that River interprets as path syntax.A dot is allowed in a JSON key. This object has one top-level key named
user.id:{"user.id": "u1"}River's uniqueness code mistakenly reads that key name as a path to an
idfield inside auserobject, as though the args had this different shape:{"user": {"id": "u1"}}The lookup finds nothing in the actual args, so the user ID is left out of the unique key. Jobs for different users then look like duplicates.
An application can explicitly choose the dotted JSON name with a struct tag:
encoding/jsonwrites the first JSON shape above. It does not turn the dot into nesting, and River does not automatically renameUserIDtouser.id. The example uses an unusual but valid JSON name; a conventionaljson:"user_id"tag is unaffected.With
UniqueOpts{ByArgs: true}, these successive insertions now behave correctly. Assume no matching jobs exist initially and the inserted jobs remain available:SyncUserArgs{UserID: "u1"}SyncUserArgs{UserID: "u2"}u1SyncUserArgs{UserID: "u1"}againu1The fix also covers uniqueness based on all args, without
river:"unique"tags. For example,{"file.name":"a.txt"}and{"file.name":"b.txt"}previously collided; now both jobs are inserted. An empty key such as{"":"a"}previously failed withpath cannot be empty; now it inserts normally, and changing its value produces a distinct job. Keys containing other path syntax, such asalice@example.comor a leading:, are also treated literally.A related fix handles unnamed JSON tags, such as this field:
Go uses the JSON key
Recipient, but River previously looked for an empty key, so different nonempty recipients collided. River now uses the Go field name too. An omitted recipient remains omitted.Internally, tagged fields use escaped gjson/sjson path components. When all args participate in uniqueness, River walks the top-level object directly and rebuilds it in key order using its raw values.
Unaffected args keep byte-for-byte identical unique keys. Affected jobs receive corrected keys, so a job inserted before upgrading may no longer deduplicate an identical insertion after upgrading; during a rolling upgrade, old and new clients may each insert it.
A before/after
UniqueKeymicrobenchmark on Go 1.27.1 / Apple M4 Pro found no statistically significant time change for ordinary flat or nested tagged fields, with unchanged allocations. All-args cases with 0, 1, 8, and 64 keys were 25–89% faster and allocated less. An unusual two-field case using$amountandYwas 3.8% slower (about 20 ns), with 32 extra bytes and three extra allocations per call. These measurements use ten alternating before/after samples with the field cache warmed and exclude JSON marshaling and database work.