feat: add mergeable containers (#759)#991
Conversation
…helpers
Introduce the shared encoding primitives that let map keys host
deterministic "mergeable" child containers. Both peers derive the same
`ContainerID::Root` for a given `(parent, key, kind)` triple, so a child
created independently on two sites resolves to one container that merges
rather than two that fork.
- `MERGEABLE_NAMESPACE_PREFIX` ("🤝:"): reserved root-name prefix mirroring
Loro's existing 🦜 brand sentinel. `check_root_container_name` rejects
user-supplied root names that collide with it.
- `ContainerID::new_mergeable` / `is_mergeable` / `parse_mergeable`: encode and
decode the hex `(parent, key, container_type)` payload, with length-prefixed
segments so arbitrary keys (empty, NUL, embedded prefix) round-trip cleanly.
- Discriminator helpers (`mergeable_discriminator`, `*_string`,
`parse_mergeable_discriminator`): the `"🤝:<kind>"` value the parent map
stores at the key to mark which child kind is active.
The cid-encoding integration tests (`mergeable_cid_encoding.rs`) live with
this commit since they exercise only the common-crate surface.
6e571b0 to
8be8ca7
Compare
Add the core state machinery for mergeable child containers. A mergeable child lives as a deterministic `ContainerID::Root` in the reserved namespace; its visibility is driven entirely by the `"🤝:<kind>"` discriminator the parent map stores at the key. The parent map's ordinary LWW picks the active discriminator, which in turn selects the active child kind — exactly as a regular child container's value-table entry selects its child. This makes deletes slot-authoritative: clearing the discriminator removes the child, and concurrent type conflicts resolve by the same LWW that governs the map slot, with no separate tombstone bookkeeping. - `state/mergeable.rs`: parent-edge index (`child_containers`) kept in sync with discriminators — seeded on snapshot and update import, updated per-op as discriminators appear or clear. Path resolution and reachability map a mergeable cid back to its key through this index. - `map_state.rs`: store per-key discriminators, expose `active_mergeable_children` / `iter_mergeable_children`, and resolve the active child kind from the parent slot. - `arena.rs` / `state.rs`: parent mergeable roots into arena walks and route mergeable map children to their deterministic root ids so reachability, child enumeration, and alive-container walks include them. - `dead_containers_cache.rs`: treat a mergeable child as dead once its parent edge (discriminator) is gone, mirroring a regular container whose value slot was overwritten.
Expose mergeable child containers on `MapHandler` via
`get_mergeable_{counter,map,list,movable_list,text,tree}`. Each accessor
writes the `"🤝:<kind>"` discriminator into the parent map at the key and
returns a handler bound to the deterministic mergeable cid, so two peers
calling the same accessor with the same key converge on one container.
Requesting a different kind under an existing key is a deliberate kind change
that overwrites the discriminator; the previous kind's container stays
reachable by its deterministic name. Detached handlers fall back to the
ordinary get-or-create path.
Wire the supporting plumbing through `loro.rs` so the discriminator op and
the derived child route through the normal commit/event path.
The internal integration suite (`tests/mergeable_container/`, split into
focused modules sharing a `common` helper) lands here since it drives the
handler API: concurrent create/convergence, discriminator-based type
conflict resolution, slot-authoritative delete and recreate, snapshot/update
import round-trips, and event/path resolution.
Add `get_mergeable_{counter,map,list,movable_list,text,tree}` to the public
`LoroMap`, wrapping the internal handler accessors and returning the public
container types (`LoroCounter`, `LoroMap`, `LoroList`, `LoroMovableList`,
`LoroText`, `LoroTree`). These are purely additive, non-breaking additions to
the public surface.
The public-API test target (`crates/loro/tests/mergeable_public_api.rs`)
confirms each wrapper forwards to the handler and the returned container is
usable end-to-end, including cross-peer convergence, the kind-change-by-
overwrite semantics, and the `has_container` carve-out for mergeable cids.
Bind `getMergeable{Counter,Map,List,MovableList,Text,Tree}` on the WASM
`LoroMap`, delegating to the internal handler accessors. Each returns the
corresponding JS container handle bound to the deterministic mergeable cid,
so independently created children on two peers merge instead of forking.
These methods emit a discriminator `MapSet` against the parent map, which
flows through the same auto-commit barrier as `LoroMap.set`; the events are
flushed by the already-decorated `commit`, so no `index.ts` allowlist change
is needed. The WASM test (`tests/mergeable.test.ts`) covers convergence for
every kind plus the subscription-flush invariant from AGENTS.md (no
`[LORO_INTERNAL_ERROR] Event not called` under an active subscription).
Extend the fuzz harness to exercise mergeable children alongside regular containers, behind an opt-in `mergeable` feature so the default fuzz runs are unaffected. Adds a `mergeable` fuzz target plus per-container action support (`MapAction::GetMergeable` and friends) so the existing convergence/replay fuzzers can create and mutate mergeable counters, maps, lists, movable lists, text, and trees. The crate compiles with and without the feature.
8be8ca7 to
e3927de
Compare
|
Fuzzed via |
P1: Invalidate deleted-cache entries when a mergeable child becomes active againA mergeable child can become reachable again after it was previously deleted. Example flow:
When P1: Clarify or restrict
|
|
Snapshot import does not need to scan/decode every Map to rebuild mergeable parent edges. For mergeable children that have state entries, the mergeable root cid already encodes The current implementation instead walks all Map containers and reads every Map's shallow value. That is much broader than necessary. One subtle case is unmutated mergeable children: they may only exist as a discriminator in the parent map and may not have a child state entry yet. If we need those to have side-table entries immediately after snapshot import, then scanning discriminators is one way to find them. But that could also be handled lazily when the child is accessed or when path/reachability is queried. So I think the import-time recovery should avoid
|
|
It's an awesome PR overall. I can push the fixes that address these comments directly to this branch if you want |
Adds mergeable child containers, which are child containers that you can create under map keys that merge across peers instead of creating two separate containers. Closes #759.
They're available as
getMergeable{Counter,Map,List,MovableList,Text,Tree}onLoroMap, in theloro,loro-internal, and WASM crates.Problem
Right now, if two peers each create a child container at the same map key, they end up with two different containers, because the
ContainerIDis derived from the creating op's id. One of them wins and the other peer's edits are stranded on a container nobody looks at. That's a problem any time you want a single shared object regardless of who created it first - a revision counter, a settings sub-map, a shared text body, etc.Approach
A mergeable child is a
ContainerID::Rootwith a deterministic name derived from(parent, key, kind), in a reserved"🤝:"namespace. Both peers compute the same id, so they're talking about the same container.Which child is "live" is decided by the parent map slot, not by separate bookkeeping. When you call
getMergeable*, the parent map stores a"🤝:<kind>"discriminator string at that key, and the active child is just whatever that key's normal LWW resolves to. Letting the existing map machinery own this gets us a few things mostly for free:MapSet { value: None }). No new op types, and no tombstone tracking to keep in sync.Commits
I split this into 6 commits by layer, each one carrying its own tests:
feat(common)- cid namespace + discriminator encode/parse helpersfeat(internal)- core mergeable state inDocStatefeat(internal)-get_mergeable_*handler APIfeat(loro)- publicLoroMapwrappersfeat(wasm)-getMergeable*bindingstest(fuzz)- opt-inmergeablefuzz surfaceThere's also a trailing
chore:commit with the changeset.Compatibility
This is all new methods - no existing signatures change, so it shouldn't break anyone. Root names that start with the
"🤝:"prefix are rejected bycheck_root_container_nameso user code can't fabricate one.Testing
57 tests across the internal (43), public-API (8), and cid-encoding (6) targets, plus a WASM suite in
mergeable.test.tsthat covers each kind and the subscription-flush invariant from AGENTS.md. There's also an opt-in fuzz target (cargo +nightly fuzz run mergeable); the fuzz crate builds with and without themergeablefeature.Between them they cover concurrent same- and different-kind creation, three-peer convergence, delete + recreate, snapshot/update import round-trips, event and path resolution, and
has_containerfor mergeable cids.AI usage disclosure
Most of the lines in this PR were written by an AI coding assistant (Claude Opus 4.7, primarily). I want to be upfront about that, and equally upfront that the engineering wasn't outsourced. I read every diff and handled every non-trivial engineering decision myself, although I did regularly seek the advice of the agent during the process.
So: the assistant did most of the typing, I did the deciding and the reviewing. I've reviewed the whole change and I stand behind its correctness; treat it as my work and review it as critically as you would anything else.