Skip to content

refactor(cpp): ♻️ extract the row hash table from OperatorIndex - #303

Open
robertodr wants to merge 2 commits into
mainfrom
split/02-row-hash-table
Open

robertodr wants to merge 2 commits into
mainfrom
split/02-row-hash-table

Conversation

@robertodr

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Stacked on #302 — review that first; this PR's diff against it is RowHashTable.h plus the OperatorIndex rewrite.

Summary

Second of five PRs carved out of #226. OperatorIndex carries two things that have nothing to do with each other: a keyless open-addressing hash table, and the packed row storage the table indexes into. A second row backend is coming, and it needs the table and not the rows, so the table moves out first, on its own, with no behaviour change.

RowHashTable is the table alone — power-of-two slots, linear probing, load factor 0.7, a 32-bit folded hash kept beside each slot as an equality pre-filter. It stores no keys: hashing, equality and prefetch arrive as callables from the owner, which is what lets a caller key its rows in whatever form it already has them in. OperatorIndex keeps its own API and forwards to it.

This is deliberately inert. The table's slot layout fixes for_each_slot order, which fixes evolved-term order, which fixes floating-point accumulation order — so a reordering here would move energies. Verified against main's implementation over 4000 random monomials: identical iteration order, find / find_batch results, clone order and memory_bytes.

Changes

  • cpp/monoprop/detail/operator/RowHashTable.h: new. The table, TermIndexCeilingReached, kIndexCeiling, fold, find / find_batch / emplace / insert_distinct / insert_distinct_range, for_each_slot, reserve, slot_bytes.
  • cpp/monoprop/detail/operator/OperatorIndex.h: keeps the rows, delegates the table. -195 lines.
  • cpp/monoprop/detail/operator/CMakeLists.txt: list the new header.

Checklist

  • Tests added or updated to cover the changes
  • Documentation updated (docstrings, docs/, CONTRIBUTING.md) if needed
  • CHANGELOG / release notes updated if applicable

Covered by the existing operator_index_tests.cpp; the gate for a change of this kind is just diff-baseline (#302), which must come out byte-identical.

AI/LLM disclosure

  • I used the following tool to help write this PR description: ClaudeCode (claude-opus-5)
  • I used the following tool to generate or modify code: ClaudeCode (claude-opus-5)

@github-actions

Copy link
Copy Markdown

Docs preview: https://pr-303.monoprop-docs.pages.dev

@robertodr
robertodr force-pushed the split/02-row-hash-table branch from 403debb to 8ded3bd Compare August 29, 2026 14:35
@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.70%. Comparing base (c5e88c8) to head (3c0d67e).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #303   +/-   ##
=======================================
  Coverage   97.70%   97.70%           
=======================================
  Files          14       14           
  Lines         742      742           
  Branches       98       98           
=======================================
  Hits          725      725           
  Misses         12       12           
  Partials        5        5           
Flag Coverage Δ
cpp 97.70% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@robertodr
robertodr force-pushed the split/02-row-hash-table branch from 8ded3bd to e4b1217 Compare August 29, 2026 14:59
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from e4b1217 to 8ba993f Compare August 31, 2026 08:57
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from 8ba993f to 1b55f23 Compare August 31, 2026 12:09
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from 1b55f23 to fe0c971 Compare September 1, 2026 13:20
@robertodr
robertodr force-pushed the split/02-row-hash-table branch 2 times, most recently from fab96fb to 99a4c81 Compare September 3, 2026 08:04
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from 99a4c81 to e271733 Compare September 3, 2026 11:37
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from e271733 to b9af47b Compare September 3, 2026 13:48
Base automatically changed from split/01-baseline-tool to main September 3, 2026 13:50
@robertodr
robertodr force-pushed the split/02-row-hash-table branch 2 times, most recently from 1a0b8be to d99049a Compare September 8, 2026 05:31
@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 74eb9f04-b1fe-4006-9662-5761fa182c50

📥 Commits

Reviewing files that changed from the base of the PR and between c5e88c8 and 3c0d67e.

📒 Files selected for processing (3)
  • cpp/monoprop/detail/operator/CMakeLists.txt
  • cpp/monoprop/detail/operator/OperatorIndex.h
  • cpp/monoprop/detail/operator/RowHashTable.h

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ludmilaasb ludmilaasb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT

Just remove the narrative comments and I think we can merge.

Comment on lines +34 to +37
// Operator-term store: entropy-packed position-list rows plus a RowHashTable keyed over them. The rows
// are this class's business and the index is not: nothing below reads a slot, and nothing in
// RowHashTable reads a row -- the two meet only through the hash and equality callables passed in.
// Row layout: slot 0 = popcount c (or kOverflowMarker if c > inline_width_), slots 1..c =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this

Suggested change
// Operator-term store: entropy-packed position-list rows plus a RowHashTable keyed over them. The rows
// are this class's business and the index is not: nothing below reads a slot, and nothing in
// RowHashTable reads a row -- the two meet only through the hash and equality callables passed in.
// Row layout: slot 0 = popcount c (or kOverflowMarker if c > inline_width_), slots 1..c =

Comment on lines +38 to +51
// The keyless open-addressing index a row store puts over its rows: power-of-2 slot count, linear
// probing, max load factor 0.7 (the group-prefetch win erodes at higher load -- longer probe chains add
// un-prefetched reads). A slot holds a row index plus a 32-bit hash used only as an equality
// pre-filter, so the table never stores or compares a key itself.
//
// Keyless is why the hash and the equality test arrive as callables rather than as members: the whole
// point is that the caller owns the row representation. `eq(row_index)` confirms a pre-filter hit
// against the caller's rows, and no operation here reads a row.
//
// The layout this produces is load-bearing, not an implementation detail: it fixes the iteration order
// of for_each_slot(), which sets the order of a propagator's user-visible evolved-term list and
// therefore its floating-point accumulation order. A store that keyed rows through its own copy of this
// logic could diverge on that while still looking correct, which is the reason the index lives apart
// from the row representation rather than inside one.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// The keyless open-addressing index a row store puts over its rows: power-of-2 slot count, linear
// probing, max load factor 0.7 (the group-prefetch win erodes at higher load -- longer probe chains add
// un-prefetched reads). A slot holds a row index plus a 32-bit hash used only as an equality
// pre-filter, so the table never stores or compares a key itself.
//
// Keyless is why the hash and the equality test arrive as callables rather than as members: the whole
// point is that the caller owns the row representation. `eq(row_index)` confirms a pre-filter hit
// against the caller's rows, and no operation here reads a row.
//
// The layout this produces is load-bearing, not an implementation detail: it fixes the iteration order
// of for_each_slot(), which sets the order of a propagator's user-visible evolved-term list and
// therefore its floating-point accumulation order. A store that keyed rows through its own copy of this
// logic could diverge on that while still looking correct, which is the reason the index lives apart
// from the row representation rather than inside one.

Comment on lines +78 to +80
// Refused before anything grows: an append that would pass the ceiling cannot be unwound, and a
// store's size must never reach a count whose last index is unrepresentable. Written as a
// subtraction because base + n is the sum that would wrap.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Refused before anything grows: an append that would pass the ceiling cannot be unwound, and a
// store's size must never reach a count whose last index is unrepresentable. Written as a
// subtraction because base + n is the sum that would wrap.

Comment on lines +132 to +135
// Insert with no duplicate probe -- callers on this path insert provably distinct keys
// (+G-injective miss batches, clone re-insertion).
// insert_distinct over consecutive row indices [base, base + n), hashing each through hash_at(k).
// The stores' bulk_insert is this and nothing else, so it lives here rather than once per backend.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's confusing, remove or change

Suggested change
// Insert with no duplicate probe -- callers on this path insert provably distinct keys
// (+G-injective miss batches, clone re-insertion).
// insert_distinct over consecutive row indices [base, base + n), hashing each through hash_at(k).
// The stores' bulk_insert is this and nothing else, so it lives here rather than once per backend.

Comment on lines +168 to +175
// Group-prefetch batch find: out[i] = row index of keys[i], or kNotFound. Same result as n find()
// calls, but overlaps dram misses via a per-group hash/probe/confirm pipeline. An h collision falls
// back to an exact find. Must not run concurrently with inserts.
//
// The three callables are what make the pipeline possible without the table knowing a row:
// hash(key) -> uint32_t, prefetch_row(row_index) issued between probe and confirm (which is the
// whole reason confirmation is deferred rather than folded into the probe), and
// eq(row_index, key) -> bool.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Group-prefetch batch find: out[i] = row index of keys[i], or kNotFound. Same result as n find()
// calls, but overlaps dram misses via a per-group hash/probe/confirm pipeline. An h collision falls
// back to an exact find. Must not run concurrently with inserts.
//
// The three callables are what make the pipeline possible without the table knowing a row:
// hash(key) -> uint32_t, prefetch_row(row_index) issued between probe and confirm (which is the
// whole reason confirmation is deferred rather than folded into the probe), and
// eq(row_index, key) -> bool.

Comment on lines +182 to +186
// find_batch over keys reached through key_at(i) rather than an array: a query form that is not an
// array of keys (a flattened position list, say) keeps this one pipeline instead of copying it.
// key_at is called once per key in the hash pass and again in the confirm pass, so it must be cheap
// and stable. When hash_out is non-empty it receives every key's folded hash, which is what an
// insert of the misses hands back to insert_distinct_range.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove or simplify

Suggested change
// find_batch over keys reached through key_at(i) rather than an array: a query form that is not an
// array of keys (a flattened position list, say) keeps this one pipeline instead of copying it.
// key_at is called once per key in the hash pass and again in the confirm pass, so it must be cheap
// and stable. When hash_out is non-empty it receives every key's folded hash, which is what an
// insert of the misses hands back to insert_distinct_range.

Comment on lines +269 to +273
// First slot on h's probe chain whose stored hash matches, or kEmptySlot if the chain ends first.
// Matches on h alone and leaves confirmation to the caller -- that deferral is what lets find_batch
// prefetch the row between probe and confirm, so do not fold eq in here (find() deliberately keeps
// its own confirming variant). `start` must already be masked; the table must not be mutated
// concurrently.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// First slot on h's probe chain whose stored hash matches, or kEmptySlot if the chain ends first.
// Matches on h alone and leaves confirmation to the caller -- that deferral is what lets find_batch
// prefetch the row between probe and confirm, so do not fold eq in here (find() deliberately keeps
// its own confirming variant). `start` must already be masked; the table must not be mutated
// concurrently.

Added contribution guidelines including CLA information and contact
details for Corporate CLA.

## Checklist

- [ ] Tests added or updated to cover the changes
- [X] Documentation updated (docstrings, `docs/`, `CONTRIBUTING.md`) if
needed
- [ ] `CHANGELOG` / release notes updated if applicable

## AI/LLM disclosure

- [ ] I did not use LLM tooling, or used it only privately for ideation
- [ ] I used the following tool to help write this PR description:
- [ ] I used the following tool to generate or modify code:

<!-- Any code generated or substantially modified by an LLM must be
noted inline too. -->

> [!IMPORTANT]
> By opening this PR I confirm that I have read
[CONTRIBUTING.md](https://github.com/Algorithmiq/monoprop/blob/fa820c3e5a90f773417c367fedbf2bd50683b496/CONTRIBUTING.md)
and I agree to the terms of the [Contributor License
Agreement](https://github.com/Algorithmiq/monoprop/blob/fa820c3e5a90f773417c367fedbf2bd50683b496/CLA.md).

> [!WARNING]
> If you're contributing on behalf of your employer, contact
[cla@algorithmiq.fi](mailto:cla@algorithmiq.fi) to arrange a Corporate
CLA.

Signed-off-by: Roberto Di Remigio Eikås <robertodr@users.noreply.github.com>
OperatorIndex was two things: a packed position-list row representation and a
keyless open-addressing index over those rows. RowHashTable is the second half,
lifted out whole -- power-of-2 slots, linear probing, load factor 0.7, a 32-bit
folded hash per slot used only as an equality pre-filter.

Keyless is what makes the split possible: the table never stores or compares a
key, so the hash and the equality test arrive as callables and the row
representation stays entirely on the caller's side. find_batch keeps its
pipeline by taking the row prefetch as a third callable -- deferring
confirmation past the probe is the whole reason that prefetch has somewhere to
go.

The table's slot layout fixes for_each_slot's iteration order, which is the
order of a propagator's evolved-term list and therefore its floating-point
accumulation order, so this had to come out inert. Checked byte-wise against
the previous implementation over 4000 random monomials: identical iteration
order, find and find_batch results, clone order and memory_bytes.

Assisted-by: ClaudeCode:claude-opus-5
@robertodr
robertodr force-pushed the split/02-row-hash-table branch from d99049a to 3c0d67e Compare September 9, 2026 15:04
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants