Skip to content

Feat/optimize rpc#718

Draft
konradkonrad wants to merge 9 commits into
mainfrom
feat/optimize_rpc
Draft

Feat/optimize rpc#718
konradkonrad wants to merge 9 commits into
mainfrom
feat/optimize_rpc

Conversation

@konradkonrad

Copy link
Copy Markdown
Contributor

Select optimizations to reduce RPC call count, based on analysis from #717:

S2 optimization

  • keyperimpl/shutterservice/eventprocessor.go — new interface
    - FilterCriteria struct carries Addresses []common.Address and Topics0 []common.Hash (empty
    Topics0 means "any topic" for that processor).
    - EventProcessor interface replaces the old FetchEvents(ctx, start, end) with FilterCriteria(ctx, start, end) + ParseEvents(ctx, start, end, logs).

  • keyperimpl/shutterservice/multieventsyncer.go — combined fetch
    - gatherFilterCriteria asks each processor for its criteria, builds the unioned address list and topic[0] list, captures per-processor scopes for routing.
    - syncRange now issues at most one FilterLogs(addresses, [topic0Union]) per range (skipped entirely if no processor wants any addresses), then routes the matching subset of logs to each processor's ParseEvents. If any processor declines to constrain topic[0], the combined query drops topic filtering.

  • keyperimpl/shutterservice/eventtriggerregisteredprocessor.go
    - Constructor now takes the contract address and parses the bindings' ABI to extract the EventTriggerRegistered topic. Returns an error from construction.
    - FilterCriteria returns {[address], [topic]}.
    - ParseEvents calls Contract.ParseEventTriggerRegistered(log) (abigen helper) per log.

  • keyperimpl/shutterservice/triggerprocessor.go
    - New activeTriggers helper reads + parses active trigger definitions from the DB.
    - FilterCriteria unions the addresses and topic[0] hashes across all active triggers; sets anyTopic when a trigger leaves topic[0] unconstrained.
    - ParseEvents iterates each trigger over the pre-filtered logs and emits TriggerEvents for ones that pass def.Match and aren't expired.

  • keyperimpl/shutterservice/keyper.go
    - Updated NewEventTriggerRegisteredEventProcessor call to pass the address and handle the new error return.

Steady-state RPC count per block range with N active triggers drops from 1 (FilterEventTriggerRegistered) + N (per-trigger FilterLogs) to 1 combined FilterLogs. The HeaderByNumber(end) is still there (that's S3, not in scope).

Note: there's one DB cost trade-off worth flagging -GetActiveEventTriggerRegisteredEvents is now queried twice per range inside TriggerProcessor (once in FilterCriteria, once in ParseEvents). S8 in the doc covers the cache that would eliminate this.

B4 optimization

  • New() precomputes the union of contract addresses, the union of event topic0 hashes, and an (address, topic) -> *EventType dispatch table.
  • syncAllInRange() now issues a single FilterLogs covering every registered (address, topic), then dispatches each returned log to its EventType via the precomputed table. Logs that match the query but have no registered handler (cross-product hits) are skipped.
  • Removed the per-event-type fan-out (syncSingleInRange) along with the sync.Mutex and golang.org/x/sync/errgroup dependencies.

For the snapshot keyper (the only current caller via chainobserver), this drops per-block eth_getLogs requests from N (one per event type) to 1. Behavior is preserved: the same logs reach Next() in the same block + log-index order.

This adds (generated) integration tests and a `mise` test task for
ensuring the changed batching behavior when responding to event based
decryption triggers.

Context is #698
ActivationBlockNumber can not be negative, so no overflow risk here.
Also apply `black` on `test-event-decryption`.
- deploy test helper in `deploy`
- untangled test and setup:
  - explicit `submit-event-registration` task
  - explicit `trigger-event` task
  - explicit `wait-for-event-decryption-key`
- New() precomputes the union of contract addresses, the union of event topic0 hashes, and a
  (address, topic) -> *EventType dispatch table.
  - syncAllInRange() now issues a single FilterLogs covering every registered (address, topic),
   then dispatches each returned log to its EventType via the precomputed table. Logs that
  match the query but have no registered handler (cross-product hits) are skipped.
  - Removed the per-event-type fan-out (syncSingleInRange) along with the sync.Mutex and
  golang.org/x/sync/errgroup dependencies.

  For the snapshot keyper (the only current caller via chainobserver), this drops per-block
  eth_getLogs requests from N (one per event type) to 1. Behavior is preserved: the same logs
  reach Next() in the same block + log-index order.
 keyperimpl/shutterservice/eventprocessor.go — new interface
  - FilterCriteria struct carries Addresses []common.Address and Topics0 []common.Hash (empty
  Topics0 means "any topic" for that processor).
  - EventProcessor interface replaces the old FetchEvents(ctx, start, end) with
  FilterCriteria(ctx, start, end) + ParseEvents(ctx, start, end, logs).

  keyperimpl/shutterservice/multieventsyncer.go — combined fetch
  - gatherFilterCriteria asks each processor for its criteria, builds the unioned address list
  and topic[0] list, captures per-processor scopes for routing.
  - syncRange now issues at most one FilterLogs(addresses, [topic0Union]) per range (skipped
  entirely if no processor wants any addresses), then routes the matching subset of logs to
  each processor's ParseEvents. If any processor declines to constrain topic[0], the combined
  query drops topic filtering.

  keyperimpl/shutterservice/eventtriggerregisteredprocessor.go
  - Constructor now takes the contract address and parses the bindings' ABI to extract the
  EventTriggerRegistered topic. Returns an error from construction.
  - FilterCriteria returns {[address], [topic]}.
  - ParseEvents calls Contract.ParseEventTriggerRegistered(log) (abigen helper) per log.

  keyperimpl/shutterservice/triggerprocessor.go
  - New activeTriggers helper reads + parses active trigger definitions from the DB.
  - FilterCriteria unions the addresses and topic[0] hashes across all active triggers; sets
  anyTopic when a trigger leaves topic[0] unconstrained.
  - ParseEvents iterates each trigger over the pre-filtered logs and emits TriggerEvents for
  ones that pass def.Match and aren't expired.

 keyperimpl/shutterservice/keyper.go
  - Updated NewEventTriggerRegisteredEventProcessor call to pass the address and handle the new
   error return.

  Steady-state RPC count per block range with N active triggers drops from 1
  (FilterEventTriggerRegistered) + N (per-trigger FilterLogs) to 1 combined FilterLogs. The
  HeaderByNumber(end) is still there (that's S3, not in scope).

  Note: there's one DB cost trade-off worth flagging — GetActiveEventTriggerRegisteredEvents is
   now queried twice per range inside TriggerProcessor (once in FilterCriteria, once in
  ParseEvents). S8 in the doc covers the cache that would eliminate this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant