Skip to content

Wire the ported core/tracker into the node graph so core_tracker_* metrics are emitted #571

Description

@emlautarom1-agent

Status

Pluto's duty tracker (crates/core/src/tracker/) was fully ported under #179 but never wired into the node, so Pluto emitted only 1 of its 12 core_tracker_* families (inclusion_delay, the lone unlabeled gauge) and the dashboard's "Duties" panels were blank for Pluto peers.

Implementation is up for review in #570 (stacked on #569#567; can only merge after both). It instantiates the tracker + analyser/deleter deadliners, emits all 10 step events across the 9 workflow stitch points, ports calculate_tracker_delay (startup-noise suppression) and the analyser/deleter lag offsets (adds INCL_CHECK_LAG, exports INCL_MISSED_LAG), and adds a networked InclusionChecker driving the existing I/O-free InclusionCore (proposer path). #570's run on a 2-charon/2-pluto kurtosis cluster: coverage 1 → 9 of 12 families, per-duty success/expect matching Charon (attester 48/71 on both; no_local_vc_signature=23 on both), proposer on-chain-inclusion reporting live (23 vs Charon's 25).

What's still missing

  1. proposer_zero_randaos not reported — fetcher issue, not tracker. Charon reports 6; Pluto reports 0 and shows proposer 10/10, under-reporting the expected total (10 duties tracked vs Charon's 16). fetch_proposer_data (crates/core/src/fetcher/mod.rs:335) awaits the aggregated randao via query_agg_sig_db (:344) — which blocks indefinitely if it's never stored — instead of erroring, so a zero-randao proposer duty fires no fetcher event and is never analysed. Needs a fetcher fix — separate follow-up.
  2. Attestation-inclusion path — feature-gated, deferred. Feature::AttestationInclusion is Alpha/off-by-default. Enabling it needs get_block_attestations_v2 + get_epoch_committees and a port of Charon's conjugateAggregationBits; driver-side only, no core type changes. (Proposer inclusion — the default — is done.)
  3. Minor. Charon also reports 2 not_included_onchain that Pluto doesn't — a small inclusion-path/timing discrepancy, not the fetcher. And 3 rare labeled families (unexpected_events_total, inconsistent_parsigs_total, inclusion_missed_total) only emit once first triggered — not missing.

Done when

The labeled core_tracker_* families emit on Pluto peers and per-duty success/expect matches Charon in a mixed cluster. Concretely: close when #570 merges, and split items 1–2 into their own follow-ups (item 1 is higher-value — it explains the still-blank proposer-failure panels for Pluto peers).

References / out of scope

Implementation: #570 (stacked on #569#567). Module port, no wiring: #179 (closed). Parents: #402, #396, #394. Sibling gap app_eth2_* (Beacon Node Score panels) is greenfield — lands with the eth2wrap port (#60).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust codetrack:validator-apiDev B — validator API + tracker

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions