Skip to content

feat(expose): add declarative HA-to-SCS bus expose engine and virtual actuator LED sync - #750

Open
GreenGrassBlueOcean wants to merge 11 commits into
v2-phase1-architecturefrom
feat/declarative-expose
Open

GreenGrassBlueOcean wants to merge 11 commits into
v2-phase1-architecturefrom
feat/declarative-expose

Conversation

@GreenGrassBlueOcean

@GreenGrassBlueOcean GreenGrassBlueOcean commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What is the problem

In a BTicino / Legrand MyHOME SCS installation, users want to project Home Assistant entities and the system clock onto the OpenWebNet bus as virtual sensors, meters, and bidirectional virtual actuators (expose, paired with OpenWebNet-HA/OWNd#129, merged into OWNd master; requires OWNd ≥ 2.0.0b11, which manifest.json already pins):

  1. Bidirectional Virtual Actuators (WHO=1 & WHO=2):
    • Repurpose an unused point-to-point SCS address (11..99, 0101..0915, 1001..1015, with optional #4#00..#4#15 local-bus suffix) on a physical BTicino wall control or touchscreen to control any non-BTicino Home Assistant entity (media_player, light, switch, fan, cover, input_boolean, lock, etc.) or execute custom Home Assistant script actions (turn_on_action, turn_off_action, step_up_action, step_down_action), while mirroring HA state back onto the bus (*1*1*W##, *1*0*W##, *#1*W*#1*...##, *2*1/2/0*W##, *#2*W*#11#001*...##).
  2. Virtual Temperature Probes (WHO=4, Dimension 0):
    • Publish Home Assistant temperature sensors (°C, within [-99.9, +99.9]°C) onto probe addresses (000..099 or 101..999 excluding N00 for N >= 1, normalizing "0" to "000") via *#4*WHERE*#0*TTTT## (OWNHeatingCommand.write_temperature when supports_who4_command_injection=True, or *#4*WHERE*0*TTTT## when supports_dimension_response_injection=True). On a gateway whose profile does not set either flag to True (today every profile except F461), the binding is disabled and a warning names it at startup.
  3. Virtual Energy Meters (WHO=18, Dimensions 113, 51..54 & 1200 Streaming):
    • Publish smart-meter / inverter power (dim=113, auto-converting kW $\rightarrow$ W) and energy counters (dim=51..54, auto-converting kWh $\rightarrow$ Wh) at meter addresses 51..5255 or 71#0..7255#0, and respond to Dimension 1200 active-power streaming handshakes (*#18*WHERE*#1200#1*DURATION##). Only on gateways whose profile sets supports_who18_response_injection=True (today F461); elsewhere the binding is disabled with a startup warning.
  4. Bus Clock Synchronization (WHO=13, Dimensions 0, 1, 22):
    • Broadcast Home Assistant local time/date across Dimensions 0, 1, and 22 without requiring an entity_id, answer *#13**0/1/22## queries when respond_to_read=True, and support a read-only query-reply mode when periodic_send: 0 (suppressing proactive *#13**#22*...## writes on startup/reconnect for NTP-synced gateways).

How is it solved

  1. ExposeManager & ExposeBinding (custom_components/myhome/expose.py):
    • Profile-Aware #DIM Write vs. Report Injection (_effective_profile, _build_frames, _async_handle_bus_message):
      • On real BTicino gateways (bt_luci and bt_termo), incoming un-hashed dimension response frames (*#1*W*1*level*speed##, *#2*W*10*state*pos*001*0##, *#4*W*0*TTTT##) on the TCP command session are NACKed or parsed as dimension reads, whereas #DIM write frames (*#1*W*#1*level*speed##, *#2*W*#11#001*pos##, *#4*W*#0*TTTT##) are ACKed in ~83–126 ms and forwarded to the SCS bus. Conversely, WHO=18 (bt_energy / bt_supervisione) does accept un-hashed *#18*W*DIM*VAL## response frames (113, 51..54, 1200#1).
      • OWNd ≥ 2.0.0b11 GatewayProfile declares supports_dimension_response_injection, supports_who4_command_injection and supports_who18_response_injection as bool | None = None, where None means unverified (no capture or oracle verdict). Only F461Profile sets them, from the live captures: False, True, True.
      • Unverified means unsupported. _injection_block_reason gates WHO=4 and WHO=18: frames are built only when the flag is exactly True (for WHO=4, supports_dimension_response_injection=True also qualifies). False, None and a not-yet-known profile each block the binding. async_sync_all() (after initial discovery and on every reconnect) logs one warning per binding and reason, e.g. "Expose WHO=4 WHERE=050 dimension=0 (sensor.temp) is disabled: gateway MH202 is unverified for WHO 4 #0 temperature writes (supports_who4_command_injection is unset in OWNd). Nothing is sent for this binding." A later model correction to a verified profile re-enables the binding without a restart.
      • With supports_dimension_response_injection not True, WHO=1 emits OWNLightingCommand.set_brightness (*#1*W*#1*(100+pct)*0##, omitting a duplicate *1*0*W## when brightness_pct == 0) and WHO=2 emits OWNAutomationCommand.set_shutter_level (*#2*W*#11#001*pos##, answering both Dimension 10 and 11 read queries) on every gateway. Where enabled, WHO=4 emits OWNHeatingCommand.write_temperature (*#4*W*#0*TTTT##) and WHO=18 emits OWNEnergyCommand.report_* (*#18*W*113*W##, 51..54, 1200#1).
    • Home Assistant Script Runner, cv.SCRIPT_SCHEMA Validation & Per-Binding Error Containment (validate.py, expose.py):
      • Validates turn_on_action, turn_off_action, step_up_action, and step_down_action at schema time via cv.SCRIPT_SCHEMA (_validate_expose_action / _coerce_action_list) and pre-compiles Home Assistant Script runners on startup via async_validate_actions_config (_async_prepare_action_scripts), unloading them cleanly via script.async_unload() on async_stop().
      • Supports full Home Assistant script syntax including Jinja2 templates (data, target, service_template with who, where, entity_id, and this via TemplateStateFromEntityId), conditions (condition: state / template), if/then/else, choose, repeat, parallel, and sequence blocks.
      • Checks static <domain>.<service> existence at startup (_warn_unknown_action_services, or on EVENT_HOMEASSISTANT_STARTED when hass.state is not yet CoreState.running) so mistyped service names are flagged immediately instead of only when someone presses the wall button.
      • Contains (HomeAssistantError, vol.Invalid) per binding in both _async_run_actions and _async_call_service, ensuring a failing action on one binding never aborts subsequent bindings during a General (*1*0*0##) or Area (*2*2*9##) bus broadcast.
    • Write-Future-Bound Echo Guard (_EchoGuardEntry, _record_echo_guard, _is_echo_guarded):
      • Tracks per-sequence _EchoGuardEntry instances (TX_ECHO_GUARD_WINDOW_S = 1.5s) bound to FrameScheduler write futures (_attach_echo_guard_future). While a frame's write future is pending in the queue (not future.done()), incoming companion frames are guarded without popping the entry; once the socket write completes, the 1.5s TTL starts from future completion (and if cancelled before write, the guard is immediately removed).
      • Absorbs the 10-gateway WHO=2 pre-move STOP translation frame (*2*1000#0*W## before *2*1000#1/2*W##) via _pre_move_stop_guard, registers and suppresses the 6-gateway WHO=2 #11#001 $\rightarrow$ #11#001#1 monitor translation (*#2*W*#11#001#1*LEVEL## emitted by F454, F459, F460, F461, MH202, and MyHomeServer1 when *#2*W*#11#001*LEVEL## is sent on cmd), preserves sibling raw loopback guards (*WHO*WHAT*W## / *#2*W*#11#001*LEVEL##) when translation echoes arrive first, and tracks consumed gateway MACs (seen_sources) so genuine human repeat presses are never swallowed.
    • Queue Superseding & NACK Tolerance (custom_components/myhome/gateway_sessions.py):
      • FrameScheduler supersedes pending low-priority status frames for the same (who, where, dimension) target (_expose_replace_key) when a newer frame or high-priority confirmation is queued, cancelling the stale frame's written future so an old mirror frame can never arrive after a wall-button press and undo it.
      • CommandWorkerPool marks exposed targets (_is_exposed_target) so virtual addresses without physical bus hardware do not trigger command retries or ERROR logs on *#*0## (NACK) on MyHomeServer1 / F460 / F461 while still resolving the written future when the session is open.
    • Trailing-Edge Cooldown Debounce, min_delta Deadband & periodic_send Heartbeat:
      • Schedules in-cooldown state changes on last_sent_at + cooldown (_cooldown_timers) for eventual consistency, filters numeric jitter below min_delta, runs periodic_send heartbeats (900s default for WHO=4, 86400s default for WHO=13 in both YAML and UI ExposeSubentryFlowHandler), and skips proactive WHO=13 clock writes in async_sync_all() when periodic_send <= 0.
    • Infeasible-Input Hardening (validate.py, expose.py, where_grammar.py, config_flow.py):
      • Schema validation (_finite_float, _non_bool_int, _non_bool_str, _validate_expose_action, _validate_expose_entry) rejects booleans disguised as integers/floats/strings, fractional ints (1.9), NaN/±inf values (which otherwise bypass Voluptuous Range(min=..., max=...)), OverflowError floats (10**400), *-containing where strings, invalid WHO=1/2 point-to-point addresses, invalid WHO=4 probe addresses (100, 200, negative -105/-0, multi-# ###105, or 4+ digit 9999) or out-of-range defaults (<-99.9 or >99.9), and invalid WHO=18 meter addresses (50, 5256, 70#0, 7256#0) or negative defaults.
      • where_grammar.py (zone_number, is_probe, normalize_expose_where, expose_message_where) uses .removeprefix("4-") and .removeprefix("#") (preventing -105 or ###105 from normalizing into valid probes), and rejects bool and ±inf with ValueError. It carries no OWNd shims for expose: the frame builders, query helpers (effective_what, is_status_request, is_dimension_writing, …) and profile flags all come from OWNd ≥ 2.0.0b11, so nothing in MyHOME patches OWNMessage or the profiles for this feature.
      • ExposeSubentryFlowHandler (config_flow.py) and ExposeBinding.from_dict (expose.py) validate who, where, cooldown, min_delta, periodic_send, brightness_step_pct, and default against booleans, NaN/±inf, and malformed numeric strings without raising unhandled ValueError/OverflowError.
      • Runtime _build_frames and _async_handle_bus_message guard against booleans, NaN/±inf/overflow attributes and states, WHO=1 missing/None/False brightness attributes, WHO=13 pre-NTP years outside 2000..2099, kW/kWh multiplication overflow, and out-of-range incoming bus dimension writes (WHO=1 level >200, WHO=2 level >100 including 255 unknown sentinel, WHO=18 energy_type outside 1..4 or duration outside 0..255).
  2. Shared-Bus Delegation, Auto-Discovery Suppression & ConfigSubentry UI (gateway.py, discovery.py, button.py, config_flow.py, validate.py, where_grammar.py):
    • Subscribes to shared-bus secondary gateways' myhome_message_{mac} signals for delegated_away_whos (including late-starting secondaries) and deduplicates cross-gateway RX frames within 0.5s (_last_rx_by_frame).
    • Suppresses phantom entity discovery and companion buttons across all gateways on the shared bus while preserving climate entities.
    • Runs async_sync_all() after normal initial discovery completion (outside finally:) and on event-session reconnects.

How is it tested

  • tests/test_expose.py (29 end-to-end & regression tests) + tests/test_f461_live_shutter_expose_trace.py (5 live F461 replay tests; 100.0% statement coverage on expose.py (1056 Stmts, 0 Miss), where_grammar.py, and ExposeSubentryFlowHandler in config_flow.py):
    • WHO=1 & WHO=2 bidirectional state mirroring (*#1*W*#1*...## and *#2*W*#11#001*...## by default, plus report_brightness / report_shutter_level when supports_dimension_response_injection=True), attribute="brightness" OFF fallback (0% / *1*0*W##), #11#001 $\rightarrow$ #11#001#1 translation echo suppression, custom Script actions with templates ({{ this.state }}, {{ entity_id }}, {{ who }}, {{ where }}), conditions (condition: template, if/then/else, choose, repeat, parallel, sequence), startup unknown-service warnings, per-binding runtime error containment, dimmer hold stepping (1000#30/31), preset levels (2..10), and General ("0") / Area ("1".."9", "00", "100") broadcasts with #4#xx isolation.
    • test_who2_oracle_40_step_echo_loop_simulation: 40-step oracle bus simulation verifying that WHO=2 UP (*2*1000#1*98##), STOP (*2*1000#0*98##), and DOWN (*2*1000#2*98##) presses followed by multi-frame gateway echoes (*2*1000#0*98##, *2*1000#1*98##, *2*1*98##) settle in exactly 1 service call each with zero oscillation.
    • test_run4_expose_live_f461_protocol_and_echo_guard_replay (tests/test_f461_live_shutter_expose_trace.py): replays the 2026-10-18 and 2026-10-20 live physical BTicino F461 (2.0.11) captures (tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json, PR test(f461): add live F461 2026-10-18 trace for F411/4 lights, F411/2 5s shutter, and Expose bus verification #753, and @gdluck's 2026-10-20 follow-up matrix) through ExposeManager, verifying that:
      1. Single and double WHO=2 event-session translation bursts (['*2*1000#0*25##', '*2*1000#1*25##'] $\times 2$) arriving while the command-session write future is pending (before the ~0.55 s NACK) are absorbed with zero self-oscillation.
      2. Outgoing *#2*25*#11#001*50## shutter position writes and their F461 event-session translation *#2*25*#11#001#1*50## are absorbed by _tx_echo_guard without looping back into cover.set_cover_position.
    • test_expose_infeasible_inputs_hardening: verifies that expose_entry_schema, ExposeSubentryFlowHandler, ExposeBinding.from_dict, _build_frames, _async_handle_bus_message, where_grammar.py, and OWN*Command builders reject booleans, fractional ints, NaN/±inf/10**400, negative/multi-# probe addresses, out-of-range numeric values, and malformed where addresses.
    • test_unverified_gateway_disables_who4_who18_with_one_warning: on an MH202 profile (flags unset) WHO=4/WHO=18 bindings send nothing while WHO=1 still syncs; exactly one warning per binding across repeated syncs; distinct reasons for a False flag and for a missing profile; and once the profile resolves to F461 the same bindings emit *#4*050*#0*0215## / *#18*51*113*450## with no further warning. The test gateway helper defaults to F461Profile, the only profile with verified flags, and the shared-bus delegation test now delegates WHO=18 to an F461 secondary.
    • Queue superseding & write-future echo-guard tests: verifies that a wall-button ON press supersedes a still-queued status-tier OFF mirror frame so a stale OFF never executes after ON, and that late startup/reconnect echoes are guarded until 1.5s after the write future resolves.
    • Trailing-edge cooldown expiry execution, min_delta deadband filtering, periodic_send timer firing, WHO=4 positive/negative temperature encoding (*#4*W*#0*TTTT## write default + report_temperature when supports_dimension_response_injection=True), WHO=18 active power/energy + DIMENSION 1200 streaming lifecycle, WHO=13 clock sync (periodic_send > 0 vs periodic_send == 0 read-only skip) and *#13**0/1/22## query replies, shared-bus secondary gateway delegation & RX deduplication, phantom entity/button suppression, YAML validation, and ExposeSubentryFlowHandler UI flows.
  • Run against OWNd master (0a82296, the b11 candidate, Features/add support open and close time #129 merged): test_expose.py, test_f461_live_shutter_expose_trace.py and test_where_usage_guard.py pass (38). The full suite gives 4319 passed, 3 failed; the 3 are test_connection.py::TestOWNGatewayDiscoveryAndClassmethods tests that fail identically on the base branch with OWNd master, so they are OWNd drift, not this PR.
  • Static Analysis & Quality Gates:
    • python scripts/typing_ratchet.py $\rightarrow$ mypy --strict: 0 error(s) in 0 module(s); baseline 0 (as CI runs it, without OWNd type information; with OWNd master installed, expose.py reports 16 errors because cmd/confirm/frames mix OWNLightingCommand, OWNAutomationCommand, OWNEnergyCommand and OWNGatewayCommand, and the base branch already reports 11 in other modules)
    • ruff check custom_components tests scripts $\rightarrow$ All checks passed!
    • python scripts/verify_ha_standards.py $\rightarrow$ SUCCESS: All Home Assistant architectural standards passed!
    • python scripts/sync_documentation.py --check $\rightarrow$ SUCCESS: All documentation is calibrated and in sync with the codebase!

State Diagram

1. Write-Future-Bound Echo Guard & Virtual Actuator State Machine (WHO=1 / WHO=2)

stateDiagram-v2
    [*] --> Synced : async_sync_all()

    state "Synced (HA state == Bus state)" as Synced
    state "Queued in FrameScheduler (future pending)" as QueuedTX
    state "Written to Gateway Socket (1.5s TTL active)" as WrittenTX
    state "Bus Command Intercepted (*WHO*1000#WHAT*W##)" as BusCommand

    Synced --> QueuedTX : HA state_changed -> _async_dispatch_frames()\n1. Supersede any stale queued TX for (who, where, dim)\n2. Arm _EchoGuardEntry (raw + 1000#what + WHO=2 pre-move STOP + #11#001#1)\n3. Attach future callback

    QueuedTX --> Synced : Superseded or Cancelled before socket write\n-> Immediately remove _EchoGuardEntry
    QueuedTX --> WrittenTX : Socket write completes (future.done())\n-> Start 1.5s TTL from now

    WrittenTX --> WrittenTX : Monitor/Command echo arrives:\n• *2*1000#0*W## -> consume _pre_move_stop_guard\n• *WHO*1000#WHAT*W## or *#2*W*#11#001#1*L## -> consume trans_entry (keep sibling raw_entry!)\n• *WHO*WHAT*W## or *#2*W*#11#001*L## -> consume raw_entry
    WrittenTX --> Synced : All echo forms consumed or 1.5s TTL expires

    Synced --> BusCommand : Physical Button Press (*WHO*1000#WHAT*W## or *WHO*WHAT*W##)
    BusCommand --> QueuedTX : 1. Run HA Script / service (templates, conditions, error-contained)\n2. Queue virtual actuator confirmation (Priority.HIGH, no NACK retry)
Loading

2. Trailing-Edge Cooldown, min_delta Deadband & periodic_send State Machine

stateDiagram-v2
    [*] --> Idle : Binding Initialized

    state "Idle (Cooldown Elapsed)" as Idle
    state "Deadband Check (|val - last_sent| >= min_delta?)" as DeadbandCheck
    state "Cooldown Active (No Pending Value)" as CooldownActive
    state "Cooldown Active (Trailing-Edge Timer Armed)" as TrailingPending

    Idle --> DeadbandCheck : HA state_changed(new_val)
    DeadbandCheck --> Idle : |new_val - last_sent| < min_delta\n(Filtered out)
    DeadbandCheck --> CooldownActive : Delta >= min_delta\nDispatch Frame (last_sent_at = now)

    CooldownActive --> TrailingPending : state_changed(val) within cooldown\nSchedule _fire_trailing at (last_sent_at + cooldown)
    TrailingPending --> TrailingPending : Additional state_changed(latest_val)\nCoalesced until trailing-edge timer fires
    TrailingPending --> CooldownActive : Cooldown timer fires\nDispatch latest HA state (Eventual Consistency!)

    CooldownActive --> Idle : Cooldown window elapsed
    Idle --> CooldownActive : periodic_send timer fires\nRe-publish current state
Loading

Where are the traces

Subsystem / Mechanism Hardware Trace / Fixture in MyHOME Firmware Oracle Suite (own-firmware-oracle)
Live F461 (2.0.11) Expose (WHO=1, 2, 4, 13, 18), F411/4 Lights & F411/2 5 s Shutter (2026-10-18 & 2026-10-20) tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json (PR #753 & issuecomment-6277050571) run1_lights_group_and_general, run2_shutter_05_travel_and_reversal, run3_climate_zone2, run4_expose_virtual_bus_verification
WHO=1 Command Translation (1000#30/31) & Dimmer Steps tests/fixtures/traces/issue_466/myhome_trace_MH202_f418u2_dimmer_2026-09-27T09-43-45.json
tests/fixtures/traces/issue_466/myhome_trace_H4890_all_2026-09-26T11-51-26.json
results/*/oracle/full/lights-level.tsv (*1*1*31##, MH202 *1*1000#1*31## session response)
WHO=2 Shutter Command Translation, Pre-Move STOP & #11#001#1 Level Echo tests/fixtures/traces/issue_645/myhome_trace_MyHomeServer1_cover_direction_then_stop_2026-10-11.json
tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json
results/*/oracle/full/trace-corpus.tsv (*2*1*31## $\rightarrow$ *2*1000#0*31##, *2*1000#1*31## on 10 gateways; *#2*31*#11#001*40## $\rightarrow$ *#2*31*#11#001#1*40## on F454/F459/F460/F461/MH202/MyHomeServer1)
WHO=4 Temperature Probe DIMENSION 0 tests/fixtures/traces/issue_429/myhome_trace_MyHomeServer1_who4_2026-09-23T18-13-49.json
tests/fixtures/traces/issue_619/myhome_trace_F453AV_who4_2026-10-11T10-29-57.json
results/*/oracle/full/thermo-ts10.tsv (*#4*1*0## $\rightarrow$ SCS $0499013000\r)
WHO=18 Active Power (113) & Streaming (1200) tests/fixtures/traces/issue_466/myhome_trace_F454_who18_2026-10-08T19-30-16.json results/*/oracle/full/energy-ts10.tsv (*#18*51*113## $\rightarrow$ SCS $06D1A10233001D0000\r)
Unpopulated Address Command-Session NACK tests/fixtures/traces/issue_466/MyHomeServer1_dead_zone_nacks_2026-09-29T22-45.txt
tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json
results/MyHomeServer1/028206/oracle/full/lights-level.tsv (Reply: NACK (*#*0##), Verdict: OUT (Bus))

What does the Oracle / Encyclopedia say

1. OpenWebNet Encyclopedia Machine KB Claims (OpenWebNet-Encyclopedia/knowledge/claims)

Claim ID Epistemic Status Encyclopedia Finding & Implementation
ownkb:claim:c000114–c000118 corroborated_interpretation Frame forms: *WHO*WHAT*WHERE## (command/state), *#WHO*WHERE## (status request), *#WHO*WHERE*DIMENSION## (dimension request), *#WHO*WHERE*DIMENSION*VALUE...## (dimension response), and *#WHO*WHERE*#DIMENSION*VALUE...## (dimension write).
ownkb:claim:c001440, c001442 specified WHO=1 Command Translation (1000#INNER_WHAT) wraps translated lighting/dimmer commands (*1*1000#WHAT*WHERE##).
ownkb:claim:c002643, c002647 specified WHO=2 Command Translation (1000#INNER_WHAT) wraps translated automation commands (*2*1000#0/1/2*WHERE##).
ownkb:claim:c001132, c001133, c003172, c003235, ownkb:claim:moc-c007713 specified / observed WHO=4 External probe class uses [PL_N]00; DIMENSION 0 encodes measured temperature in 4-digit tenths of °C (0000..0500 in c003235; values $\ge 1000$ negative-offset encoding in moc-c007713).
ownkb:claim:c002222, c002264, c002287, ownkb:claim:moc-c007644 specified / observed WHO=18 DIMENSION 113 reports active power in Watts (*#18*W*113*WATT##) and is emitted during DIMENSION 1200 active-power streaming; DIMENSION 51..54 report energy totalizers in Wh.
ownkb:claim:c001598–c001604, c001674–c001676, c001742–c001743 specified WHO=13 DIMENSION 22 (H*M*S*T*W*D*M*Y) combines DIMENSION 0 (time + timezone) and DIMENSION 1 (weekday + date) in a single atomic R/W frame.

2. Empirical Firmware Oracle & Live BTicino F461 (2.0.11, 2026-10-18 & 2026-10-20) Hardware Verdicts

  • Report Form (*#WHO*WHERE*DIM*VAL##) vs. #DIM Write Form (*#WHO*WHERE*#DIM*VAL##) on Physical F461 (2.0.11):
    • WHO=1: *#1*08*1*150*0## (report form) is rejected with NACK in 5 ms, whereas *#1*08*#1*150*0## (set_brightness #1 write form) is ACKed in 88–126 ms.
    • WHO=2: *#2*25*10*10*50*001*0## (report form) is rejected with NACK in 82 ms, whereas *#2*25*#11#001*50## (set_shutter_level #11#001 write form) is ACKed in 83 ms and echoed on evt as *#2*25*#11#001#1*50##.
    • WHO=4: *#4*2*0*0999## and *#4*50*0*0215## (report form without #0) are parsed by bt_termo as Dimension 0 read requests (*#4*2*0*0999## returns Zone 2's measured temperature *#4*2*0*0249##), whereas *#4*50*#0*0215## (write_temperature #0 write form) is ACKed in 84–155 ms.
    • WHO=18: Unlike bt_luci and bt_termo, bt_energy / bt_supervisione does accept un-hashed dimension response frames (*#18*51*113*450##, 51..54, 1200#1), which are ACKed in 140 ms – 1.7 s on F461.
  • Live F461 WHO=2 Virtual Cover (WHERE="25", unpopulated vs physical BT-F411/2 at WHERE="05"):
    • Physical F411/2 (WHERE="05", 5 s travel cutoff): *2*1*05## emits ['*2*1000#0*05##', '*2*0*05##', '*2*1000#1*05##', '*2*1*05##'] on evt, and a mid-flight reversal (*2*1*05## $\rightarrow$ *2*2*05##) is followed 5.0 s later by the hardware relay's spontaneous end-of-travel *2*0*05##.
    • Virtual cover (WHERE="25", no physical actuator): sending *2*1*25##, *2*2*25##, or *2*0*25## on the live F461 immediately broadcasts ['*2*1000#0*25##', '*2*1000#1*25##'] (sometimes twice before the ~0.55 s NACK), ['*2*1000#0*25##', '*2*1000#2*25##'], and ['*2*1000#0*25##'] on the evt monitor session. Replaying these exact evt bursts through ExposeManager confirms that _pre_move_stop_guard and _tx_echo_guard absorb all translation frames with zero self-oscillation.
  • Residual Hardware Caveats & Custom Action Security Model (documented in expose.py):
    • Physical wall-control LED illumination without a backing actuator at WHERE and answering respond_to_read status queries from a TCP monitor session remain best-effort subject to gateway/control firmware.
    • Bidirectional turn_on_action / turn_off_action / step_up_action / step_down_action entries are validated via cv.SCRIPT_SCHEMA / async_validate_actions_config and executed via Home Assistant's Script helper (with sandboxed Template evaluation) in response to bus frames, and should only be configured on trusted physical/network SCS installations.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@gdluck

gdluck commented Oct 19, 2026

Copy link
Copy Markdown

Plant check of the mirror frames on the F461 (2.0.11), each frame sent twice (as command and as status request), virtual addresses 08 / 25 / 50 / 51:

Report form (report_brightness / report_shutter_level / report_temperature) Result Write form Result
*#1*08*1*150*0## NACK in 6–10 ms, not forwarded *#1*08*#1*150*0## ACK 83 ms
*#2*25*10*10*50*001*0## NACK 10 ms *#2*25*#11#001*50## ACK 83 ms, echoed *#2*25*#11#001#1*50##
*#4*50*0*0215## treated as a read, NACK after 4.2 s *#4*50*#0*0215## ACK 88 ms
*#18*51*113*450## ACK 1.2–1.7 s — —

_build_frames picks the report forms whenever profile.supports_dimension_response_injection is true, and that attribute exists in no OWNd version (master, #129, #132); the shim in where_grammar.py sets it to True on every profile. So on an F461 the level of a virtual dimmer, the position of a virtual cover and a virtual temperature probe never reach the bus (silently, since exposed targets skip retries and error logs); only the on/off and up/down/stop mirrors and the WHO 18 / WHO 13 write frames do. The emulated bt_termo has only the *#4*%s*#0*…## write formatter for an incoming dimension 0, and bt_luci accepts only the #1 write (*#1*11*#1*134*50## ACK, oracle3/out/luci.tsv), which matches. The 2026-10-18 live verification used the write forms, not the frames the engine sends.

Two smaller observations from the same session: the gateway relays the virtual-cover translation burst twice (*2*1000#0*25##, *2*1000#1*25##, then both again) before the NACK, whereas the fixture has one burst; and the unpopulated-address NACKs came after 4.0 s (*1*1*08##) and 1.07 s (*2*1*25##) here rather than 2.06 s / 0.55 s. Also: the WHO 13 write frame really sets the gateway clock (verified, then restored), so the default periodic_send for WHO 13 overrides the gateway's own NTP time with HA's.

…, and skip WHO 13 sync when periodic_send=0 (#750)
@GreenGrassBlueOcean

Copy link
Copy Markdown
Contributor Author

Thank you @gdluck — this second live F461 (2.0.11) matrix is super helpful, especially the side-by-side comparison between the un-hashed report forms (*#WHO*WHERE*DIM*VAL##) and the #DIM write forms (*#WHO*WHERE*#DIM*VAL##) across WHO=1, WHO=2, WHO=4, and WHO=18, plus the #11#001 $\rightarrow$ #11#001#1 monitor translation capture!

Here is what we updated in OpenWebNet-HA/OWNd#129 (c68ffd5) and this PR (48ca3d5a) based on your findings:

1. Default to #DIM Write Frames for WHO=1, WHO=2, and WHO=4 (while keeping Report Form for WHO=18)

Your matrix proved that on real BTicino gateways (bt_luci and bt_termo), incoming un-hashed dimension response frames on the TCP command session are either rejected (*#*0##) or parsed as dimension reads (*#4*W*0*...##), whereas #DIM write frames are ACKed in ~83–88 ms and forwarded to the bus:

  • OWNd (GatewayProfile):
    • Changed supports_dimension_response_injection default from True to False across gateway profiles.
    • Added a dedicated supports_who18_response_injection: bool = True flag (False on MH200 / MH200N) because bt_energy / bt_supervisione does accept un-hashed *#18*WHERE*DIM*VAL## response frames (113, 51..54, 1200#1).
  • MyHOME (ExposeManager._build_frames & _async_handle_bus_message):
    • WHO=1: Uses OWNLightingCommand.set_brightness (*#1*WHERE*#1*(100+pct)*0##, ACKed in 88 ms on your F461) instead of report_brightness (*#1*WHERE*1*...##, NACKed), and avoids emitting a duplicate *1*0*WHERE## frame when brightness_pct == 0.
    • WHO=2: Uses OWNAutomationCommand.set_shutter_level (*#2*WHERE*#11#001*pos##, ACKed in 83 ms on your F461) instead of report_shutter_level (*#2*WHERE*10*...##, NACKed), and answers both Dimension 10 and 11 read queries.
    • WHO=4: Uses OWNHeatingCommand.write_temperature (*#4*WHERE*#0*TTTT##, ACKed in 84 ms) by default on all gateways that support WHO=4 command injection.
    • WHO=18: Continues emitting OWNEnergyCommand.report_* (*#18*WHERE*113*W##, 51..54, 1200#1).

2. Echo Guard for WHO=2 #11#001 $\rightarrow$ #11#001#1 Monitor Translation

  • Your capture showed that sending *#2*25*#11#001*50## on cmd causes the F461 to emit *#2*25*#11#001#1*50## on evt (we also confirmed the same #11#001#1 translation across F454, F459, F460, MH202, and MyHomeServer1 in our trace corpus).
  • _record_echo_guard and _is_echo_guarded now register and suppress the companion *#2*WHERE*#11#001#1*LEVEL## frame alongside *#2*WHERE*#11#001*LEVEL## so our own position updates never loop back into cover.set_cover_position.
  • Added both *#2*25*#11#001*50## and *#2*25*#11#001#1*50## to the OWNd golden corpus (who02_automation.yaml).

3. WHO=13 Read-Only Clock Mode (periodic_send: 0) & Double Translation Burst Replay

  • WHO=13 with periodic_send: 0: async_sync_all() now skips the proactive *#13**#22*...## write on startup/reconnect when periodic_send <= 0, so NTP-synced gateways (like your F461) can use WHO=13 in pure query-reply mode without overwriting the gateway clock.
  • Double WHO=2 translation burst (['*2*1000#0*25##', '*2*1000#1*25##'] $\times 2$): Added a regression test in tests/test_f461_live_shutter_expose_trace.py replaying the 4-frame burst while the command-session write future is still pending (before the ~0.55 s NACK), verifying zero self-oscillation.

GreenGrassBlueOcean and others added 3 commits October 20, 2026 16:49
…rop OWNd shims (#750)

OWNd 2.0.0b11 (with #129) declares the expose injection flags as
bool | None, where None means no capture or oracle verdict exists. Only
F461Profile sets them. Expose read them with bool(), and the shim's
hasattr() patch never fired on b11, so WHO 4 and WHO 18 bindings
silently sent nothing on every gateway except F461.

- Gate WHO 4 and WHO 18 on _injection_block_reason(): frames are built
  only when the flag is exactly True (WHO 4 also accepts
  supports_dimension_response_injection=True). False, None and an
  unknown profile each block the binding.
- async_sync_all() logs one warning per binding and reason, so a
  disabled binding is named at startup and again only if the reason
  changes (for example after a model correction).
- _effective_profile() reads the gateway's .profile only.
- Delete _ensure_ownd_expose_helpers and the profile-flag patches in
  where_grammar: the manifest pins OWNd 2.0.0b11, which ships them.
- Read binding.who/where directly in the error logs instead of
  getattr(..., "where"), which test_where_usage_guard rejects.
- Widen _iter_static_service_calls to list[Any] so mypy accepts the
  isinstance guard on nested choose/parallel items.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Bring in #742 (sweep gating, myhome.yaml copy in entry options) and
#744 (#732 modularization of config_flow, gateway and platforms).

Conflicts:
- validate.py: import list; keep both CONF_STEP_*_ACTION and
  CONF_STRATEGY.
- legacy_yaml.py: keep the base's _load_live_yaml/_apply_yaml_platforms
  structure and re-apply the expose handling on top: a top-level
  `expose:` key counts as single-gateway config, the where-filling loop
  skips `expose`, and the validated expose list is collected into
  configured_expose (threaded through _load_live_yaml).
- config_flow.py: take the base's split module and move
  ExposeSubentryFlowHandler into config_flow_expose.py, next to
  config_flow_options.py; config_flow.py keeps
  async_get_supported_subentry_types and re-exports the handler.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants