EEE support for rp1 mac

[nbuchwitz]

This adds EEE support to the macb driver for the Pi 5 / CM5. Right now EEE is broken on the Pi because the PHY advertises EEE but the MAC has no LPI support, so it was marked as broken in #6900. This series fixes that by implementing proper software-managed TX LPI in the driver and reverting the DT workaround.

See #6855

This should go through mainline as the EEE registers are the same across all Cadence GEM variants (SAMA5D2, SAME70, PIC32CZ, RP1, ...) and the code is generic, just gated on a capability flag. I'm looking for early feedback from Pi users before sending it upstream though. If you've got a Pi 5 or CM5, give it a spin and let me know how it goes — link stability, ethtool --show-eee, ethtool -S | grep lpi, that kind of thing.

[nbuchwitz]

f0d41dc is not directly EEE related, but I noticed it was missing while debugging

[pelwell]

Thanks, Nicolai - this looks great.

For anyone wishing to try the new EEE capability, you can install a trial build using sudo rpi-update pulls/7251, after reading the caveats.

[pelwell]

For comparison, these are the values I get after being up for a few minutes:

pi@philpi5:~$ ethtool --show-eee eth0
EEE settings for eth0:
        EEE status: enabled - active
        Tx LPI: 0 (us)
        Supported EEE link modes:  100baseT/Full
                                   1000baseT/Full
        Advertised EEE link modes:  100baseT/Full
                                    1000baseT/Full
        Link partner advertised EEE link modes:  100baseT/Full
                                                 1000baseT/Full
pi@philpi5:~$ ethtool -S eth0 | grep lpi
     rx_lpi_transitions: 0
     rx_lpi_time: 0
     tx_lpi_transitions: 88
     tx_lpi_time: 85760892

[nbuchwitz]

For better comparison: is the ethernet link actively used or not? If yes, how do the tx lpi rate change when the ethernet is mostly idle (eg wifi used)?

[nbuchwitz]

After some more testing, I've improved the patch set further and fixed a bug.

Changes since v1:

• Use mod_delayed_work() instead of schedule_delayed_work() for LPI idle timer re-entry. schedule_delayed_work() is a no-op when work is already pending, so the timer was never reset by TX completions - causing LPI to be asserted periodically even under full load.
• Set MACB_TX_LPI_TIMER_DEFAULT_MS to 500ms (was 250ms). Chosen as a good compromise between energy savings and throughput stability - lower values cause more frequent LPI wake transitions with associated retransmission overhead on bursty workloads.
• Remove unused RX LPI status bit change interrupt definitions (RXLPISBC). The interrupt is not wired up in this series and can be added later if needed for diagnostics.
• Include fix for a probable upstream bug in phy_ethtool_set_eee() where EEE advertised modes were not correctly restored after an off/on toggle, because eee_cfg.eee_enabled was updated after genphy_c45_ethtool_set_eee() instead of before. -> already fixed in a615309 (not part of rpi-6.12.y yet)

[nbuchwitz]

Changes since v2:

• Fix a bug in macb_tx_lpi_work_fn() where LPI was entered without checking whether all TX queues are actually idle. The delayed work was scheduled when one queue drained, but other queues could still have active transmissions. Now verify tx_head == tx_tail on all queues before asserting TXLPIEN, and reschedule the timer if any queue is still busy. With this fix, iperf3 measured throughput with EEE enabled is now identical to without EEE - earlier versions showed an initial bandwidth drop caused by premature LPI entry.
• Set MACB_TX_LPI_TIMER_DEFAULT_MS back to 250ms (the v1 default). The multi-queue idle check makes this safe - LPI is no longer entered prematurely under load, so a shorter idle timeout improves energy savings without risking throughput.

Metric                                 EEE off          EEE on
------------------------------ --------------- ---------------
TCP TX throughput                     937.0 Mb/s        937.3 Mb/s
  TX first 5s avg                     939.5 Mb/s        939.8 Mb/s
  TX remaining avg                    936.5 Mb/s        936.8 Mb/s
TCP RX throughput                     936.5 Mb/s        936.5 Mb/s
Ping 1s avg RTT                        0.188 ms         0.272 ms
Ping 10ms avg RTT                      0.172 ms         0.205 ms
Flood ping loss                 0% packet loss  0% packet loss
Flood ping avg RTT                     0.155 ms         0.199 ms
Ping under load avg RTT                1.915 ms         1.904 ms

LPI counters during 1s-interval ping (EEE on):
  rx_lpi_transitions: 0
  rx_lpi_time: 0
  tx_lpi_transitions: 101
  tx_lpi_time: 16234605

Note on RX LPI

RX LPI monitoring is not implemented in this series. The BCM54213PE PHY on the Raspberry Pi 5 operates in "native EEE" mode - it handles received LPI internally without forwarding it to the MAC over the RGMII interface. The PHY's vendor-specific LPI_FEATURE_EN bit (register 0x803d) is required for EEE negotiation to work at all, and with it enabled the PHY absorbs incoming LPI transparently. As a result, the GEM MAC's RX LPI hardware counters (RXLPI/RXLPITIME at 0x270/0x274) and the RXLPISBC status change interrupt (ISR bit 27) will never trigger, regardless of driver configuration. The PHY itself does detect and count received LPI (confirmed via PCS status register and the vendor-specific LPI counter at 0x803f), but this information is not exposed on the RGMII interface. This appears to be a PHY hardware limitation, not a driver issue - though this conclusion is based on register-level experimentation without access to Broadcom documentation.

@pelwell, could you check with your Broadcom contacts whether the BCM54213PE supports exposing RX LPI on the RGMII interface?

[pelwell]

could you check with your Broadcom contacts whether the BCM54213PE supports exposing RX LPI on the RGMII interface?

Warning: I'm at the limits of my EEE knowledge here, so this might be painful.

The BCM54213PE PHY on the Raspberry Pi 5 operates in "native EEE" mode - it handles received LPI internally without forwarding it to the MAC over the RGMII interface.

The PHY datasheet says that Native EEE is for MACs that support LPI signalling over RGMII, while AutogrEEEn mode is for MACs that don't support LPI signalling over RGMII. This seems to contradict your statement.

[nbuchwitz]

The BCM54213PE PHY on the Raspberry Pi 5 operates in "native EEE" mode - it handles received LPI internally without forwarding it to the MAC over the RGMII interface.

The PHY datasheet says that Native EEE is for MACs that support LPI signalling over RGMII, while AutogrEEEn mode is for MACs that don't support LPI signalling over RGMII. This seems to contradict your statement.

I probably messed up some of the Broadcom terminology here -- never understood why they need their own marketing names for what is just 802.3az. Will keep poking at the PHY registers and might find something useful eventually.