shared/usb: correctly report the SD LUN as removable media

[mikeysklar]

What was wrong

The SD card exposed as a USB MSC LUN doesn't mount on macOS. ioreg shows the SCSI nub appears but never progresses to publishing an IOMedia. Linux handles the same firmware fine.

Why: CircuitPython was telling the host "this LUN's medium is always present" for every LUN, including the SD card. On macOS that means "skip TEST_UNIT_READY polling" — so if the SD wasn't ready at the one-shot enumeration probe, macOS never re-checked and gave up on the LUN. Linux ignored the hint and kept polling, which is why it worked there.

@hathach documented this exact macOS behavior back in #6555; the per-LUN distinction wasn't applied at the time.

What changed

Report the SD LUN as removable media, report internal flash and SAVES as non-removable. macOS now keeps polling TUR on the SD LUN and correctly picks up card insertion and removal.

~10 lines in supervisor/shared/usb/usb_msc_flash.c — per-LUN branch on the existing PREVENT_ALLOW_MEDIUM_REMOVAL handler. No RAM or flash cost.

Tested on

Board	Result
Feather RP2040 Adalogger (with SD pin defines from #10968)	SD mounts first try on macOS 26.x
Metro RP2040 (with SD pin defines from #10968)	SD mounts
Metro RP2350	SD mounts, no regression
Fruit Jam (3-LUN: flash + CPSAVES + SD)	All three mount; CPSAVES correctly stays non-removable
Linux (Ubuntu 24.04)	/dev/sdc + /dev/sdd correct sizes, no regression

Windows not tested.

Related

• Fixes Metro RP2040: SD card LUN never probed by macOS via USB MSC (works on RP2350) #10965
• Companion: Mount SD card at filesystem init so macOS can see it via USB MSC #10963 (automount at filesystem init)
• Companion: add DEFAULT_SD_* for Metro RP2040 and Feather RP2040 Adalogger #10968 (SD pin defines for Metro RP2040 + Feather RP2040 Adalogger)
• Historical context: Multiple luns #6555
• Supersedes the workaround in bump MSC buffer for RP2040 boards exposing SD-over-USB #10964 (closed)

Credit to @dhalbert for pointing at #6555 and @hathach for the original analysis.

[tannewt]

I think we want them all removable.

[mikeysklar]

You're right, thanks — simplifying to apply the unsupported reply to all LUNs.

[mikeysklar]

Quick cross-board, cross-OS regression test of this PR combined with #10963 on main (which already has #10968 merged as a37aaaf).

Board	Arch	Host OS	LUNs enumerated	CIRCUITPY mounts	SD mounts	Readable	Regression?
Metro RP2040 (onboard SD)	RP2040	macOS 26.x	2	✅	✅ (ADALOGGER)	✅	None
Metro RP2040 (onboard SD)	RP2040	Ubuntu 24.04	2	✅	✅ (ADALOGGER)	✅	None
Metro RP2350 (onboard SD)	RP2350	macOS 26.x	2	✅	✅ (ADALOGGER)	✅	None
Metro RP2350 (onboard SD)	RP2350	Ubuntu 24.04	2	✅	✅ (ADALOGGER)	✅	None

Notes:

• Same 8 GB microSD card moved between the two boards (both use their built-in onboard microSD slot — no Adalogger FeatherWing involved)
• Stack built from adafruit/circuitpython main + cherry-picked Mount SD card at filesystem init so macOS can see it via USB MSC #10963 + shared/usb: correctly report the SD LUN as removable media #10967 (latest revision — reply ILLEGAL_REQUEST to PREVENT_ALLOW on all LUNs per @tannewt's review)
• macOS: ioreg shows both LUNs publishing IOMedia with Removable = Yes
• Linux: /sys/block/<dev>/removable = 1 for both LUNs; udisks2 auto-mounts both volumes; SD contents (pre-existing LoRa logger files) read back intact
• Firmware string on both boards: 10.2.0-4-g834be2c430

[dhalbert]

It would be good to test on Windows as well, and also on boards that are not RP2xxx, such as a PyPortal, and a Metro ESP32-S3 (onboard SD socket), etc. I'm assuming you don't have a Windows box or you would have done so. I could test some of these on Windows.

[bablokb]

I don't think that reporting /saves as removable is a good idea. Host polling seems to considerable slow down CircuitPython (see #10733). This is why I usually disable CIRCUITPY_SDCARD_USB. I never had problems with the saves-partition though. And from a logical perspective, I don't think it should be removable (of course you can umount it, but there are better ways to shoot yourself in the foot).

[dhalbert]

I don't think that reporting /saves as removable is a good idea.

I agree with this.

[mikeysklar]

Thanks both.

On /saves: agreed. CPSAVES is a partition of the onboard QSPI flash (same chip as CIRCUITPY, just carved at offset 0), not physically removable — TUR polling on it is wasted traffic, and #10733 is a real concern I don't want to aggravate.

Comparing the two versions for the 3-LUN case (Fruit Jam: CIRCUITPY + SAVES + SD):

Current PR (all LUNs reply ILLEGAL_REQUEST):
  CIRCUITPY : TUR TUR TUR TUR TUR TUR ...   ← wasted (can't be re-inserted)
  SAVES     : TUR TUR TUR TUR TUR TUR ...   ← wasted (can't be re-inserted)
  SD        : TUR TUR TUR TUR TUR TUR ...   ← needed (catches card re-insert)

Per-LUN version (SD only — original PR):
  CIRCUITPY : TUR ─ [OK → host stops polling] ──────────
  SAVES     : TUR ─ [OK → host stops polling] ──────────
  SD        : TUR TUR TUR TUR TUR TUR ...

The per-LUN version keeps identical user-visible behavior (eject buttons from RMB=1 are unchanged, SD re-insert still auto-remounts) but drops 2/3 of the poll traffic on 3-LUN boards.

@tannewt — OK to revert to per-LUN given the polling-cost point?

On Windows / non-RP2xxx boards: I don't have a Windows host, but I do have a PyPortal and a Metro ESP32-S3 here — happy to run macOS + Linux checks on both. @dhalbert — gladly hand off Windows coverage to you. Let's agree on which version of the PR we're testing against (current / per-LUN revert) before I spin up the non-RP2xxx builds, so we're all benching the same stack.

[tannewt]

I don't think the polling cost issue has been proven out. The linked issue says it occurs even when a host isn't connected. When I've looked at traces from Linux it only queried the drive state once a second. That'd shouldn't be too much overhead.

Although the flash can't be physically removed it is advantageous to "eject" it. The host flushes before doing this and then sends a signal to the device to eject. We have to remember this state so it doesn't immediately pop up again. Now, we can do storage.remount() to change writability and then "uneject" the LUN. So, it is good to tell the host it is removable.

[mikeysklar]

Ran bablokb's chaos benchmark on a Metro RP2040 to check the polling overhead question. 5 passes each, SD auto-mounted by firmware, USB host connected (macOS):

Firmware	chaos avg (5 passes)
All-removable (this PR, current)	0.225s
Per-LUN (SD-only removable)	0.212s

~6% difference, ~13ms absolute. The 3–4x slowdown in #10733 appears to be a separate CP9→CP10 regression unrelated to the removable flag — current code shows no significant overhead either way.

Agreed on keeping all LUNs removable for the eject/storage.remount() benefit.

[mikeysklar]

Tested 4 boards across 4 architectures with firmware built from 10.2.0-4-g834be2c430 (main + #10963 + #10967).

No user code — SD mounted via automount_sd_card() defaults on every board.

Board	Arch	macOS 26.x	Ubuntu 24.04
Metro RP2040	RP2040	✅	✅
Metro RP2350	RP2350	✅	✅
Memento	ESP32-S3	✅	✅
PyPortal	SAMD51	✅	✅

macOS: both CIRCUITPY and SD appear as separate volumes, eject/reinsert works cleanly.

Linux: both mount read-only as expected (USB MSC path).

No regressions observed on any board.

[bablokb]

@mikeysklar

Ran bablokb's chaos benchmark on a Metro RP2040 to check the polling overhead question. 5 passes each, SD auto-mounted by firmware, USB host connected (macOS):

Did you average from POR or did you run 5 iterations in one go? In the latter case, averaging 5 passes is not necessarily relevant. Many use-cases for microcontrollers use a "startup-work-deepsleep" scenario. So the results for the first pass are more interesting. You can see this in the tests that Dan and I ran in #10733.

Anyway, 6% difference is relevant because this adds up with all the other things that slow down CircuitPython. So I would really like to see this "feature" compile-time configurable.

[mikeysklar]

@bablokb — you were right on both counts.

Methodology: the 5 passes were in-loop from a single soft-reset, not power-cycled. That captured steady state only. Re-ran with physical power cycles (unplug/replug → full USB re-enumeration) on the same Metro RP2040:

Condition	chaos.bm_run timing	iter-to-iter gap
Soft-reset (USB persistent)	0.210s flat	0.8s
Power cycle, iters 1–8 (~t=5–20s post-boot)	0.596–0.604s	2.0s
Power cycle, iters 9–10 (~t=21–23s post-boot)	0.211s	1.1s

First-pass overhead: ~2.86× (not 6%). The slow phase lasts ~20s after USB re-enumeration then drops sharply. For a startup-work-deepsleep program that finishes in under 20s, every run is ~3× slower — exactly the scenario you described.

The 6% figure was real but measured the wrong thing: it's the steady-state delta once the host has settled its polling. First-boot cost is what matters for your use case.

Compile-time config: the data now supports this more strongly. The SD LUN has to keep replying ILLEGAL_REQUEST for the macOS fix to work, but CIRCUITPY/SAVES don't need it for the fix itself. A flag like CIRCUITPY_USB_MSC_REMOVABLE_INTERNAL (defaulting to 1) would let boards opt back to per-LUN behavior where the storage.remount() eject workflow isn't needed.

@tannewt — given the ~20s first-boot overhead on the all-LUN path, does a compile-time opt-out make sense, or does the eject benefit on CIRCUITPY/SAVES outweigh this for typical use?