fix(dialog): fire onCancel when AlertDialog is dismissed via Escape key

[arham766]

Closes #1773

Re-implements the approach from #10136 (which was approved but closed over an unsigned CLA — CLA is signed on my end), and covers the Spectrum 2 AlertDialog as requested in that review.

Problem

Pressing Escape closes an AlertDialog via the overlay (state.close() directly), so the onCancel callback — wired only to the cancel button's onPress — never fires. Consumers can't distinguish a keyboard dismissal from a confirmation.

Fix

V3 (@adobe/react-spectrum): AlertDialog provides a DialogContext with an added onKeyDown that calls onCancel on Escape (chained with any existing context handler, guarded against IME composition like the overlay's own keyboard handling). Dialog merges the context's onKeyDown into dialogProps after useDialog — required because useDialog/filterDOMProps strips event handlers, so the handler can't ride through the existing mergeProps(contextProps, props) path. The overlay still closes the dialog exactly as before; nothing is prevented or stopped.

S2 (@react-spectrum/s2): keyboard events can't reach the RAC Dialog's section through props or context (filterDOMProps(props, {global: true}) deliberately excludes them), so S2 Dialog accepts a /** @private */ onKeyDown (same precedent as isSticky on S2 TableView's CellProps) and attaches it via RAC Dialog's public render prop — sectionProps carries ref/role/aria through, so it's a plain React handler on the real <section role="alertdialog">. The render path is only used when the handler is present. Nested-overlay Escapes stop propagation in react-aria, so they won't misfire onCancel.

One behavior note for review: if a V3 DialogTrigger has isKeyboardDismissDisabled, Escape won't close the dialog but onCancel still fires (the flag isn't visible through DialogContext). #10136 had the same behavior. Happy to thread the flag through the context if you'd rather suppress it. (Not reachable in S2 — AlertDialog doesn't expose the prop.)

✅ Pull Request Checklist:

• Included link to corresponding React Spectrum GitHub Issue.
• Added/updated unit tests and storybook for this change (for new code or code which already has tests).
• Filled out test instructions.
• Updated documentation (if it already exists for this component).
• Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

1. Open an AlertDialog with an onCancel handler (e.g. the AlertDialog story, or the S2 stackblitz from onCancel should fire in AlertDialog if Escape is pressed #1773 (comment)) and press Escape — onCancel fires once and the dialog closes.
2. Press the cancel button — onCancel still fires exactly once (no double-fire).
3. Omit onCancel — Escape still closes the dialog with no errors.

Unit tests: 4 new cases in packages/@adobe/react-spectrum/test/dialog/AlertDialog.test.js, 3 in packages/@react-spectrum/s2/test/AlertDialog.test.tsx (Escape fires once, no-onCancel safety, button no-double-fire, and trigger-integration close). The Escape tests fail without the source change.

🧢 Your Project:

Personal contribution

[snowystinger]

Suggested change

	onCancel();
	onCancel();
	e.stopPropagation();
	e.preventDefault();

[arham766]

Applied — with the consequence that consuming the event means the overlay no longer closes the dialog, so the handler now runs the full cancel-button path (onClose() then onCancel()). Escape is now literally equivalent to pressing the cancel button, which I think is the right semantic anyway.

One behavior change to rule on: with isKeyboardDismissDisabled on the trigger, Escape previously kept the dialog open (while leaking onCancel — flagged in the PR description); now it closes it, because the flag isn't visible from DialogContext. Arguably isKeyboardDismissDisabled + AlertDialog is contradictory once Escape ≡ cancel, but happy to thread the flag through the context instead if you'd rather preserve it.

[snowystinger]

I think it's probably better to update RACDialog->useDialog to accept onKeyDown?

This wasn't in the original PR, we didn't have the render prop back then, how'd you arrive at this solution?

[arham766]

Done — useDialog now accepts onKeyDown (doc-commented on AriaDialogProps, so the docs table picks it up), RAC Dialog inherits it with zero code change since keyboard events aren't in GlobalDOMEvents, and the S2 render function is deleted; onKeyDown just flows through props. Added a RAC-level test that the handler reaches the section.

How I got to render: I traced why props/context didn't work — RAC filters with filterDOMProps(props, {global: true}) and GlobalDOMEvents deliberately excludes keyboard events — and reached for the narrowest hook that didn't touch public types. First-class support is clearly better now that you've blessed the API addition.

One S2 detail: in a DialogContainer (no trigger state at AlertDialog's position in the tree), the handler falls back to not consuming the event so the container's overlay still closes the dialog; onCancel fires exactly once either way.

[snowystinger]

Would this be easier if onClose was expanded to give the reason?
#2192

[arham766]

Yes — a close reason on onClose would make this whole dance trivial (map escape/click-outside → cancel at the AlertDialog level) and would also solve the isKeyboardDismissDisabled visibility problem cleanly, since the overlay would stay the single owner of dismissal. That's a wider API change across DialogContext/overlay state though, so I kept this PR targeted. I'd be glad to take #2192 on as a follow-up if you want it.