Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1989,24 +1989,17 @@ class NativeUIEngineTest {
val cssWidth = js("innerWidth").toFloat()
main {
assertEquals(View.VISIBLE, native.visibility)
if (mode == NativeUIProjection.CaptureMode.COMPOSITED)
assertEquals(
(20 + iteration * 20) * activity.web.width / cssWidth,
native.translationY,
0.05f,
) // CSS layout quantization stays below 0.05 physical px.
else {
val location = IntArray(2)
val origin = IntArray(2)
native.getLocationInWindow(location)
activity.web.getLocationInWindow(origin)
assertEquals(
(120 + iteration * 20 + (if (authoritativePlacement) 20 else 0)) *
activity.web.width / cssWidth,
(location[1] - origin[1]).toFloat(),
1f,
)
}
val location = IntArray(2)
val origin = IntArray(2)
native.getLocationInWindow(location)
activity.web.getLocationInWindow(origin)
assertEquals(
"Restored view must match the DOM host within one physical pixel",
(120 + iteration * 20 + (if (authoritativePlacement) 20 else 0)) *
activity.web.width / cssWidth,
(location[1] - origin[1]).toFloat(),
1f,
)
}
}
if (auditReuse) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ class NativeUIProjectionContactTest {
js(
"""
JSON.stringify({events:nativeEvents.filter(e=>!e.capturePolicy).slice(-25),
transactions:nativeRequests.filter(e=>!['change','track'].includes(e.request.action)).slice(-30).map(e=>({...e,reply:nativeReplies.find(r=>r.key===e.key)})),
virtualChanges,attributes:[...document.querySelector('#slot').attributes].map(a=>[a.name,a.value]),
images:[...document.querySelectorAll('#slot img')].map(i=>({src:i.src,complete:i.complete,width:i.naturalWidth,opacity:getComputedStyle(i).opacity,display:getComputedStyle(i).display}))})
"""
Expand Down Expand Up @@ -195,14 +194,11 @@ class NativeUIProjectionContactTest {
"""
<meta name="viewport" content="width=device-width,initial-scale=1">
<script>
window.testEvents={};window.nativeEvents=[];window.nativeReplies=[];window.nativeRequests=[];window.virtualChanges=0;
const nativePostMessage=nativeUIEngine.postMessage.bind(nativeUIEngine);
nativeUIEngine.postMessage=message=>{nativeRequests.push({...JSON.parse(message),t:performance.now()});return nativePostMessage(message)};
window.testEvents={};window.nativeEvents=[];window.virtualChanges=0;
const runtime=globalThis.WebViewNativeUIEngineRuntime;
const receive=runtime.receive;
globalThis.WebViewNativeUIEngineRuntime={...runtime,receive:(id,key,value,error)=>{
if(key===null) nativeEvents.push({...value,t:performance.now()});
else nativeReplies.push({key,value,error,t:performance.now()});
if(key===null) nativeEvents.push(value);
return receive(id,key,value,error);
}};
window.Capacitor={
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,6 @@ Use these to check contracts and supported behavior while implementing either ro
[Development and releases](maintenance.md) covers repository structure, local
verification, CI and publication. It is not required for consuming the Engine.

For the optional iOS Window hook, state ownership and timing events, see [pre-interaction capture](advanced/preinteraction-capture.md).
For native contact acquisition, Window integration and timing events, see [pre-interaction capture](advanced/preinteraction-capture.md).

For the default-enabled private-API workaround for iOS cold-launch Liquid Glass capture and how to disable it, see [experimental glass capture](experimental/glass-capture.md). It is not a supported Apple API.
17 changes: 13 additions & 4 deletions docs/advanced/native-handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ success. `frames` may run repeatedly while geometry settles; it must
recheck the requested revision and capture eligibility. It must not mutate layout.
If multiple controls share a native container, `views` must reject a partial set
that would hide uncaptured controls. The Engine cannot infer that relationship.
The renderer also keeps its existing native interaction notifications.
Register the snapshot manager's Window observer and forward its complete events; see [pre-interaction capture](preinteraction-capture.md#window-integration). Do not add duplicate clip-level touch notifications.

`cover` installs prepared pixels before hiding those views. Destination parts are
validated against the cached token and used for both cover and restore, so moving
Expand All @@ -70,7 +70,10 @@ Use these only when your integration owns native placement during handoff. `Nati

Missing captures or placement use the renderer's ordinary fallback. See the [Web request contract](web-projection.md#multiple-fallbacks-in-one-native-transaction) for payloads.

Use `NativeUICoveredSnapshots.observe` to notify the JS listener when interaction starts/ends. On iOS, captures rasterize the visible window with `drawHierarchy(in:afterScreenUpdates:)`, including the background under the clipped native component. Screen snapshot views are display replicas and are not used as bitmap sources. UIKit rasterization can differ from the live compositor for materials such as Liquid Glass; validate the appearance on physical devices with the backgrounds your plugin supports. Cache validity therefore includes layout and the plugin's visual-change notifications. The engine does not capture continuously or guarantee every possible renderer can be captured.
For acquisition before native input, stop-before-measure ordering, invalidation
and timing events, use [pre-interaction capture](preinteraction-capture.md).
Capture limitations, including backdrop-dependent materials, are documented in
[platform limits](../reference/platforms.md).

## Native scroll tracking

Expand All @@ -86,15 +89,21 @@ On iOS, `NativeUIScrollBindings.replace` groups sources by owner and binds only

An established binding rejected by native loses ownership and may be freshly acquired through the **same** update/ACK path. A failed fresh acquisition stays on the fallback until a relevant layout/scroll-end event. Never add an immediate retry loop or replay user actions. A frame/props change does not recalibrate an established coordinate origin.

This uses WKWebView's internal overflow hierarchy without private class names. Missing/ambiguous views, nested scrolling, document scrolling, unsupported transforms, shaped clipping, or a moving unconfirmed owner cannot be bound. Do not claim support solely because a `UIScrollView` exists. A native view may intercept gestures that begin inside it; routing those gestures and SDK accessibility behavior remain renderer-specific.
This uses WKWebView's internal overflow hierarchy without private class names. Missing/ambiguous views, nested scrolling, document scrolling, unsupported transforms, shaped clipping, or a moving unconfirmed owner cannot be bound. Do not claim support solely because a `UIScrollView` exists. For gestures starting inside SDK views, opt into [native scroll hosting](native-scroll-hosting.md) rather than adding another gesture-forwarding implementation. SDK accessibility behavior remains renderer-specific.

### Renderer-only capture on iOS

A `prepare` request may specify `captureMode: 'view'` to draw only the renderer views returned by the view selector, preserving transparent pixels outside their material. The default remains composited window capture. Renderer-only capture can change or omit backdrop-dependent blur and refraction; it does not guarantee transparency inside a glass surface. Missing, hidden or detached renderer views reject preparation. Android and older native binaries keep their existing capture route.

Moving DOM surfaces can provide a fixed viewport transfer layer through `WebProjectionOptions.imageSurface` (`HTMLElement`). The Web projection marks its prepare requests with `imageSurface: 'web'` so Native concealment preserves renderer layout. This changes placement, not the handoff protocol: every surface uses the same temporary Native entry and return copies. During motion, the decoded DOM images live on their source surface so its transform carries them. Translation does not invalidate their reusable capture; dimensions are compared at display-pixel resolution, while actual size and source changes still invalidate it. On iOS, each image part includes optional `anchorBounds` relative to the image crop, in viewport units. Renderers can supply `anchorFrames` for composite controls, such as the first actual toolbar item in a group. Moving targets use these bounds to retain capture padding without mixing capture-time Native coordinates with later DOM placement.

Keep the Native renderer and source ownership alive while DOM images own the pixels. For entry, the common handoff installs prepared Native pixels and conceals the live renderer in the same Native turn, then displays the DOM images on their source surface. During that transfer the DOM image shares the temporary Native copy’s viewport coordinates. After Web painting, the Engine retires the Native copy and returns the DOM image to its source surface. Live translucent controls never sample their own Web image during entry. The optional `layoutFrame(target, part, parts)` adapter supplies current Native anchor placement from the captured cohort; it does not own visibility or handoff timing. Return uses that placement to freeze both prepared copies at the same destination until the accepted Native update. The renderer applies its ordinary Native layout and calls `updated(token, ids, revision)` on the accepted acknowledgement. Only a revision newer than the return request can retire the DOM images. The displayed image remains leased until the reveal acknowledgement or explicit retirement, independently of whether its cached capture is still reusable. No snapshot is captured during either transfer.
Keep renderer and source ownership alive during projection. The common entry and
return protocol above still applies; `imageSurface` changes placement, not image
lifetime. `layoutFrame(target, part, parts)` supplies current anchor placement
from the captured cohort and does not own visibility. Call
`updated(token, ids, revision)` only for the accepted native update; a revision
newer than the return request retires the displayed image. Cache reuse eligibility
and the currently displayed image lease remain separate.

On iOS, use `NativeUICoveredSnapshots.concealPixels(in:)` on the main thread when layout must remain measurable. It masks pixels and disables interaction/accessibility without changing `isHidden` or geometry. Its returned restoration closure is idempotent; overlapping leases restore the original mask and flags only after the last lease ends. Both composited and renderer-only capture use the same Native copy and drawing-completion contract.

Expand Down
Loading
Loading