Define bounded lists

[Felix-El]

Add bounded lists (list<T, ..N>)

Closes #385.

[cpetig]

mostly indentation doubts

[lukewagner]

Thanks! This is looking generally good, a few comments:

[lukewagner]

Pre-existing, but it looks like the grammar already covers this twice: once by using <u32> (unsigned) and once with the (if maxlen > 0). We could also remove the (if maxlen > 0). But should we specify a maximum for maxlen?

[Felix-El]

For u32, zero is valid and the maxlen > 0 checks forbids zero. What would be a legitimate upper bound... i32::MAX?

[lukewagner]

There's already (just recently added) a MAX_LIST_BYTE_LENGTH = 2²⁸-1. That's just bytes, but even still, it seems like a reasonable upper bound.

[Felix-El]

Ok, defined that bound in BINARY.md but where would we check this in definitions.py ?

[cpetig]

Associated with Implement WIT Fixed-Length-Lists

[Felix-El]

Here is another shot at an ABI tweak for efficient passing of fixed/bounded lists. It is already implemented in the commit I just pushed. @cpetig 's ideas with refinement by discussion, my write-up; more versatile / less invasive compared to my previous proposal.

---

Flat ABI for Fixed and Bounded Length Lists

Motivation

The current flat ABI for fixed length lists expands all N element slots
into the core function's parameter/result list.
For large N this causes parameter overflow (exceeding
MAX_FLAT_PARAMS or MAX_FLAT_RESULTS), forcing an early fallback to
memory passing for the entire argument list when even one or two lists
are large.

If the current approach was naively adopted for bounded length lists
(which are still theory at this time), expanded form would even waste
flat-register space on uninitialized elements for partially-populated lists.

A second, independent motivation is that fixed and bounded length lists
are, unlike tuples, indexed at runtime and registers in core WebAssembly
cannot be indexed efficiently. Therefore, the expanded-element flat representation
forces register/memory (de)composition on either guest, adding to overhead.

This proposal replaces the expanded-element flat representation with a
pointer-based one for direct fixed and bounded length lists in parameters —
lists which are eligible for flat representation (not nested, immediately or
transitively, in another list).

Flat representation

Type	Flat representation
list<T> (unbounded)	[ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed)	[ptr] — one pointer to N contiguous elements
list<T, ..N> (bounded)	[ptr, len] — pointer to len contiguous elements + actual length

Memory representation

Type	Memory representation
list<T> (unbounded)	[ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed)	[e0, e1, ...] — N contiguous elements
list<T, ..N> (bounded)	[len*, e0, e1, ...] — N contiguous elements, uninitialized beyond actual length

bounded actual len: smallest integer type that can represent values up to (including) N

Lowering and Lifting

Parameters

Below explanation is for bounded length lists which carry a runtime length.
The flow is similar for fixed length lists - length just would not be passed
as it is statically known.

If all core params fit within MAX_FLAT_PARAMS, they are passed flat, otherwise
we fall back to a in-memory tuple representation (param area). This is not new.

1. Caller

   • assume list L is already constructed in guest memory as L_ptr and L_len (from API use)
   • if passing flat: pass L_ptr and L_len directly, classic unbounded list ownership semantics
   • if passing in memory: move list elements inplace into param area

2. Host, before call forwarded

   • if passing flat:
     • allocate twin list L2 in the exporting guest (optimization: bulk alloc, cache, etc.)
     • move elements between backing memories (import -> export)
     • pass L2_ptr and L2_len = L_len, classic unbounded list ownership semantics
   • if passing in memory:
     • move elements from inbound param area to outbound param area

3. Callee

   • if passing flat: take L2_ptr and L2_len from core params
   • if passing in memory: read L2_len from param area, let L2_ptr point to first inplace element in param area
   • either way: construct list from L2_ptr, L2_len for API use

Results

Memory-passing is enforced when even a single direct fixed/bounded length list is in the result type -
that's a new second switchover reason beyond exceeding MAX_FLAT_RESULTS.
A return area (ret area) is used to forward all results as an in-memory tuple.

1. Caller

   • result area is preallocated with enough room to hold all fixed/bounded length lists inline
   • result area pointer is passed as out core param

2. Host, after call forwarded

   • copy inplace elements from returned export-side ret area into import-side ret area provided by out param

3. Callee

   • construct the list by moving elements from ret area for API use

Fused allocation

When multiple direct lists exist among parameters and passing flat, the host might service
their backing (export) memory needs from a single allocation as an optimization.

[lukewagner]

Thanks for the clear write-up (and sorry for the delay replying)! That all sounds pretty good; just one suggestion: if a component exports a function with type func(p: list<u8, 4>, q: list<u8, 4>), iiuc, the above suggests there would be 2 realloc calls in the callee for p and q. I wonder if we should instead have this work like return values: everything goes into a contiguous param-buffer (so that there's always only 1 realloc call), as-if MAX_FLAT_PARAMS had been exceeded. (A component that imported this function type would work the same as you outlined.) WDYT?

[Felix-El]

if a component exports a function with type func(p: list<u8, 4>, q: list<u8, 4>), iiuc, the above suggests there would be 2 realloc calls in the callee for p and q.

I addressed that in the trailing "fused allocation" section - yes it should be done but we need not guarantee that by design. In practice, the host can alloc one big chunk for everything and hand out pointers to subobjects (list1, list2, list3, etc. correctly aligned and sized of course). The allocation will continue to be owned by the host.

I wonder if we should instead have this work like return values: everything goes into a contiguous param-buffer (so that there's always only 1 realloc call), as-if MAX_FLAT_PARAMS had been exceeded.

Passing all params in a param-area, even if callee side only, feels a bit inefficient. We should try to utilize registers for non-list value passing.
Additionally it would break the symmetry between caller core func signature and callee core func signature much further than just the current return-area result vs. out pointer story.

But that's just my 2 cents...

PS: definitions.py modifications compared between the previous/initial proposal and this one, I have to admit this one felt way less intrusive to implement (thinking of differences in definitions.py in 91a0eae vs 1fb578e over the predecessor.)

[lukewagner]

I addressed that in the trailing "fused allocation" section - yes it should be done but we need not guarantee that by design.

I think the host can do that when given a list, b/c the host can do whatever, but if we're talking about lowering a list into the guest, the realloc calls would have to happen exactly when specified, so no fusion would be possible when lowering a fixed-/bounded-length list into guest code.

Passing all params in a param-area, even if callee side only, feels a bit inefficient.

I suppose, but I would expect this to be amortized by the cost of at least 1 realloc. FWIW, I think the way realloc would get removed in a lazy-lowering future is that, when parameters aren't flat (including when there are any fixed- or bounded-length lists), the callee entry function would call a (new) task.start built-in passing a pointer to a buffer to receive all the parameters, which is sortof analogous.

[flelchuk]

I think the host can do that when given a list, b/c the host can do whatever, but if we're talking about lowering a list into the guest, the realloc calls would have to happen exactly when specified, so no fusion would be possible when lowering a fixed-/bounded-length list into guest code.

Sorry if I'm repeating myself. I might be missing some crucial C-M insight...

• This proposal leaves most of lifting/lowering as-is, including dynamic lists.
• The only special new treatment is for fixed/bounded lists.
  • Memory path: all params in param area - caller passes to host, host copies*, callee receives param area. Fixed/bounded lists are inlined.
  • Register path: fixed/bounded lists enter the host as 1-2 slots each. The host allocates export-side memory possibly as a single longlived allocation holding elements of all lists, host copies* elements and forwards to the callee also as 1-2 slots each. The callee uses memory from host-given address directly, no further realloc needed
• Return is always inlined in ret area.

What obligation would make fusion impossible?