Skip to content

the interpreter's [extern] call marshals through an arm64 layout past six arguments, and the SystemV wrapper's float mask and extra-table result class are fixed - #3983

Merged
borisbat merged 1 commit into
masterfrom
bbatkin/dasbind-apple-arm64
Sep 10, 2026
Merged

the interpreter's [extern] call marshals through an arm64 layout past six arguments, and the SystemV wrapper's float mask and extra-table result class are fixed#3983
borisbat merged 1 commit into
masterfrom
bbatkin/dasbind-apple-arm64

Conversation

@borisbat

@borisbat borisbat commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Behavior change: an interpreted [extern] with a float in position 6 on x86-64 Linux now refuses to compile unless its shape is listed in the extra wrapper table - it was silently miscalled before; the five dasOpenGL functions with that shape are listed here.

Why. The interpreter calls an [extern] through a generated C prototype that spells every stack argument as an 8-byte slot and masks six float positions, which is wrong on arm64 and was mis-indexed on SystemV: on every Mac LLVMDIBuilderCreateFunction's tail arguments read one slot late (the debug-info test red since #3981), a double in position 4 or 5 rode an integer register on every SystemV host, the extra table's hand-listed wrappers read rax where xmm0 held the result, and a float& or double& argument was classed by its base type and passed as a float instead of a pointer. The JIT emits typed calls and was never affected.

What changes.

  • On Apple and Linux aarch64 a call with more than six arguments marshals through an Arm64Layout (packed stack on Apple, 8-byte slots on the GNU ABI) and the das_arm64_call assembly trampoline; layouts live for the process, since a re-applied extern replaces the function object under a live call node.
  • The wrapper choice masks all six SystemV register positions, not four, and classes a ref as a pointer whatever it points to.
  • The extra table's result class matches the main table's, and a float in position 6 routes to it instead of being dropped.
  • tests/dasbind binds a probe library built into lib/; each function sums its arguments under distinct multipliers, so one argument in the wrong register or slot changes the total.

Observable behavior.

  • modules/dasLLVM/tests/llvm_jit_debug_info.das on Apple arm64: red -> green.
  • An interpreted extern with a double in position 4 or 5 on Linux, Mac or Android: wrong value -> right value.
  • A float in position 6 or 7 on Linux/Android arm64: wrong value -> right value.
  • The hand-listed GL extra wrappers returning GLenum on x86-64 Linux: garbage -> the value.
  • An interpreted extern with a float const& argument on Windows or x86-64 Linux: a crash on a garbage pointer -> the value.
  • A tests walk on a 64-bit tree without the probe library: fails the suite instead of skipping it.

Where to look. module_builtin_dasbind.cpp: the trampoline and computeArm64Layout, getWrapper's mask loop, the two res computations; generate_x86_64_calls.das for the result-class constants.

Validation, claims, ledger

Validation

  • Negative controls CI cannot produce: the suite against master's dasbind sources is red on every ABI - 3 of 4 original arms on Apple arm64, 4 of 6 arms on Linux arm64 (the position-6 arm passes there only by register residue unless an integer follows the double, which the probe now does), refused at compile time on x86-64.
  • Pre-push runs of the final tip on all three ABIs (Apple arm64 M5, Linux arm64 Debian 12 VM, x86-64 Debian with clang-19, Release and Debug): tests/dasbind 11/11 interpreted and JIT, modules/dasLLVM/tests 110/110; modules/dasOpenGL/opengl/opengl_func.das compiles on x86-64 with zero refusals after listing the five shapes. The reference-class defect was found by CI's Linux Debug lane on this PR's own probe and reproduced on x86-64 Release too.
  • Mutation controls the TDD audit named, each red as predicted: the unrounded stack size (SIGBUS on Apple arm64), the old 1<<7 extra-table threshold (the position-6 arm on x86-64), a dropped extra row (compile refusal on x86-64), and master's dasbind sources against the final suite (compile refusal on every ABI).
  • The local preflight fast tier is red only on this box's pre-existing Vulkan-less compile sweep; the skipped lanes were run by name.

Not done

  • A program with an [extern] does not survive dastest's --ser/--deser round trip: the reader never applies the annotation, so the __dasbind__ function it names is missing from the dasbind module and the stream cascades. Pre-existing; the suite skips that sweep.
  • Windows arm64 under MSVC keeps the four-position float mask (no CI lane, no box).
  • An x86-64 SystemV trampoline would retire the hand-listed extra table; the table stays.

Copilot AI lite review requested due to automatic review settings September 10, 2026 00:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new src/builtin/ARCHITECTURE.md section contains an inaccurate statement about when tests/dasbind is skipped vs when the probe is built, and it should be corrected to match the actual build/skip conditions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR fixes ABI mismatches in the interpreter path for [extern] calls: arm64 now uses an explicit layout + trampoline for calls that run past the first six arguments, and the x86-64 SystemV wrapper selection/extra-table handling is corrected (including result-class handling).

Changes:

  • Add an arm64-specific marshaling path (Arm64Layout + das_arm64_call) for interpreted [extern] calls with >6 arguments on aarch64 Apple/Linux.
  • Fix SystemV wrapper selection (mask all six register positions), correct extra-wrapper result class handling, and extend the extra wrapper table for newly-supported shapes.
  • Add a dedicated tests/dasbind probe shared library + ABI tests (and integrate into CMake and the test/AOT suite selection).
File summaries
File Description
tests/REVIEW.md Adds a tests checklist item to prevent weakening the new dasbind ABI coverage.
tests/dasbind/test_extern_abi.das New interpreter/JIT ABI probes validating register/stack argument placement and re-apply behavior.
tests/dasbind/probe/dasbind_probe.cpp C++ probe library implementing deterministic “sum under multipliers” signatures for ABI verification.
tests/dasbind/CMakeLists.txt Builds the probe shared library and places it where dasbind’s loader can find it.
tests/dasbind/_reapply_fixture.das Fixture that re-applies the extern in-process to validate layout lifetime assumptions.
tests/aot/CMakeLists.txt Adds dasbind to the AOT suites list (with the test file itself marked no_aot).
tests/.das_test Gates visiting tests/dasbind based on pointer size to avoid silent skips on 32-bit.
src/builtin/systemV_64_extra_wrapper.inc Fixes extra wrappers’ return types/result-class handling and adds new extra-wrapper shapes.
src/builtin/REVIEW.md Extends review rules to cover any addExtern* binds and requires ARCHITECTURE updates for wrapper/layout changes.
src/builtin/module_builtin_dasbind.cpp Implements arm64 layout/trampoline path and fixes SystemV wrapper mask + extra-table threshold/result-class wiring.
src/builtin/generate_x86_64_calls.das Updates result-class constants and extends the extra wrapper generator inputs.
src/builtin/ARCHITECTURE.md Documents the interpreter [extern] call mechanism and the new arm64 layout/trampoline behavior.
CMakeLists.txt Adds tests/dasbind to the build so the probe library is produced in CI/local builds.
Review details
  • Files reviewed: 13/13 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/builtin/ARCHITECTURE.md Outdated
Copilot AI review requested due to automatic review settings September 10, 2026 00:29
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 684bf8e to 05e0c04 Compare September 10, 2026 00:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level cross-ABI extern-call mechanics (including inline arm64 assembly) where small mistakes can cause subtle miscalls or crashes across platforms.

Review details

Suppressed comments (2)

tests/dasbind/test_extern_abi.das:69

  • equal takes expected first and got second; swap arguments here so failures report the computed extern-call value as "got".
        t |> equal(probe_ref_args(fv, 1l, 2l, 3l, 4l, 5l, 6l, 7l, iv), 269l)

tests/dasbind/test_extern_abi.das:80

  • equal takes expected first and got second; swap arguments here so failures report the computed extern-call value as "got".
        t |> equal(probe_stack14(int8(1), int16(2), 3, 4l, 0.5, 0.25lf, 6, int8(7), int8(8), int16(9), 10, 11l, true, 13), 5576l)
  • Files reviewed: 13/13 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread tests/dasbind/test_extern_abi.das
Copilot AI review requested due to automatic review settings September 10, 2026 00:54
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 05e0c04 to 285d5f6 Compare September 10, 2026 00:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new arm64 return-path currently relies on an implementation-defined unsigned→signed conversion when preserving raw return bits, which should be made explicitly bit-preserving.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/builtin/module_builtin_dasbind.cpp Outdated
Copilot AI review requested due to automatic review settings September 10, 2026 01:01
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 285d5f6 to 98afff8 Compare September 10, 2026 01:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level cross-platform calling-convention behavior (including new inline arm64 assembly), which warrants final human review despite the added test coverage.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/builtin/generate_x86_64_calls.das Outdated
Copilot AI review requested due to automatic review settings September 10, 2026 01:12
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 98afff8 to 196d790 Compare September 10, 2026 01:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

tests/dasbind/_reapply_fixture.das should be marked options no_aot (it can be picked up by the AOT sweep once dasbind is added as a suite, and its [extern] bindings are noAot).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread tests/dasbind/_reapply_fixture.das
Copilot AI review requested due to automatic review settings September 10, 2026 01:20
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 196d790 to ac4f2d3 Compare September 10, 2026 01:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level cross-platform ABI marshalling and introduces inline arm64 assembly, which warrants final human review despite added tests.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread tests/.das_test Outdated
Copilot AI review requested due to automatic review settings September 10, 2026 01:30
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from ac4f2d3 to 6e80bfa Compare September 10, 2026 01:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level cross-platform ABI/assembly call mechanics for interpreted [extern] and should get final human validation despite the added test coverage.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/builtin/module_builtin_dasbind.cpp
Copilot AI review requested due to automatic review settings September 10, 2026 01:37
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 6e80bfa to 636a55b Compare September 10, 2026 01:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The changes touch cross-ABI call marshalling and introduce inline arm64 assembly, which warrants final human verification despite added probe coverage.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 10, 2026 02:05
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 636a55b to 4b32b90 Compare September 10, 2026 02:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level ABI marshalling/trampoline behavior across multiple platforms and warrants a final human review for ABI correctness on all target environments.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/builtin/ARCHITECTURE.md
Copilot AI review requested due to automatic review settings September 10, 2026 02:49
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 4b32b90 to 4f058ad Compare September 10, 2026 02:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new arm64 layout path misclassifies float&/double& return types as FP results (baseType-only check), which would route the return through d0 instead of x0 and can yield incorrect results.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread src/builtin/module_builtin_dasbind.cpp Outdated
Comment thread src/builtin/generate_x86_64_calls.das
… six arguments - the generated C wrappers spell every stack argument as an 8-byte slot and mask six float positions, while Apple packs stack arguments at natural size and every arm64 has eight float registers, so LLVMDIBuilderCreateFunction's tail arguments read one slot late on every Mac (the debug-info test red since #3981) and a float in position 6 or 7 rode an integer register on Linux and Android arm64; the layout maps each argument to x0-x7, d0-d7 or a byte offset of a stack image (packed on Apple, slotted on the GNU ABI), an assembly trampoline copies the image and calls, and layouts live for the process beside the bound-function table since a re-applied extern replaces the function object an earlier context's node still names; two table defects fall with it - the wrapper choice read the float mask of the first four SystemV positions instead of six, so a double in position 4 or 5 went through an integer register on every SystemV host, and the extra table's result class was inverted against the main table's, so its hand-listed GL wrappers returned rax where the callee's xmm0 held the value - and a float in position 6 now routes to the extra table instead of being dropped by the six-bit mask; tests/dasbind probes every shape against a library of functions that sum their arguments under distinct multipliers, built into lib/ by its own CMake, gated by the .das_test walk on the library's presence and a 64-bit host, registered in the AOT suites as a no_aot file, with its two SystemV shapes listed in the generator; src/builtin/ARCHITECTURE.md sec.3 carries the mechanism and REVIEW.md the three-box duty

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 10, 2026 03:00
@borisbat
borisbat force-pushed the bbatkin/dasbind-apple-arm64 branch from 4f058ad to 6fbe5ca Compare September 10, 2026 03:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level cross-platform ABI marshaling and introduces new inline arm64 assembly/trampoline behavior, which warrants final human review despite the added probe coverage.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@borisbat
borisbat merged commit c38177f into master Sep 10, 2026
51 of 52 checks passed
@borisbat
borisbat deleted the bbatkin/dasbind-apple-arm64 branch September 10, 2026 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants