-
Notifications
You must be signed in to change notification settings - Fork 5.2k
[wasm][coreclr] 2nd part of call helpers generator, reverse calls #122007
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[wasm][coreclr] 2nd part of call helpers generator, reverse calls #122007
Conversation
|
Tagging subscribers to this area: @BrzVlad, @janvorli, @kg |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements the second part of the call helpers generator for WASM and CoreCLR, focusing on reverse thunks for methods decorated with UnmanagedCallersOnlyAttribute. The implementation enables native code to call managed methods through generated C++ thunks, using a dual hash-based lookup system with primary keys (based on assembly FQN and method token) and fallback keys (based on method properties) to handle assembly changes during trimming.
Key changes:
- Adds new
PInvokeTableGeneratorandPInvokeCollectorclasses for both mono and coreclr runtimes - Implements hash-based reverse thunk lookup with fallback mechanism in the CoreCLR VM
- Generates C++ reverse thunks with lazy MethodDesc lookup and proper argument marshaling
Reviewed changes
Copilot reviewed 8 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
src/tasks/WasmAppBuilder/mono/PInvokeTableGenerator.cs |
New generator for mono runtime that creates PInvoke tables and native-to-interp entry functions |
src/tasks/WasmAppBuilder/mono/PInvokeCollector.cs |
New collector for mono that scans assemblies for PInvoke and callback methods |
src/tasks/WasmAppBuilder/coreclr/PInvokeTableGenerator.cs |
New generator for CoreCLR that creates reverse thunks with dual-key hash table entries |
src/tasks/WasmAppBuilder/coreclr/PInvokeCollector.cs |
New collector for CoreCLR that scans assemblies and builds callback metadata |
src/tasks/WasmAppBuilder/coreclr/SignatureMapper.cs |
Adds TypeToNameType helper for converting types to name strings in generated code |
src/tasks/WasmAppBuilder/WasmAppBuilder.csproj |
Updates target framework and output path configurations for the generator task |
src/coreclr/vm/wasm/helpers.cpp |
Implements dual-key lookup mechanism with primary and fallback hash tables |
src/coreclr/vm/wasm/callhelpers.hpp |
Adds fallbackKey field to ReverseThunkMapEntry structure |
src/coreclr/vm/wasm/callhelpers-reverse.cpp |
Generated reverse thunks with lazy MethodDesc lookup for UnmanagedCallersOnly methods |
src/coreclr/vm/wasm/callhelpers-interp-to-managed.cpp |
Removes duplicate entry in thunk table |
…sm-callhelpers-generator-2nd-part
| // Try primary key, it is based on Assembly fully qualified name and method token | ||
| const ReverseThunkMapValue* thunk; | ||
| if (table->Lookup(key, &thunk)) | ||
| { | ||
| return thunk; | ||
| } | ||
|
|
||
| // Try fallback key, that is based on method properties and assembly name | ||
| // The fallback is used when the assembly is trimmed and the token and assembly fully qualified name | ||
| // may change. | ||
| table = VolatileLoad(&reverseThunkFallbackCache); | ||
| if (table == nullptr) | ||
| { | ||
| table = CreateReverseThunkHashTable(true /* fallback */); | ||
| } | ||
|
|
||
| key = CreateFallbackKey(pMD); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if we get a key collision? Is it mechanically prevented from happening somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking deeper, I'm concerned that we're using a low quality 32-bit hash here, so I would hope that at least if we get a hash collision during thunk lookup, it will fail deterministically with a clear error message so we can go back and fix it. Ideally we would use a more robust 64-bit hash to make the odds of a collision way lower, or do an actual string check. But I don't know if thunk lookup is on a hot path to the point that we can't afford it.
| string.Equals(EntryPoint, other.EntryPoint, StringComparison.Ordinal) && | ||
| string.Equals(Module, other.Module, StringComparison.Ordinal) && | ||
| string.Equals(Method.ToString(), other.Method.ToString(), StringComparison.Ordinal); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should override object.Equals too just to be safe
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd also prefer to see GetHashCode live here instead of in the comparer.
| => other != null && | ||
| string.Equals(EntryPoint, other.EntryPoint, StringComparison.Ordinal) && | ||
| string.Equals(Module, other.Module, StringComparison.Ordinal) && | ||
| string.Equals(Method.ToString(), other.Method.ToString(), StringComparison.Ordinal); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be checking WasmLinkage too?
|
|
||
| internal sealed class PInvokeTableGenerator | ||
| { | ||
| private readonly Dictionary<Assembly, bool> _assemblyDisableRuntimeMarshallingAttributeCache = new(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: PInvokeCollector has this cache too, can we unify them?
| using TempFileName tmpFileName = new(); | ||
| using (var w = new JoinedStringStreamWriter(tmpFileName.Path, false)) | ||
| { | ||
| // WASM-TODO: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An explanation of why it's commented out or when we can uncomment it would be cool
| // Special case for __Internal and * modules to indicate static linking without specifying the module | ||
| modules.Add(pinvoke.Module, pinvoke.Module); | ||
| Log.LogMessage(MessageImportance.Low, $"Adding module {pinvoke.Module} for static linking"); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we have an else here that logs a message that this pinvoke was ignored?
|
|
||
| #pragma warning disable CA1067 | ||
| #pragma warning disable CS0649 | ||
| internal sealed class PInvoke : IEquatable<PInvoke> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is precisely what C# records were designed for.
/cc @agocke
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, these are a good candidate for record class. Note that records will check all members when comparing, which might be a problem for Skip and WasmLinkage since right now we don't compare them.
| #include <mono/utils/details/mono-error-types.h> | ||
| #include <mono/metadata/assembly.h> | ||
| #include <mono/utils/mono-error.h> | ||
| #include <mono/metadata/object.h> | ||
| #include <mono/utils/details/mono-logger-types.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these correct for coreclr?
| var imports = pinvokes | ||
| .Where(l => l.Module == module && !l.Skip) | ||
| .OrderBy(l => l.EntryPoint, StringComparer.Ordinal) | ||
| .GroupBy(d => d.EntryPoint, StringComparer.Ordinal) | ||
| .Select(l => $"{{\"{EscapeLiteral(l.Key)}\", {CEntryPoint(l.First())}}}, // {ListRefs(l)}{w.NewLine} ") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could use a comment. Is the order and grouping important?
| var moduleImports = new Dictionary<string, List<string>>(); | ||
| foreach (var module in modules.Keys) | ||
| { | ||
| static string ListRefs(IGrouping<string, PInvoke> l) => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Local function should be at the end of the method.
| return candidates | ||
| .Skip(1) | ||
| .Any(c => !TryIsMethodGetParametersUnsupported(c.Method, out _) && | ||
| c.Method.GetParameters().Length != firstNumArgs); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please comment on the intent here.
|
|
||
| // FIXME: System.Reflection.MetadataLoadContext can't decode function pointer types | ||
| // https://github.com/dotnet/runtime/issues/43791 | ||
| private static bool TryIsMethodGetParametersUnsupported(MethodInfo method, [NotNullWhen(true)] out string? reason) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please flip this logic to be TryIsMethodGetParametersSupported and then in the call you can negate it. Most try logic is about valid state, unsupported is not "valid".
| private static string? EscapeLiteral(string? input) | ||
| { | ||
| if (input == null) | ||
| return null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| private static string? EscapeLiteral(string? input) | |
| { | |
| if (input == null) | |
| return null; | |
| private static string EscapeLiteral(string? input) | |
| { | |
| if (input == null) | |
| return string.Empty; |
Is it possible to remove the string? by replacing with string.Empty?
| public static bool IsFunctionPointer(Type type) | ||
| { | ||
| object? bIsFunctionPointer = type.GetType().GetProperty("IsFunctionPointer")?.GetValue(type); | ||
| return (bIsFunctionPointer is bool b) && b; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this needed? I'm assuming we can target .NET 10.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like legacy code that was copied over from the mono based tasks that needed to support old SRM
| } | ||
| } | ||
|
|
||
| w.WriteLine( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I'm aware that the call sites for this seem to be commented out but we probably should try to avoid checking in references to mono stuff if we can. On the other hand this isn't a problem since the moment we start using it, it will break in an obvious way
| return sb.ToString(); | ||
| } | ||
|
|
||
| // this is eqivalent to `ULONG HashString(LPCWSTR szStr)` in CoreCLR runtime |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Please be more specific in this comment by specifying which file it's from
kg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing the painful work to make this happen
The 2nd part generates reverse thunks for methods decorated with
UnmanagedCallersOnlyAttribute. It also generates entrypoints when specified by the attribute.These methods can be called either through the entrypoint or as function pointers (from IL or native code).
Sample code that will use the generated callhelpers:
The calls using function pointers use hashing tables with keys and fallback keys. The primary key is used when the assembly containing the UMCO method hasn't changed, and the key is based on assembly FQN and method token. The fallback key is used when the assembly has changed (for example during trimming), and is based on assembly name, method name, namespace, type, and number of parameters.