[cDAC] Add interpreter support for stack walking and diagnostics#126520
[cDAC] Add interpreter support for stack walking and diagnostics#126520max-charlamb wants to merge 9 commits intodotnet:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the cDAC (contract-based DAC) to understand interpreter execution artifacts so diagnostics tooling can correctly resolve interpreter precodes, validate interpreted MethodDescs, and walk stacks that include interpreter frames (including mixed JIT/interpreter stacks).
Changes:
- Adds new interpreter-related data descriptors/types and wires them into stack walking, precode resolution, and execution manager lookups.
- Updates legacy SOS DAC surface area to report interpreter JIT type and to resolve interpreter precodes before passing addresses to the execution manager.
- Adds unit + dump-based integration tests, including a new “InterpreterStack” debuggee and dump-generation plumbing for debuggee environment variables.
Reviewed changes
Copilot reviewed 41 out of 41 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/native/managed/cdac/tests/SOSDacInterface5Tests.cs | Updates test contract registry setup to include PrecodeStubs + PlatformMetadata mocks. |
| src/native/managed/cdac/tests/PrecodeStubsTests.cs | Adds contract v3 coverage and new interpreter-precode test scaffolding. |
| src/native/managed/cdac/tests/MockDescriptors/MockDescriptors.ExecutionManager.cs | Extends mock ExecutionManager descriptors/builders for interpreter code heaps + headers. |
| src/native/managed/cdac/tests/Microsoft.Diagnostics.DataContractReader.Tests.csproj | Adds Microsoft.DotNet.XUnitExtensions dependency for conditional skip support. |
| src/native/managed/cdac/tests/MethodDescTests.cs | Adds validation tests for native code slot precode fallback and plumbs PrecodeStubs into target creation. |
| src/native/managed/cdac/tests/FrameIteratorTests.cs | New unit tests validating InterpreterFrame MethodDesc resolution + naming behavior. |
| src/native/managed/cdac/tests/ExecutionManager/ExecutionManagerTests.cs | Adds execution manager tests for interpreter code heaps and precode-range behavior. |
| src/native/managed/cdac/tests/DumpTests/InterpreterStackDumpTests.cs | New dump-based integration tests validating mixed interpreter/JIT stack walking and precode resolution. |
| src/native/managed/cdac/tests/DumpTests/DumpTestStackWalker.cs | Enriches resolved frame model to include runtime frame name and raw handle for assertions. |
| src/native/managed/cdac/tests/DumpTests/DumpTests.targets | Adds support for passing per-debuggee environment variables into dump generation. |
| src/native/managed/cdac/tests/DumpTests/Debuggees/InterpreterStack/Trampoline/Trampoline.csproj | New trampoline library project used to force a JIT-only gap between interpreter regions. |
| src/native/managed/cdac/tests/DumpTests/Debuggees/InterpreterStack/Trampoline/Trampoline.cs | Provides a no-inlining “Bounce” method in a separate assembly to create a mixed stack. |
| src/native/managed/cdac/tests/DumpTests/Debuggees/InterpreterStack/Program.cs | New debuggee program building the deterministic mixed JIT/interpreter call chain. |
| src/native/managed/cdac/tests/DumpTests/Debuggees/InterpreterStack/InterpreterStack.csproj | New debuggee configuration (Full dumps, Jit R2R mode, DOTNET_Interpreter filter). |
| src/native/managed/cdac/tests/DumpTests/Debuggees/Directory.Build.targets | Propagates EnvironmentVariables metadata into dump generation itemization. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Legacy/SOSDacImpl.cs | Maps interpreter JIT type to legacy SOS constant; resolves interpreter precodes at multiple call sites. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Legacy/ClrDataMethodInstance.cs | Resolves interpreter precode before IL-address-map resolution. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/RuntimeTypeSystemHelpers/MethodValidation.cs | Adds PrecodeStubs fallback when execution manager can’t map NativeCodeSlot pointers. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpreterRealCodeHeader.cs | New contract data type for interpreter “real code header” method metadata. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpreterPrecodeData.cs | New contract data type for interpreter precode data (ByteCodeAddr + Type). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpreterFrame.cs | New contract data type for InterpreterFrame (top context frame pointer). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpMethodContextFrame.cs | New contract data type for interpreter context frames (StartIp + ParentPtr). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpMethod.cs | New contract data type for interpreter method records (MethodDesc pointer). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Data/InterpByteCodeStart.cs | New contract data type linking bytecode start to interpreter method record. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/StackWalk/StackWalk_1.cs | Expands InterpreterFrame into per-method synthetic frames and resolves MethodDesc from context frame chain. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/StackWalk/FrameHandling/FrameIterator.cs | Adds InterpreterFrame + CLRToCOMMethodFrame support; adds interpreter context-chain resolution helpers. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/PrecodeStubs_Common.cs | Adds interpreter precode support and the “resolve interpreter precode to code” API implementation. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/PrecodeStubs_3.cs | Implements InterpreterPrecode_GetMethodDesc for v3 by walking interpreter bytecode metadata chain. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/PrecodeStubs_2.cs | Adds required v2 stub for interpreter precode method-desc resolution (currently not implemented). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/PrecodeStubs_1.cs | Adds required v1 stub for interpreter precode method-desc resolution (currently not implemented). |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/ExecutionManager/ExecutionManagerCore.InterpreterJitManager.cs | New interpreter JIT manager for mapping interpreter code heap addresses to CodeBlock metadata. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/ExecutionManager/ExecutionManagerCore.cs | Adds interpreter range-section flag + routes lookups through the interpreter JIT manager. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Abstractions/DataType.cs | Adds interpreter-related DataType entries. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Abstractions/Contracts/IPrecodeStubs.cs | Adds the new “GetInterpreterCodeFromInterpreterPrecodeIfPresent” API. |
| src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Abstractions/Contracts/IExecutionManager.cs | Extends JitType enum with Interpreter. |
| src/coreclr/vm/frames.h | Exposes InterpreterFrame fields via cdac_data specialization for descriptors. |
| src/coreclr/vm/datadescriptor/datadescriptor.inc | Adds interpreter data descriptors (headers/precodes/frames/context structures). |
| src/coreclr/vm/datadescriptor/datadescriptor.h | Includes interpreter headers under FEATURE_INTERPRETER for descriptor compilation. |
| docs/design/datacontracts/StackWalk.md | Documents interpreter-related stackwalk descriptors. |
| docs/design/datacontracts/PrecodeStubs.md | Documents interpreter precode method-desc resolution and precode→code translation API. |
| docs/design/datacontracts/ExecutionManager.md | Documents interpreter real-code-header usage and Interpreter JitType semantics. |
Comments suppressed due to low confidence (1)
src/native/managed/cdac/Microsoft.Diagnostics.DataContractReader.Contracts/Contracts/StackWalk/FrameHandling/FrameIterator.cs:104
- FrameType.CLRToCOMMethodFrame is treated as a transition FramedMethodFrame in GetMethodDescPtr, but UpdateContextFromFrame doesn’t include it in the transition-frame case list. That means the context won’t be updated/unwound correctly when encountering CLRToCOMMethodFrame, which can break subsequent stack-walk frame resolution. Add CLRToCOMMethodFrame to the transition-frame cases so it’s handled via FramedMethodFrame/HandleTransitionFrame like the other transition frames.
// TransitionFrame type frames
case FrameType.FramedMethodFrame:
case FrameType.PInvokeCalliFrame:
case FrameType.PrestubMethodFrame:
case FrameType.StubDispatchFrame:
case FrameType.CallCountingHelperFrame:
case FrameType.ExternalMethodFrame:
case FrameType.DynamicHelperFrame:
case FrameType.InterpreterFrame:
// FrameMethodFrame is the base type for all transition Frames
Data.FramedMethodFrame framedMethodFrame = target.ProcessedData.GetOrAdd<Data.FramedMethodFrame>(CurrentFrame.Address);
GetFrameHandler(context).HandleTransitionFrame(framedMethodFrame);
return;
7e1ec2a to
cdaefe0
Compare
d425631 to
d5c6cd8
Compare
b184da2 to
07ddd0d
Compare
Implement interpreter support in the cDAC diagnostic subsystem: - Add InterpreterFrame data type and frame iteration support - Add interpreter precode resolution (GetInterpreterCodeFromInterpreterPrecodeIfPresent) - Add GetCodeForInterpreterOrJitted API to IRuntimeTypeSystem - Add interpreter RealCodeHeader and EECodeInfo support in ExecutionManager - Support InterpreterMethodInfo, InterpreterRealCodeHeader data descriptors - Update design docs for ExecutionManager, PrecodeStubs, RuntimeTypeSystem - Add unit tests for precode stubs, interpreter frames, SOSDacInterface5 - Add InterpreterStack dump test debuggee and tests
- Build cDAC dump tests with Checked runtime for FEATURE_INTERPRETER - Pass debuggee EnvironmentVariables through Helix dump generation - Fix env var leaking between Helix debuggees - Skip interpreter dump tests when FEATURE_INTERPRETER not enabled
…nsolidate helpers - Add GetCodeForInterpreterOrJitted to IRuntimeTypeSystem to unify the pattern of getting native code then resolving interpreter precodes - Add ResolveInterpreterPrecode private helper in SOSDacImpl to replace 3 verbose call sites - Use GetCodeForInterpreterOrJitted in ClrDataMethodInstance and InterpreterStackDumpTests - Collapse duplicate GetMethodInfo pseudocode in ExecutionManager.md - Fix double-registration of BumpAllocator fragments in tests - Fix int->string contract version parameters after rebase Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
During the rebase conflict resolution, buildConfig was accidentally changed from 'release' to 'checked' in the DumpCreation, DumpTest, and XPlatDumpTest pipeline stages. The -rc checked flag in buildArgs already ensures the CLR is built as Checked; buildConfig controls the overall build including libs which must remain Release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- FrameIterator.cs: fix XML comment to use MethodDesc instead of methodHnd - PrecodeStubs.md: fix pseudocode to use TargetPointer.Null and TargetCodePointer Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rPrecode Remove the GetCodeForInterpreterOrJitted contract API since it only had a single consumer. Instead, call GetNativeCode followed by GetInterpreterCodeFromInterpreterPrecodeIfPresent directly at each call site. Also inline the ResolveInterpreterPrecode private helper in SOSDacImpl since it was a single-line wrapper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
d7ad6bf to
63f4e8e
Compare
|
/azp run |
|
You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list. |
| ``` | ||
|
|
||
| There are two JIT managers: the "EE JitManager" for jitted code and "R2R JitManager" for ReadyToRun code. | ||
| There are three JIT managers: the "EE JitManager" for jitted code, the "Interpreter JitManager" for interpreter code, and the "R2R JitManager" for ReadyToRun code. |
There was a problem hiding this comment.
A nit:
| There are three JIT managers: the "EE JitManager" for jitted code, the "Interpreter JitManager" for interpreter code, and the "R2R JitManager" for ReadyToRun code. | |
| There are three JIT managers: the "EE JitManager" for jitted code, the "Interpreter JitManager" for interpreted code, and the "R2R JitManager" for ReadyToRun code. |
|
|
||
| #### Interpreter Frame Expansion | ||
|
|
||
| When the stack walker encounters an `InterpreterFrame`, it expands it into multiple logical frames by walking the `InterpMethodContextFrame.ParentPtr` chain. The runtime maintains a linked list of `InterpMethodContextFrame` nodes representing each interpreted method currently on the call stack within a single `InterpreterFrame`. The `TopInterpMethodContextFrame` field points to the most recently entered interpreted method, and each node's `ParentPtr` points to its caller. |
There was a problem hiding this comment.
This is not completely true. The TopInterpMethodContextFrame is just approximate hint and the real top needs to be found by scanning the pNext or pParent chains. See the InterpreterFrame::GetTopInterpMethodContextFrame():
runtime/src/coreclr/vm/frames.cpp
Lines 1777 to 1810 in 2b16e8b
| | `HijackArgs` (arm/arm64/x86) | For each register `r` saved in HijackArgs, `r` | Register names associated with stored register values | | ||
| | `InterpreterFrame` | `TopInterpMethodContextFrame` | Pointer to the InterpreterFrame's top `InterpMethodContextFrame` | | ||
| | `InterpMethodContextFrame` | `StartIp` | Pointer to the `InterpByteCodeStart` for resolving the MethodDesc | | ||
| | `InterpMethodContextFrame` | `ParentPtr` | Pointer to the parent `InterpMethodContextFrame` in the call chain (null for outermost frame) | |
There was a problem hiding this comment.
- You'll also need the
NextPtrfor locating the real topmost frame. - It seems you'd also need the
Ip,StartIpis the IP of the start of the method while the 'Ip` is the actual instruction pointer.
| internal static IEnumerable<TargetPointer> WalkInterpreterFrameChain(Target target, TargetPointer frameAddress) | ||
| { | ||
| Data.InterpreterFrame interpFrame = target.ProcessedData.GetOrAdd<Data.InterpreterFrame>(frameAddress); | ||
| TargetPointer interpMethodFramePtr = interpFrame.TopInterpMethodContextFrame; |
There was a problem hiding this comment.
See my previous comment, this is not correct since the TopInterpMethodContextFrame is just a hint and you need to find the actual one the same way as the InterpreterFrame::GetTopInterpMethodContextFrame() does.
There was a problem hiding this comment.
Ahh, I missed that. I'll update to reflect the native method. Thanks
| if (frameAddress != TargetPointer.Null | ||
| && stackWalkData.FrameIter.GetCurrentFrameType() == FrameIterator.FrameType.InterpreterFrame) | ||
| { | ||
| foreach (TargetPointer contextFramePtr in FrameIterator.WalkInterpreterFrameChain(_target, frameAddress)) |
There was a problem hiding this comment.
I am not sure if the CDAC code would work correctly in case of debugger breakpoint in an interpreted method. There was a bug in the native stack walk that I have fixed few days ago (still pending PR, it is actually Milos Kotlar's PR #126953 that I have contributed to) and that resulted in interpreted frames belonging to the first InterpreterFrame being walked twice. The thing is that when debugger breakpoint is hit, the debugger is passed the context set to the interpreter IP/SP. The native stack walker originally walked those interpreted frames, then hit the actual InterpreterFrame and that kicked in a second walk over those.
…nd prevent double-walk - Add Ip and NextPtr fields to InterpMethodContextFrame data descriptor - Implement ResolveTopInterpMethodContextFrame() to scan chain for first active frame (non-null Ip), since the hint may point to an inactive frame - Add SkipNextInterpreterFrame flag to prevent double-walking interpreter frames when a debugger breakpoint fires in interpreted code - Add unit tests for hint resolution and double-walk prevention - Add dump test StackWalk_NoDoubledInterpreterFrames - Fix typo in ExecutionManager.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a multi-threaded debuggee (InterpreterStackDoubleWalk) that validates interpreter stack frame walking on a non-crashing thread. A worker thread builds a full interpreter call chain (MethodA->B->Bounce->C->D->spin loop) while the main thread triggers FailFast. Tests walk the worker thread to verify: - Interleaved JIT/interpreter frame layout is correct - No interpreter method appears more than once (no doubled frames) The worker spins in interpreted code (volatile field-read loop) so the full interpreter frame chain is on the stack at dump time. The CPU IP is inside the native interpreter engine, so the walk starts from SW_FRAME state. Fix a pre-existing infinite loop in StackWalk_1.Next() where Context.Unwind() fails to advance IP/SP on certain frames (e.g. FailFast PInvoke, SleepEx). When IP and SP are unchanged after Unwind, fall back to Frame chain if available, otherwise mark walk as complete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
0626b75 to
a8246a4
Compare
There was a problem hiding this comment.
revert changes to this file
| { | ||
| Data.InterpreterFrame interpreterFrame = target.ProcessedData.GetOrAdd<Data.InterpreterFrame>(frame.Address); | ||
| return ResolveMethodDescFromInterpFrame(target, interpreterFrame.TopInterpMethodContextFrame); | ||
| TargetPointer topContextFrame = ResolveTopInterpMethodContextFrame(target, interpreterFrame.TopInterpMethodContextFrame); |
There was a problem hiding this comment.
Lets change ResolveTopInterpMethodContextFrame to take the interpreterframe
…veTopInterpMethodContextFrame Revert the whitespace-only change to IRuntimeTypeSystem.cs. Change ResolveTopInterpMethodContextFrame to take Data.InterpreterFrame instead of the raw hint pointer, simplifying call sites. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| { | ||
| layout = targetTestHelpers.LayoutFields([ | ||
| new(nameof(Data.InterpreterPrecodeData.Type), DataType.uint8), | ||
| new(nameof(Data.InterpreterPrecodeData.ByteCodeAddr), DataType.pointer), | ||
| new("Target", DataType.pointer), | ||
| ]); | ||
| types[DataType.InterpreterPrecodeData] = new Target.TypeInfo() |
There was a problem hiding this comment.
The mock TypeInfo layout for InterpreterPrecodeData here doesn’t match the actual CoreCLR layout. In CoreCLR (src/coreclr/vm/precode.h), struct InterpreterPrecodeData is ByteCodeAddr, then Target, then Type (so Type is after two pointers). This test currently lays out Type first, which means the offsets used by PrecodeStubs_3_Impl.InterpreterPrecode_GetMethodDesc / GetInterpreterCodeFromInterpreterPrecodeIfPresent in the test target won’t reflect real dump layouts. Reorder the mock layout to ByteCodeAddr + (placeholder) Target + Type (and update the corresponding writes in AddInterpreterPrecodeEntry).
| <MSBuild Projects="$(MSBuildProjectFile)" | ||
| Targets="_GenerateDumpsForDebuggee" | ||
| Properties="DebuggeeName=%(_DebuggeeWithDumpTypes.Identity);_DebuggeeDumpTypes=%(_DebuggeeWithDumpTypes.DumpTypes);_DebuggeeR2RModes=%(_DebuggeeWithDumpTypes.R2RModes)" /> | ||
| Properties="DebuggeeName=%(_DebuggeeWithDumpTypes.Identity);_DebuggeeDumpTypes=%(_DebuggeeWithDumpTypes.DumpTypes);_DebuggeeR2RModes=%(_DebuggeeWithDumpTypes.R2RModes);_DebuggeeEnvVars=%(_DebuggeeWithDumpTypes.EnvironmentVariables)" /> | ||
|
|
There was a problem hiding this comment.
_DebuggeeEnvVars is documented as semicolon-separated, but it’s being forwarded via the MSBuild task’s Properties="...;_DebuggeeEnvVars=..." string. If %(EnvironmentVariables) contains ; (multiple variables), MSBuild will treat those as property separators and truncate/corrupt the value (and potentially create unintended properties). Use $([MSBuild]::Escape(...)) when embedding this metadata into the Properties string (and apply the same escaping at the other _DebuggeeEnvVars=... forwarding sites in this targets file).
Summary
Adds interpreter support to the cDAC (contract-based Data Access Component), enabling diagnostic tools to correctly walk stacks containing interpreter frames, resolve interpreter precodes, and retrieve method information for interpreted methods.
Changes
Native Data Descriptors
datadescriptor.inc:InterpreterRealCodeHeader,InterpreterPrecodeData,InterpByteCodeStart,InterpMethod,InterpMethodContextFrame,InterpreterFrameInterpreterFrametoframes.hexplicit frame list for cDAC visibilityExecution Manager - Interpreter JIT Manager
ExecutionManagerCore.InterpreterJitManagerhandles code address lookups for interpreter code heapsGetCodeBlockHandlenow searches interpreter code heaps when JIT heaps don't contain the addressGetMethodDescresolves MethodDesc from interpreter code headersPrecode Resolution (
GetInterpreterCodeFromInterpreterPrecodeIfPresent)IPrecodeStubsmatching the native DAC pattern: each call site resolves interpreter precodes before passing addresses to ExecutionManagerVirtualReadExceptioncatchGetMethodDescData,CopyNativeCodeVersionToReJitData,GetTieredVersions,GetILAddressMapStack Walking
FrameIteratorhandlesInterpreterFrame- extracts MethodDesc and native code pointer fromInterpMethodContextFrameStackWalk_1resolves interpreter frames during enumerationRuntimeTypeSystem
MethodValidationupdated to handle interpreter method descriptors (IsInterpreterStubflag, chunk validation)Documentation
ExecutionManager.md,PrecodeStubs.md,StackWalk.mdwith interpreter support detailsTests
ExecutionManagerTests(interpreter JIT manager),FrameIteratorTests(interpreter frame handling),PrecodeStubsTests(interpreter precode resolution),MethodDescTests(interpreter method validation)InterpreterStackDumpTests- 3 integration tests using a mixed JIT/interpreter stack debuggee (InterpreterStack) that validates interleaved frame layout, precode resolution, and thread enumeration