Clear pending Python error when a Try-style conversion fails#138
Open
jhonabreul wants to merge 3 commits into
Open
Clear pending Python error when a Try-style conversion fails#138jhonabreul wants to merge 3 commits into
jhonabreul wants to merge 3 commits into
Conversation
PyObject.TryAsManagedObject converted with setError: true and returned false on failure, leaving the Python error indicator set on the thread with no one left to surface it. Callers of TryAs/TryAsManagedObject only observe the boolean, so the stale error survived on the thread state and was thrown by the next unrelated Python call that checks the error indicator. This was previously masked because the outermost Py.GIL scope deleted the thread state (and its pending error) on dispose; since thread states are pinned for the lifetime of the run (QuantConnect#137), the leaked error persists and poisons subsequent calls on the same thread - e.g. Lean's PythonUtilTests failing with "int() argument must be a string, a bytes-like object or a real number, not 'function'" leaked by an earlier test that exercises a failed TryAs<int> on a function. Convert with setError: false in TryAsManagedObject and clear any error a conversion sub-path may have left pending before returning false. AsManagedObject keeps converting with setError: true so the fetched error still becomes the InvalidCastException cause, which also consumes the indicator.
Bump package <Version>, AssemblyVersion/AssemblyFileVersion and the perf-test baseline reference to 2.0.63.
11 tasks
Martin-Molinero
approved these changes
Jul 13, 2026
Several Converter failure paths could leave a Python error pending even when called with setError: false, relying on the caller to clean up: - ToList did not handle a failed PyObject_GetIter: the raised TypeError survived the call (and the iterator reference leaked). Handle it like ToArray does, disposing the reference and clearing when not setting errors. - MakeList treated a null PyIter_Next result as normal exhaustion, but null also means the iterator raised: the conversion now fails instead of returning a silently truncated list, clearing the error when not setting errors and leaving it pending for the caller when setting errors. The swallowed CLR-exception catch and the element-conversion failure path also clear when not setting errors, since some element conversion paths (failed implicit operators, deleted types) raise regardless of setError to enrich MethodBinder's no-match message. - The type_error and overflow labels now clear pending errors when not setting errors, mirroring what convert_error already did, so a probing sub-path that raised before jumping cannot leak. The unconditional raises in the implicit-conversion catches are kept: MethodBinder.Invoke deliberately surfaces a pending error as the binding failure reason instead of its generic no-match message, and Converter.TryAsManagedObject clears at the API boundary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this implement/fix? Explain your changes.
PyObject.TryAsManagedObject(and thereforeTryAs<T>/TrySafeAs<T>) converted withsetError: trueand returnedfalseon failure, leaving the Python error indicator set on the thread with no one left to surface it. Callers of aTry*API only observe the boolean, so the stale error survived on the thread state and was thrown by the next unrelated Python call on that thread that checks the error indicator.This was previously masked because the outermost
Py.GILscope deleted the thread state (and its pending error) on dispose. Since #137 pins thread states for the lifetime of the run, the leaked error persists and poisons subsequent calls on the same thread. This is the root cause of thePythonUtilTests.BothInheritedAndNonInheritedClassesWork(True)failure in QuantConnect/Lean#9614's CI:ExtensionsTests.PyObjectTryConvertToNonDelegateFailexercises a failedTryAs<int>on a Python function, leavingTypeError: int() argument must be a string, a bytes-like object or a real number, not 'function'pending on the NUnit worker thread, whichPyModule.FromStringin the next Python-touching test then throws. Reproduced deterministically by running both fixtures on a single NUnit worker; fixed with this change.The fix:
TryAsManagedObjectnow converts withsetError: falseand clears anything a conversion sub-path may have left pending before returningfalse.AsManagedObjectkeeps converting withsetError: trueso the fetched error still becomes theInvalidCastExceptioncause (fetching consumes the indicator, so that path never leaked).Also bumps the version to 2.0.63 (package
<Version>,AssemblyVersion/AssemblyFileVersion, perf-test baseline reference).A follow-up commit extends the same guarantee to the converter's collection and label paths, which could leak a pending error even with
setError: false:ToListnow handles a failedPyObject_GetIterlikeToArraydoes (also fixing a leaked iterator reference);MakeListnow fails the conversion when the iterator raises mid-iteration instead of returning a silently truncated list, and clears swallowed errors when not setting errors; and thetype_error/overflowlabels clear pending errors when not setting errors, mirroringconvert_error. The unconditional raises in the implicit-conversion catches are kept on purpose —MethodBinder.Invokesurfaces a pending error as the binding failure reason (covered byImplicitConversionErrorHandling).Does this close any currently open issues?
Fixes the
PythonUtilTests.BothInheritedAndNonInheritedClassesWorkCI failure in QuantConnect/Lean#9614.Any other comments?
Regression tests added:
RaisingIteratorFailsArrayConversion— a raising iterator fails the conversion in bothsetErrormodes, leaving an error pending only whensetErroris true.OverflowConversionOnlyLeavesErrorWhenSettingErrors— overflowing numeric conversions leave an error pending only whensetErroris true.FailedTryAsDoesNotLeavePythonErrorSet— a failedTryAs<int>/TryAsManagedObjecton a Python function leaves no pending error and subsequentPyModule.FromStringcalls work.FailedAsManagedObjectRaisesWithConversionErrorAsCause—AsManagedObjectstill raisesInvalidCastExceptionwith the Python conversion error as its cause and no pending error afterwards.Verified end-to-end against Lean: with 2.0.62, running
Common.Util.ExtensionsTests+Common.Util.PythonUtilTestson a single NUnit worker reproduces the exact CI failure; with this build (2.0.63) all 352 tests pass. The full pythonnet embedding test suite passes (960 passed, 8 skipped). A full Lean test-suite sweep with a per-test PyErr-leak detector found thisTryAspath to be the only leak source across 36,316 tests.Checklist
Check all those that are applicable and complete.
AUTHORSCHANGELOG