Repository navigation
Reuse the wrapper of a Python consolidator registered more than once - #10160
quantconnect-kronos[bot] wants to merge 2 commits into
Conversation
Adding the same custom Python consolidator twice, through SubscriptionManager.AddConsolidator or RegisterIndicator, built a second DataConsolidatorPythonWrapper that subscribed to the Python consolidator's event but was dropped by the consolidator collections as a duplicate, so RemoveConsolidator never disposed it and it kept forwarding bars to the handlers attached to it. Reuse the wrapper already registered for that symbol instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Approving. The change fixes the cause #1824 reproduces and nothing more. Registering the same Python consolidator again now reuses the wrapper that is already registered for that symbol, through the existing FindPythonConsolidator lookup that RemoveConsolidator(symbol, pyConsolidator) already uses. Before, it built a second wrapper. That wrapper subscribed to the Python event, the consolidator collections dropped it as a duplicate, and it was never disposed.
What I checked:
- Every place a wrapper is built:
DataConsolidatorPythonWrapperis created only inAddConsolidator(Symbol, PyObject),RegisterIndicator(Symbol, PyObject, PyObject, PyObject)andRemoveConsolidator(Symbol, PyObject). The change covers both add paths, and the remove path already reused the wrapper, so no other path still leaks one. - Behavior while registered is unchanged: before, the second indicator's handler sat on the leaked wrapper, which still forwarded the Python bars. Now it sits on the one shared wrapper. Either way both indicators get every bar, as the second test shows.
- The common
register_indicator(...)+subscription_manager.add_consolidator(...)pattern on one Python consolidator had the same leak. I checked it with a local test: onmasterit leaves 2 handlers on the Python event (expected 1). With this change it leaves 1, and none afterRemoveConsolidator. - The timedelta fallback in
RegisterIndicator: the lookup now runs inside thetry. A lookup that throws falls through to the same timespan path as before.existing.Equals(pyObject)compares Python references under the GIL (BasePythonWrapper.Equals), so a non-consolidator argument doesn't match. - The
internalvisibility: Common already hasInternalsVisibleTo("QuantConnect.Algorithm"), and there is no public API change. - Tests: I ran both new tests on the PR head (pass) and against
mastersources (both fail,Expected: 1 But was: 2), as the description says. - CI: green on
adf9e92. API Tests was skipped by its workflow condition, not a failure.
Not caused by this change: registering, removing and registering again from C# with the same PyObject handle crashes the test host at teardown. It does the same on master and comes from reusing one C# handle after its wrapper disposed it, which a Python algorithm doesn't do, since each call passes a new handle. I'm not raising it.
No other repositories need an update.
Label: risk:low. It is a small bug fix that is easy to revert, with no public API change. Registrations that work today behave the same. The evidence is regression tests that fail without the fix, green CI and the end-to-end backtest the description reports.
There was a problem hiding this comment.
Requesting changes for one blocking regression: reusing a Python consolidator's adapter makes deregistering both of its indicators unsafe before the first consolidated bar. The first deregistration disposes the shared adapter; the second accesses its disposed Python object and crashes the process. This reproduction calls the APIs from Python, rather than reusing a disposed C# PyObject handle.
Evidence checked:
- CI is green on
adf9e92; API Tests is skipped. - The subscription-manager, indicator-registration and automatic-deregistration suites pass: 380 passed, 0 failed.
- Both new regression tests pass on the PR and fail against the base implementations with two Python-event handlers instead of one.
- An additional Python-only test registers an SMA and a standard deviation on one custom consolidator, then deregisters both before any consolidated output. It passes against
master(8a0c6494f). On the PR, the first deregistration completes and the second crashes the test host. - Tested end to end with a SPY-minute Python backtest. The original removal scenario stops both indicators at five samples and produces no post-removal updates. A variant deregistering both before any consolidated output completes both cleanup calls and processes 795 data points on the base, but segfaults on the PR head (exit 139) before algorithm completion. Completed runs subsequently hit the already-reported shutdown-only abort in #9708 (exit 134), including on the base; these are not clean process exits, and that separate shutdown failure is not attributed to this change.
Please preserve safe repeated cleanup of the reused adapter and add regression coverage for this lifecycle. The new tests exercise only one explicit RemoveConsolidator call, so they do not cover it.
No other repositories need an update.
Label: risk:high, because the introduced regression can terminate an algorithm. Fixing the lifecycle regression and covering it with a base/head regression test and end-to-end validation would lower that risk.
| { | ||
| dataConsolidator = new DataConsolidatorPythonWrapper(pyObject); | ||
| // reuse the wrapper of an earlier registration, a new one would never be disposed on removal | ||
| dataConsolidator = SubscriptionManager.FindPythonConsolidator(symbol, pyObject) |
There was a problem hiding this comment.
[P1] Keep the reused adapter safe for both indicator deregistrations. With two indicators registered on a custom Python consolidator that has not emitted a bar, deregister_indicator(sma); deregister_indicator(std) now removes the same wrapper twice. The first call disposes it but leaves it in the second indicator's Consolidators collection. The second call reaches SubscriptionManager.RemoveConsolidator(null, consolidator): Consolidated is null, so it reads WorkingData through the disposed Python object and crashes the process. A Python-only NUnit reproduction passes against master and crashes the test host on this head, after the first deregistration completes. Please make this existing cleanup sequence safe and add regression coverage; this is not the disposed C#-handle reuse case mentioned in the earlier review.
Two indicators sharing one Python consolidator now share its wrapper, so deregistering both removes it twice. The second removal read WorkingData through the Python object the first removal had released, crashing the process. A disposed wrapper now reports no working data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Fixed in d175011, thanks for the reproduction.
One behavior to note, now in the description: deregistering one of the two indicators removes the shared consolidator, so the other indicator stops receiving bars too. That already happens with a shared C# consolidator. Before this change, the other indicator kept receiving bars only through the leaked second wrapper. |
There was a problem hiding this comment.
Approving d17501107. It resolves the blocking regression from the previous review: deregistering two indicators that share one custom Python consolidator, before it emits a bar, no longer crashes.
What changed since my last review, and what I checked:
- The fix: a disposed
DataConsolidatorPythonWrappernow returnsnullforWorkingData. On the secondRemoveConsolidator(null, consolidator), that was the only read of the released Python object. The rest of that path is safe on a disposed wrapper.HashSet.Remove, the_consolidatorslookup andEqualsgo throughPythonReferenceComparer, which reads onlyrawPtr(0 after dispose) and never calls into Python._indicatorsByConsolidatoris aConditionalWeakTable, keyed by reference.DeregisterIndicatorthen clears the indicator's set. No other code reads the indicator'sConsolidatorsafter the first removal disposes the wrapper. The guard sits on the property every caller ofRemoveConsolidatorreaches, so it coversRemoveConsolidators(symbol)too. - The regression test does its job: with the guard reverted,
DeregisteringIndicatorsSharingAPythonConsolidatorUnsubscribesFromItsEventaborts the test host. With the guard in place it passes. - Lifecycle edge cases: I checked these with a local test, all from Python. I registered two indicators on one custom consolidator, emitted a bar, and deregistered the first. Then I registered a third indicator on the same Python consolidator, which gets a fresh wrapper with one handler, and deregistered the second indicator, which still held the disposed wrapper. The third indicator kept its consolidator and received the next bar, and the first two didn't. Deregistering the third left no handler on the Python event. The test passes.
- Related suites (
SubscriptionManagerTests,RegisterIndicator,DeregisterIndicator, consolidator Python wrapper tests): 414 passed, 0 failed. - CI: every workflow passed on
d17501107. API Tests was skipped by its workflow condition. - End to end: the description reports both runs, the original removal scenario and deregistering both indicators before any bar. Those are the runs that would expose the leak and the crash.
- Description: it states the one visible behavior change, below. Nothing private appears in the diff, the commits or the description.
A behavior change for a person to note: with two indicators on one custom Python consolidator, deregistering one now stops the other too, because both share the removed consolidator. Shared C# consolidators already behave this way. On master, the other indicator kept updating only through the leaked wrapper this PR removes. So an algorithm that deregisters one of the pair gets different values for the other from now on. I don't consider it a defect: it is what removing the consolidator means, and it matches the C# behavior.
No other repositories need an update.
Label: risk:high stays. The code is small and well covered. The behavior change above, though, alters indicator values in existing Python algorithms that deregister one of two indicators sharing a custom consolidator. A person confirming that this behavior is wanted would lower the risk.
Closes #1824
Description
Registering the same custom Python consolidator more than once, with
SubscriptionManager.AddConsolidator(symbol, pyConsolidator)orRegisterIndicator(symbol, indicator, pyConsolidator)(for example two indicators on one consolidator), built a newDataConsolidatorPythonWrappereach time. Each wrapper subscribes to the Python consolidator'sDataConsolidatedevent in its constructor, but the subscription's consolidator collections treat equal wrappers as duplicates and keep only the first one.RemoveConsolidatorthen found and disposed only that first wrapper: the others stayed subscribed to the Python consolidator and kept forwarding its bars to the indicators attached to them, after removal.Both paths now reuse the wrapper already registered for the same Python consolidator on that symbol (
FindPythonConsolidator, whichRemoveConsolidator(symbol, pyConsolidator)already used, nowinternalsoQCAlgorithmcan call it), and build a new one only when there is none. Indicators registered on the shared consolidator attach their handlers to the one wrapper, whichRemoveConsolidatordisposes, so removal leaves no subscription on the Python consolidator.Since the indicators now share one wrapper, deregistering both of them removes that wrapper twice (
DeregisterIndicatorremoves each indicator's consolidators). The second removal used to readWorkingDatathrough the Python object the first removal had disposed, which crashed the process. A disposedDataConsolidatorPythonWrappernow returnsnullforWorkingData, so removing it again is a no-op.Deregistering one of the indicators removes the shared consolidator, so the other indicator stops receiving its bars too. This is what already happens when two indicators share a C# consolidator. Before this change, the other indicator kept receiving bars only through the leaked second wrapper.
Tests
New tests in
SubscriptionManagerTests, all failing onmaster(2 handlers on the Python event where 1 is expected) and passing with the change:RemovingAPythonConsolidatorAddedTwiceUnsubscribesFromItsEvent:AddConsolidatortwice with the same Python consolidator leaves one handler on its event, andRemoveConsolidatorleaves none.RemovingAPythonConsolidatorSharedByIndicatorsUnsubscribesFromItsEvent: two indicators registered on the same Python consolidator both receive its bars; afterRemoveConsolidator, no handler remains and a bar emitted by the Python consolidator updates neither indicator.DeregisteringIndicatorsSharingAPythonConsolidatorUnsubscribesFromItsEvent: from Python, registers an SMA and a standard deviation on one custom consolidator, then deregisters both before it emits a bar. The other consolidator on the symbol stays registered and no handler remains on the Python event. Without theWorkingDataguard it crashes the test host. Onmasterit doesn't crash, but it fails on the leak this PR fixes (2 handlers on the event where 1 is expected).Full test suite as CI runs it (Release build,
TestCategory!=TravisExclude&TestCategory!=ResearchRegressionTests):Passed! - Failed: 0, Passed: 39715, Skipped: 100, Total: 39815.Tested end to end with a Python backtest (SPY minute data) registering an SMA and a standard deviation on one custom
PythonConsolidator, removing it, then emitting a bar from it by hand: onmasterthe standard deviation still updated after removal and the algorithm failed; with this change both indicators received the same 90 samples while registered and none after removal.A second end-to-end Python backtest registered both indicators on one custom consolidator and deregistered both before any bar was consolidated. It completed (1,582 data points), and neither indicator received a sample afterwards.