Skip to content

Reset stale explicit input when a feature is called without input - #538

Open
Adelagric wants to merge 4 commits into
DeepTrackAI:developfrom
Adelagric:fix/feature-input-reset-352
Open

Adelagric wants to merge 4 commits into
DeepTrackAI:developfrom
Adelagric:fix/feature-input-reset-352

Conversation

@Adelagric

@Adelagric Adelagric commented Oct 28, 2026 •

Copy link
Copy Markdown

Fixes #352.

Problem

An input passed explicitly to Feature.__call__() stays in the _input node, so a later call without input reuses it:

s = dt.Stack(value=2)
s(3)
s()          # [3, 2], expected [2]

s = dt.Stack(value=2)
s(1)
(1 & s)()    # [1, 1, 2], expected [1, 2]

The second case is the one from the issue: 1 & s builds Value(1) >> Stack(value=s), and s is evaluated as a property, i.e. without input, so it picks up the 1 left over from the direct call.

Why "no input ⇒ reset" is not enough

Writing None into _input whenever no input is given (the approach of #357) fails test_broadcast_list, and a variant that only resets inputs previously set by __call__ fails test_Feature_arithmetic. Two existing behaviours get in the way:

  1. _input can be a computed node. Reducer (hence Mean, Sum, …) accepts feature= and wires it as _input. Overwriting that node with None discards the feature's output, so inp - Mean(inp) ends up with [].
  2. "No input" means two things. In inp - inp * 2 the same inp receives 10 and is then re-evaluated as a property, without input, within the same resolution. It must keep the 10. The issue is about an input left over from a previous call.

Fix

Resolution epochs. A process-wide counter gives every top-level Feature.__call__() a new epoch; nesting depth and current epoch live in contextvars.ContextVars, so independent threads or asyncio tasks do not see each other's calls as nested. A small context manager wraps the existing with config.with_backend(...).

Per-cell records. When a feature receives an input explicitly, it records the current epoch against the cache cell of _input that _ID resolves to, in a WeakKeyDictionary. Keying by cell rather than by raw _ID means that _IDs the cache resolves to the same entry share one record, and that records disappear with their cells when update() replaces the caches, for every component of a pipeline.

Stale check. When called without input, the feature looks at the records of the cells _ID refers to. A record from the current epoch is kept (same-pipeline reuse, as in inp - inp * 2). A record from a previous epoch is stale: the record is dropped and the _input cell is invalidated, so it falls back to its action (the default [], or the wired feature for Reducer). Invalidating rather than overwriting is what keeps Reducer(feature=...) working.

_ID resolution per cache. The caches resolve a read at an _ID longer than their key length by trimming it, but reject a write there (DeepTrackDataDict.create_index). __call__ therefore resolves, per cache, the _ID under which that cache holds the entry: _input operations (explicit input, stale check, reset) use the _ID of the _input cache, the output node is evaluated under the _ID of its own cache, and properties and arguments keep the logical _ID, so distinct _IDs still sample independently. This is what makes a feature called directly and then nested in Repeat (which calls it at _ID + (n,)) behave, and it keeps sibling cells and key lengths untouched. An _ID shorter than the key length still fails at store, as on develop.

Results:

s = dt.Stack(value=2); s(3); s()                     # 2        (was [3, 2])
s = dt.Stack(value=2); s(1); (1 & s)()               # [1, 2]   (was [1, 1, 2])
p = dt.Stack(value=2) >> dt.Add(b=1); p(5); p()      # 3        (was [6, 3])
s = dt.Stack(value=2); s(3); dt.Repeat(s, N=1)()     # 2        (was [3, 2])
s = dt.Stack(value=2); s(3); (dt.Repeat(s, N=1) & s)()   # [2, 2]  (was [3, 2, 3, 2])
inp = dt.DummyFeature(); (inp - inp * 2)(10)         # -10      (unchanged)
inp = dt.Value([1, 0]); (inp - dt.statistics.Mean(inp))()   # [0, 0] (unchanged)

Behaviour change to confirm

The __call__ docstring documented the reported behaviour as a feature:

>>> feature()  # Uses stored input
array([3, 4, 5])

With this change, feature() after feature(x) evaluates on an empty input ([] for a distributed feature), which is what the issue asks for. No test depended on the old behaviour. The __call__ and Stack docstrings are updated. Flagging it since it is user-visible.

Tests

New tests in tests/test_features.py:

  • the issue cases, the nested-pipeline case and same-resolution reuse (inp - inp * 2);
  • a get() call counter for the explicit → empty → empty sequence (one call, one recomputation on [], then cached) and a Mean(inp) falling back to its wired feature after an explicit input;
  • _ID aliases: reset through a longer _ID then a call at the original one, Repeat, the Repeat(s, N=1) & s composition, the same-resolution alias case, per-cell independence, a stale sibling next to a cell set in the current resolution, round trips between _ID lengths;
  • distinct _IDs sampling independently after a targeted property.update(), with a deterministic counter and an _ID-dependent value;
  • purge of component records after update();
  • thread isolation with a deterministic blocked call.

python -m unittest discover -s tests on this revision: 520 tests, OK (129 skipped), numpy backend only (no PyTorch, OpenCV or Deeplay installed here). The two pytest collection errors (test_elementwise.py::test_method, test_features.py::test_operator) are pre-existing on develop: helper functions named test_* that pytest picks up.

Notes

  • Epoch numbers are drawn from one locked counter, so they never collide across contexts.
  • If a feature is first touched in an epoch through the node API (e.g. as the _input of a Reducer, which calls action() directly) and only later through Feature.__call__() without input, the stale reset triggers on that later call and re-evaluates the downstream nodes once more in the same epoch. Results are consistent; it is one extra computation in a rare pattern.

Files

  • deeptrack/features.py: resolution scope, per-cell records, per-cache _ID resolution, stale check in __call__, docstrings of __call__ and Stack.
  • tests/test_features.py: regression tests.

An input passed explicitly to `Feature.__call__()` was kept in the
`_input` node forever, so a later call without input silently reused
it. This leaked into pipelines that reuse the feature as a property:

    s = dt.Stack(value=2)
    s(1)
    (1 & s)()   # [1, 1, 2] instead of [1, 2]

Simply writing `None` into `_input` when no input is given breaks two
existing behaviours: `Reducer(feature=...)` wires a feature directly as
`_input` (its value would be discarded), and a feature evaluated with
input and then as a property within the same pipeline resolution, as in
`inp - inp * 2`, must keep the input it just received.

Track resolution epochs instead. A module-level counter is incremented
on every top-level `Feature.__call__()` and each feature records, per
`_ID`, the epoch in which its input was set explicitly. When called
without input, an input set in the current epoch is kept, while one set
in a previous epoch is stale and the `_input` node is invalidated so
that it falls back to its action (an empty list, or the wired feature).
`update()` clears the records.

Update the `__call__` docstring, which documented the stale reuse, and
add a regression test.

Fixes DeepTrackAI#352
Follow-up to the epoch-based reset of stale explicit inputs.

Key records by cache cell instead of by raw `_ID`. The `_input` cache
trims an `_ID` longer than its key length to an existing cell, so two
tuples can name one cell while the raw-`_ID` records disagreed about
its epoch: `s(3); s(_ID=(0,))` and `s(3); Repeat(s, N=1)()` kept the
stale `3`, and an input set during the current resolution through one
alias could be discarded through another. Records now live in a
`WeakKeyDictionary` keyed by the `DeepTrackDataObject` the cache
resolves `_ID` to, so aliases share one record and records vanish with
their cells when `update()` replaces the cache. `update()` therefore
no longer needs an explicit clear, and components of a pipeline are
purged as well.

When a stale cell is reached only by trimming, the cache cannot store a
recomputed value at that `_ID` next to its existing key length (the
same limit already rejects `s(3); s(4, _ID=(0,))` on `develop`). The
reset then resets the input and everything downstream, as `update()`
does, instead of only invalidating.

Keep the resolution depth and epoch in context variables, with epoch
numbers drawn from one locked process-wide counter. Independent graphs
evaluated from different threads no longer see each other's calls as
nested, which made a top-level call share the epoch of a call blocked
elsewhere and keep its stale input.

Update the `Stack` docstring, which still showed the pre-fix output of
the example from the issue.

Tests: count `get()` calls through the explicit / empty / empty sequence
and check a reducer falls back to its wired feature; cover `_ID`
aliases, `Repeat`, the same-resolution alias case and per-cell
independence; check records are purged for pipeline components after
`update()`; check thread isolation with a deterministic blocked call.
The previous reset for a stale input reached through a trimmed `_ID`
replaced the `_input` cache and everything downstream, as `update()`
does. That rebuilt the caches with the alias's key length, so a later
call at the original `_ID` failed in `DeepTrackDataDict.create_index`
(`s(3); s(_ID=(0,)); s()`, or `(Repeat(s, N=1) & s)()` after `s(3)`,
which returns on `develop`), and it also discarded sibling cells holding
inputs received during the current resolution.

The caches resolve a read at an `_ID` longer than their key length by
trimming it, but reject a write there. So `Feature.__call__()` now
trims such an `_ID` up front and uses the trimmed `_ID` throughout:
resets, explicit inputs and outputs all land in the entry reads come
from. A stale cell is then always reset with a plain per-cell
`invalidate`, sibling cells keep their inputs, and the key length never
changes. As a side effect, writes at a longer `_ID` that failed on
`develop` now succeed (`s(3); s(5, _ID=(0,))`, a property invalidated
and re-read at a longer `_ID`).

An `_ID` shorter than the key length still fails at `store`, as on
`develop`.

Tests: `s()` after a reset through an alias; the `Repeat(s, N=1) & s`
composition; a stale sibling reached through an alias next to a cell
set in the current resolution; round trips between `_ID` lengths. The
thread test now releases the blocked call in `finally`, checks both
threads terminated and re-raises their exceptions.
Trimming `_ID` to the key length of the `_input` cache for the whole
call also trimmed it for property sampling. After a targeted update
such as `f.value.update()`, the property and output caches are empty
but `_input` keeps its key length, so `f(_ID=(0,))` and `f(_ID=(1,))`
sampled the same value and `Value(lambda _ID: _ID)` returned `()` for
`(7,)`; both behave on `develop`.

Resolve the trimmed `_ID` per cache instead, after sources have been
activated: the `_input` operations use the `_ID` the `_input` cache
holds the entry under, the output node is evaluated under the `_ID` its
own cache holds it under, and properties and arguments keep the logical
`_ID`. `action()` reads `_input` through the same resolution.

Test: two distinct `_ID`s sample independently after a targeted update,
with a deterministic counter and with an `_ID`-dependent value.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature caching not cleared

1 participant