Skip to content

feat: add configurable evaluation exposure deduplication - #516

Open
abelonogov-ld wants to merge 15 commits into
v11from
andrey/flag-exposure-dedupe
Open

feat: add configurable evaluation exposure deduplication#516
abelonogov-ld wants to merge 15 commits into
v11from
andrey/flag-exposure-dedupe

Conversation

@abelonogov-ld

@abelonogov-ld abelonogov-ld commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Apps that evaluate a flag on every render or inside a loop invoke their hooks for each call, even when the evaluation resolves to the same result every time. This lets a hook ask for a dedupe window so those redundant evaluation series are collapsed.

Only hooks are affected. Analytics events are untouched — feature, debug, and summary events are still recorded for every evaluation, so the evaluation counts LaunchDarkly reports for your flags do not change.

Deduplication is opt-in per hook, and there is no client-wide setting. A hook observes every evaluation until it returns a deduper, so an audit hook can see everything while an observability hook on the same client keeps a long window:

class MetricsHook: Hook {
    // Told about every evaluation (default: evaluationExposureDeduper == nil).
}

class ObservabilityHook: Hook {
    let evaluationExposureDeduper: EvaluationExposureDeduper? =
        EvaluationExposureDeduper(window: 30, maxSize: 5_000)
}

config.hooks = [MetricsHook(), ObservabilityHook()]

Behavior when a hook carries a window:

  • The hook observes an evaluation at most once per window per unique result, keyed on flag key, variation, event version, experiment status, and the fully qualified context key.
  • Suppression covers the whole evaluation series: a suppressed evaluation invokes neither beforeEvaluation nor afterEvaluation, so a hook that pairs its stages never sees an unmatched before.
  • Each hook is deduplicated independently, so one hook observing an evaluation never suppresses it for another.
  • identify clears every hook's deduper, even when the context is unchanged, so identify stays a reliable way for an app to mark a new phase of a session.

This started as a port of flagExposureDedupeWindowMillis from the Web Observability SDK, moved down to the flag SDK level. Companion PR for the Android SDK: launchdarkly/android-client-sdk#380. Sample app: launchdarkly/hello-ios#73.

What changed since the first revision

Reviewers who read an earlier description will find three design decisions reversed:

  1. Deduplication moved off the event pipeline and onto hooks. It originally suppressed the feature event and the summary event together, which reduced volume the most but also dropped the evaluation counts LaunchDarkly reports. The telemetry the feature is really aimed at is produced by hooks, so gating hooks alone gets the volume reduction without touching what LaunchDarkly reports.
  2. The window became a per-hook policy rather than one client-wide setting. A single window forces unrelated hooks to share a deduplication policy, which does not hold up once an app registers hooks with different purposes alongside the observability plugin's.
  3. The LDConfig options are gone entirely, and the default is now no deduplication. Returning nil (or .disabled) means the hook sees every evaluation. Deduplication is a property of what a hook does with an evaluation, so the hook is now the only place that decides.

EvaluationExposureDeduper.disabled is therefore equivalent to returning nil. It stays because returning it states the intent explicitly, and because the SDK recognizes it by identity to skip building exposure keys.

Notes for reviewers

  • The decision is made before the series opens, not after the evaluation completes. The mobile observability plugins start a span in beforeEvaluation and end it in afterEvaluation. Suppressing only the after stage would leave those spans orphaned. So hooks are selected up front using the stored flag to identify the exposure.
  • Experiment status is its own key component. versionForEvents prefers flagVersion, which only moves when the flag itself changes, so a prerequisite flipping can move an evaluation into or out of an experiment while it lands on the same variation of the same flag version.
  • Two hooks given the same deduper instance share its window, so give each hook its own instance unless you intend that.
  • EvaluationExposureDeduper is public and open for subclassing. Cap default is defaultMaxSize (2000). Windows are TimeInterval in seconds, matching every other duration on this SDK.
  • Hooks are snapshotted into RegisteredHook at client start so the resolved deduper is fixed for the client's lifetime.
  • Sample app PR demonstrates two independently-deduplicated counting hooks: feat: demonstrate per-hook evaluation exposure deduplication hello-ios#73

Known limitation

The key has no notion of where a flag was read, so two evaluations from different code paths that land on the same variation collapse into one. A hook that needs per-operation fidelity can subclass the deduper and fold ambient scope (e.g. active span id) into the key.

Test plan

  • EvaluationExposureDeduperSpec — unit coverage for disabled, suppression, window expiry, keys, reset, eviction
  • Hook / client specs covering per-hook policies, identify reset, and events still firing when hooks are suppressed
  • xcodebuild test / swift test green on the branch
  • hello-ios sample builds against this branch (feat: demonstrate per-hook evaluation exposure deduplication hello-ios#73)

Note

Overview
Opt-in hook deduplication collapses repeated evaluations that resolve to the same exposure, so hooks (e.g. observability) are not invoked on every render or loop iteration. Analytics events are unchanged—only the hook evaluation series is gated.

Hooks can expose evaluationExposureDeduper (default nil = observe every evaluation). The SDK pairs each hook with a deduper at client init via RegisteredHook, tracks environmentName on multi-environment clients, and filters hooks before beforeEvaluation/afterEvaluation using an EvaluationExposureKey built from the stored flag (variation, event version, experiment status, context key). Suppressed evaluations skip the entire hook series so paired spans are not left open.

EvaluationExposureDeduper implements a configurable time window (default 10 minutes), per-flag last-result tracking, thread-safe shouldRecord/reset, and .disabled. identify resets all hook dedupers, including when the context is unchanged, so apps can mark a new session phase.

FeatureFlag gains isInExperiment for exposure identity. Extensive unit and integration tests cover deduper behavior, multi-hook policies, shared vs separate deduper instances, and secondary environments.

Reviewed by Cursor Bugbot for commit c322334. Bugbot is set up for automated code reviews on this repo. Configure here.

Apps that evaluate a flag on every render or inside a loop report an
exposure for each call, even though the evaluation resolves to the same
result every time. This produces a high volume of redundant events with
no added analytical value.

Adds two config options, both leaving existing behavior unchanged by
default:

- flagExposureDedupeWindowMillis (default 0, which disables dedupe)
- flagExposureDedupeMaxSize (default 2000)

With a window configured, an exposure is recorded at most once per window
per unique result, keyed on flag key, variation, flag version, and the
fully qualified context key. Suppression covers the full feature event
and the summary event together, so evaluation counts reported to
LaunchDarkly drop along with the event volume.

identify resets the cache even when the context is unchanged, so that
identify stays a reliable way for an app to mark a new phase of a session.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread LaunchDarkly/LaunchDarkly/ServiceObjects/EventReporter.swift Outdated
abelonogov-ld and others added 3 commits August 4, 2026 17:22
flagExposureDedupeWindowMillis was an Int in milliseconds. That matches
the Android SDK's convention, but not this one: every other duration on
LDConfig is a TimeInterval in seconds, including connectionTimeout,
eventFlushInterval, flagPollingInterval, and diagnosticRecordingInterval.

Renames the option to flagExposureDedupeWindow and types it as a
TimeInterval so it reads like its neighbors, and threads seconds through
ExposureDeduper instead of converting units at the boundary. Sub-second
windows are now expressible, which a new spec case covers.

Co-authored-by: Cursor <cursoragent@cursor.com>
The guard that returns early once expired-key cleanup brings the map back
within maxSize was untested. Bugbot found the Android port was missing
that guard, so cover the path here to keep the two suites in parity and
to catch the same regression if it is ever introduced.

Uses a maxSize of 8 because the batch term is maxSize / 4, which integer
division makes zero for the smaller caps the other eviction tests use.

Co-authored-by: Cursor <cursoragent@cursor.com>
"Flag" carries no information in a flag SDK, where every value being
deduplicated is a flag, and the SDK already calls the thing being
recorded an evaluation: recordFlagEvaluationEvents, EvaluationDetail,
evaluation events.

Renames the public options to evaluationExposureDedupeWindow and
evaluationExposureDedupeMaxSize, ExposureDeduper to
EvaluationExposureDeduper along with its file and spec, and the
EventReporting hook to resetEvaluationExposureDedupeCache. Mocks
regenerated with sourcery.

Prose that says "feature flag" is left alone, since that is the
established wording throughout these doc comments.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abelonogov-ld abelonogov-ld changed the title feat: add configurable flag exposure deduplication feat: add configurable evaluation exposure deduplication Aug 5, 2026
abelonogov-ld and others added 2 commits August 4, 2026 18:22
Singling out the oldest keys meant sorting the whole cache, because
Dictionary is unordered. Sorting to pick a batch is more machinery than
this path deserves: it only runs when more keys are live at once than
maxSize allows, which means the configured cap is already too small for
the workload.

Reclaim expired keys as before, and if that is not enough, start over
instead of ranking what is left. Refilling takes another maxSize
exposures, so the cost stays amortized, and dropped keys are suppressed
again as soon as they are re-recorded.

The key being recorded when the reset fires is re-inserted, since its
window opened a moment ago and dropping it would report the very next
evaluation of that same result again.

Android needs no equivalent change: LinkedHashMap already iterates in
record order, so it drops the oldest keys without sorting.

Co-authored-by: Cursor <cursoragent@cursor.com>
The version reported on events is the flag's own version, so it does not
move when a prerequisite flip changes an evaluation's reason. Without the
experiment bit in the key, an evaluation entering or leaving an experiment
on the same variation of the same flag version stays suppressed.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abelonogov-ld abelonogov-ld reopened this Aug 5, 2026
@abelonogov-ld
abelonogov-ld marked this pull request as draft August 5, 2026 03:46
abelonogov-ld and others added 3 commits August 4, 2026 22:21
Analytics events now record every evaluation again. Deduplication instead
gates the evaluation hook series, which is what feeds plugin telemetry, so
enabling it no longer changes the evaluation counts LaunchDarkly reports.

The decision is made before the series opens rather than after the
evaluation, because hooks pair their stages: the observability plugin
starts a span in beforeEvaluation and ends it in afterEvaluation, so
suppressing only the after stage would leave that span open. Reading the
stored flag identifies the same exposure the result would.

The deduper is now reachable from arbitrary threads, so it synchronizes
itself rather than relying on the event queue.

Co-authored-by: Cursor <cursoragent@cursor.com>
A hook now carries its own deduper, so an audit hook can observe every
evaluation while an observability hook on the same client keeps a long
window. Hooks that return nil fall back to the window configured on
LDConfig, each with its own instance, since a shared one would let the
first hook to observe an evaluation suppress it for the rest.

EvaluationExposureDeduper becomes public: implementations can be built
with different parameters, opted out of with .disabled, or replaced by a
subclass. Swift hooks are protocol witnesses rather than instances the
SDK can configure, so the deduper is a protocol requirement defaulting to
nil rather than the fluent setter the Android SDK offers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Match the Android SDK: remove the LDConfig window and max-size options so
deduplication is no longer a client-wide default that every hook inherits.
A hook observes every evaluation until it returns its own
evaluationExposureDeduper; nil and .disabled mean the same thing.

Fold the parallel hooks and dedupers arrays into RegisteredHook so the pair
cannot drift apart, and move the cache cap onto
EvaluationExposureDeduper.defaultMaxSize.

Co-authored-by: Cursor <cursoragent@cursor.com>
abelonogov-ld and others added 2 commits August 6, 2026 16:59
Building a deduper required picking both a window and a cap, with no
guidance on what a reasonable window is. Both parameters now default, so
a hook that just wants the SDK's policy can write EvaluationExposureDeduper().

Co-authored-by: Cursor <cursoragent@cursor.com>
A hook set on LDConfig is one instance shared by the clients for every
environment in secondaryMobileKeys, and so is its deduper. The exposure key
carried no environment identity, so two environments resolving a flag to the
same variation of the same version looked like a repeat of each other and only
the one evaluating first reached the hook.

Co-authored-by: Cursor <cursoragent@cursor.com>
@abelonogov-ld
abelonogov-ld marked this pull request as ready for review August 7, 2026 16:49

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e407016. Configure here.

// Built on demand, since every hook wanting every evaluation is both the default and more common than not.
var key: String?
var reporting: [Hook] = []
reporting.reserveCapacity(hooks.count)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hot-path hook array allocations

Medium Severity

hooksForEvaluation checks hooks.isEmpty and hooks.count, but hooks is now a computed map over registeredHooks. Every flag evaluation allocates at least one new array, and evaluations with hooks allocate two, on the SDK’s hottest path. The same stored property check previously avoided that cost.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e407016. Configure here.

featureFlag?.versionForEvents.map { String($0) } ?? "",
String(featureFlag?.isInExperiment ?? false),
context.fullyQualifiedKey()
].joined(separator: "\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong-type eval suppresses later hooks

Medium Severity

exposureKey is built from the stored flag before evaluation, including its variation and experiment fields. A WRONG_TYPE result still opens that window even though hooks observe a nil-variation error, so a later correctly typed evaluation of the same flag can be suppressed and never reach the hook.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e407016. Configure here.

abelonogov-ld and others added 4 commits August 7, 2026 15:40
Building the exposure key by joining its components meant every
evaluation allocated a string proportional to the flag key, context key
and environment name, and forced nil variations and versions into
sentinel empty strings. EvaluationExposureKey holds the components
instead, and Swift synthesizes its hashing from them.

Co-authored-by: Cursor <cursoragent@cursor.com>
…t seen

Tracking every distinct result meant a flag that flipped from A to B and
back suppressed the return to A, because A's own window was still open,
leaving a hook reconstructing a timeline to believe the flag never came
back. The deduper now remembers only the result each flag last reported
and tells the hook about the flag whenever that result changes, or once
the window elapses while it stays the same.

The cache is now bounded by the flag set rather than by how many results
those flags have taken, which leaves the cap as a safety net that a
typical application never reaches.

Co-authored-by: Cursor <cursoragent@cursor.com>
Tracking one result per flag means the cache is already bounded by the
flag set, so a cap was a knob with nothing to tune: the SDK now keeps its
own bound of 2000 flags, which only an application that generates flag
keys rather than naming them can reach. The window is all a hook
configures.

Co-authored-by: Cursor <cursoragent@cursor.com>
A record per flag, in each environment it is evaluated in, is the flag set
the environments serve, which LaunchDarkly already bounds. Evicting from
it only cost the hook a suppression it should have had, so the reclaim and
start-over pass is gone.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant