Epic: Segment membership inspection V1 #7405

Ship the ability to count, list, sample, and stream the identities matching a segment, in both self-hosted and Edge API projects.

Settings

• settings.SEGMENT_MEMBERSHIP_ENABLED — Django boolean. Off by default, gates index maintenance signals so the feature is a dark no-op in self-hosted and SaaS until enabled.

Feature flags

• segment_membership_index (Flagsmith on Flagsmith) — gates the SaaS rollout per organisation, plus the denylist for outlier orgs.

Core concept

Reduce segment evaluation to set algebra over a finite atom basis, materialise atoms as Roaring bitmaps in core Postgres, compose bitmaps to compute membership. Async maintenance via task processor. Read API exposes count / sample / members. Edge envs populated via DDB scan during backfill (no CDC in v1).

PK-as-ordinal: Identity.id is the bitmap ord directly — no IdentityOrdinal table, no allocation hot path.

User-facing documentation PR: #7399.

Notes

The RFC is the source of truth for design decisions: https://www.notion.so/355a2b634bf9815289b3d070e164f9eb (private to the Flagsmith team). Notable choices baked into v1:

• PK-as-ordinal (no IdentityOrdinal table). Saves ~270 GB of ordinal mapping at SaaS scale.
• Single bitmap per atom; sharding deferred to Segment membership: Bitmap sharding and parallel backfill #7419.
• No CDC for edge. Freshness surfaced via as_of and manual refresh. CDC tracked in Flagsmith/edge-api#612.
• No atom GC — orphans tolerated. GC tracked in Segment membership: Automated atom GC #7418.

Tech debt knowingly taken on:

• Per-Python backfill ceiling is ~700 ids/s/single-thread. In V1, rebuild time for 870M identities is days. Closes once Vectorised evaluation flagsmith-engine#299 and Segment membership: Bitmap sharding and parallel backfill #7419 land.
• Per-atom blob size grows linearly with env identity count. At 870M, single blobs reach ~100 MB. Read latency at the largest single env hits the 100M+ SLO bucket. Closes once Segment membership: Bitmap sharding and parallel backfill #7419 lands.