feat(data-collection): foundation — option, resolution, accessors by ericapisani · Pull Request #6676 · getsentry/sentry-python

ericapisani · 2026-06-26T17:19:42Z

Introduces the data_collection init option, A/B/C/D resolution precedence, and the should_collect_* accessors plus the scrubbing primitives (with unit tests).

Behavior-neutral: in legacy mode the resolved DataCollection mirrors send_default_pii, and nothing consumes the new filtering yet — so merging this changes no observed behavior.

Base of the data_collection split. Consumers stack on top:

frames (stack-frame variables + source context lines)
http (request data collection)
genai (AI input/output collection)

…cessors

github-actions · 2026-06-26T17:29:28Z

Codecov Results 📊

✅ 90796 passed | ⏭️ 6305 skipped | Total: 97101 | Pass Rate: 93.51% | Execution Time: 313m 25s

📊 Comparison with Base Branch

Metric	Change
Total Tests	📈 +645
Passed Tests	📈 +645
Failed Tests	—
Skipped Tests	—

All tests are passing successfully.

✅ Patch coverage is 88.50%. Project has 2419 uncovered lines.
✅ Project coverage is 89.94%. Comparing base (base) to head (head).

Files with missing lines (3)

File	Patch %	Lines
sentry_sdk/data_collection.py	90.37%	⚠️ 18 Missing and 14 partials
sentry_sdk/client.py	86.67%	⚠️ 4 Missing and 1 partials
sentry_sdk/scope.py	50.00%	⚠️ 4 Missing

Coverage diff

@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.92%    89.94%    +0.02%
==========================================
  Files          192       193        +1
  Lines        23809     24035      +226
  Branches      8218      8314       +96
==========================================
+ Hits         21409     21616      +207
- Misses        2400      2419       +19
- Partials      1339      1354       +15

Generated by Codecov Action

ericapisani · 2026-06-29T15:58:29Z

This would need to be removed

ericapisani · 2026-06-29T17:29:58Z

+#: Safe default used by non-recording clients: collect nothing PII-gated.
+#: This is a shared, process-wide singleton. Treat it as read-only — do not
+#: mutate the returned ``DataCollection`` or its nested config objects.
+OFF_DATA_COLLECTION = _map_from_send_default_pii(False, True, True)


Change OFF_DATA_COLLECTION name to something better - this could be clearer.

Use named args, not positional - this is hard to understand otherwise

ericapisani · 2026-06-29T17:49:48Z

+COLLECTION_OFF = "off"
+COLLECTION_DENYLIST = "denyList"
+COLLECTION_ALLOWLIST = "allowList"
+_VALID_MODES = (COLLECTION_OFF, COLLECTION_DENYLIST, COLLECTION_ALLOWLIST)


Improve the names of these variables, put the possible collection modes into an enum

ericapisani · 2026-06-29T17:53:03Z

+BODY_TYPE_INCOMING_REQUEST = "incomingRequest"
+BODY_TYPE_OUTGOING_REQUEST = "outgoingRequest"
+BODY_TYPE_INCOMING_RESPONSE = "incomingResponse"
+BODY_TYPE_OUTGOING_RESPONSE = "outgoingResponse"
+
+#: All valid body types. ``http_bodies`` defaults to this (collect everything the
+#: platform supports); an empty list is the explicit opt-out.
+ALL_BODY_TYPES = [
+    BODY_TYPE_INCOMING_REQUEST,
+    BODY_TYPE_OUTGOING_REQUEST,
+    BODY_TYPE_INCOMING_RESPONSE,
+    BODY_TYPE_OUTGOING_RESPONSE,
+]


Clean this up as well:

ALL_BODY_TYPES explore whether this should also be a tuple/set (should be consistent with _VALID_MODES below unless there's a good reason for this not to be

BODY_TYPE_* could potentially become an enum. If it's not being used outside of this file, should prefix with an underscore

ericapisani · 2026-06-29T17:54:31Z

+#: Canonical sensitive denylist from the spec. Values of keys that contain any of
+#: these terms (partial, case-insensitive) are always replaced with
+#: ``"[Filtered]"`` regardless of the configured collection mode.
+SENSITIVE_DENYLIST = [


See if this can be consolidated with the existing DEFAULT_DENYLIST in scrubber/py

ericapisani · 2026-06-29T18:08:18Z

+
+#: Additional GDPR-sensitive terms users may opt into via custom deny terms.
+#: Not applied automatically; documented here for convenience.
+EXTENDED_GDPR_DENYLIST = ["forwarded", "-ip", "remote-", "via", "-user"]


I can see this was added because of this section in the spec.

We don't currently have an extended GDPR denylist - maybe something worth considering exposing to users for convenience

ericapisani · 2026-06-29T18:15:00Z

+    if isinstance(val, str):
+        return KeyValueCollectionBehavior(mode=val)


Reviewing the spec - I don't think this is a valid option. It looks like if it's provided, it needs at least a dict with a mode (see here)

ericapisani · 2026-06-29T18:27:36Z

+) -> "DataCollection":
+    """
+    Fill in any omitted fields of a user-supplied ``DataCollection`` with their
+    spec defaults (resolution case A). Frame fields fall back to the legacy


no one has any context on what "resolution case X" is. remove this, and the other references of 'resolution case' within this PR

ericapisani · 2026-06-29T18:44:28Z

+        gen_ai=user_dc.gen_ai,
+        # http_bodies: None means "all valid types"; materialize for clarity.
+        http_bodies=(
+            list(user_dc.http_bodies)


I think this list may be redundant as this should be coming in as list of strings when provided.

Same goes for list(ALL_BODY_TYPES) a couple of lines below

ericapisani · 2026-06-29T18:57:04Z

+        Returns whether the SDK should automatically populate ``user.*`` fields
+        (id, email, username, ip_address) from instrumentation.
+        """
+        return bool(self.data_collection.user_info)


user_info it initialized to a boolean value, so we shouldn't need to cast here.

Edit: should this, similar to _should_collect_gen_ai_content, also fall back to should_send_default_pii if self.data_collection.explicit is not True (i.e. fallback to legacy behaviour)

ericapisani · 2026-06-29T18:59:51Z

+        return self._should_collect_gen_ai_content("outputs", include_prompts)
+
+    def _should_collect_gen_ai_content(
+        self, direction: str, include_prompts: "Optional[bool]"


we should try to lock down the possible values of direction to be one of inputs or output - it's not necessary for us to have this type be so broad

ericapisani · 2026-06-29T19:00:43Z

+            # Integration-level override wins over the global gen_ai setting.
+            if include_prompts is not None:
+                return include_prompts
+            return bool(getattr(dc.gen_ai, direction))


We also shouldn't need to cast to bool here, the values within the GenAICollection are boolean

ericapisani · 2026-06-29T19:17:58Z

+    def data_collection(self) -> "DataCollection":
+        return OFF_DATA_COLLECTION
+
+    def should_collect_user_info(self) -> bool:
+        return False
+
+    def should_collect_gen_ai_inputs(
+        self, include_prompts: "Optional[bool]" = None
+    ) -> bool:
+        return False
+
+    def should_collect_gen_ai_outputs(
+        self, include_prompts: "Optional[bool]" = None
+    ) -> bool:
+        return False


To reduce the API surface on the client, I think we should only expose the data_collection and put the should_collect_X methods on the data_collection class itself.

Edit: alternatively, we could introduce helper methods similar to what we do for streamed spans (has_span_streaming_enabled)

ericapisani · 2026-06-29T19:20:13Z

+                        True,
+                        self.options["include_local_variables"] is not False,
+                        self.options["include_source_context"] is not False,


Similar to another spot I mentioned this on, this is not easy to follow, should pass this in as named args, not as positional args

ericapisani · 2026-06-29T19:27:11Z

+def should_collect_user_info() -> bool:
+    """Shortcut for `Scope.get_client().should_collect_user_info()`."""
+    return Scope.get_client().should_collect_user_info()
+
+
+def should_collect_gen_ai_inputs(include_prompts: "Optional[bool]" = None) -> bool:
+    """Shortcut for `Scope.get_client().should_collect_gen_ai_inputs(...)`."""
+    return Scope.get_client().should_collect_gen_ai_inputs(include_prompts)
+
+
+def should_collect_gen_ai_outputs(include_prompts: "Optional[bool]" = None) -> bool:
+    """Shortcut for `Scope.get_client().should_collect_gen_ai_outputs(...)`."""
+    return Scope.get_client().should_collect_gen_ai_outputs(include_prompts)


This mirrors what happens for should_send_default_pii, but as mentioned in another comment, I think this API surface should be reduced as if we do this for every type of configuration, this will become a large list (database queries, GQL documents/variables, etc.)

ericapisani · 2026-06-29T19:52:19Z

+    return False
+
+
+def apply_key_value_collection(


This name could be better

ericapisani · 2026-06-29T19:55:17Z

+        http_headers=HttpHeadersCollection(),
+        # Bodies are collected regardless of PII today, bounded by
+        # ``max_request_body_size``.
+        http_bodies=list(ALL_BODY_TYPES),


ALL_BODY_TYPES is already a list, it doesn't need to be cast as a list again

feat(data-collection): add data_collection option, resolution, and ac…

98022a4

…cessors

This was referenced Jun 26, 2026

feat(data-collection): configurable stack-frame variables and context lines #6677

Draft

feat(data-collection): gate HTTP request data collection #6678

Draft

feat(data-collection): split generative-AI input/output collection #6679

Draft

ericapisani added the Do not merge label Jun 26, 2026

ericapisani commented Jun 29, 2026

View reviewed changes

Comment thread CHANGELOG.md

ericapisani Jun 29, 2026

Copy link
Copy Markdown

Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need to be removed

ericapisani commented Jun 29, 2026

View reviewed changes

		if isinstance(val, str):
		return KeyValueCollectionBehavior(mode=val)

Uh oh!

Conversation

ericapisani commented Jun 26, 2026

Uh oh!

github-actions Bot commented Jun 26, 2026

Codecov Results 📊

📊 Comparison with Base Branch

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ericapisani Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ericapisani Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ericapisani Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ericapisani Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ericapisani Jun 29, 2026 •

edited

Loading

ericapisani Jun 29, 2026 •

edited

Loading

ericapisani Jun 29, 2026 •

edited

Loading

ericapisani Jun 29, 2026 •

edited

Loading