docs: what AI features send and how org consent works - #67
docs: what AI features send and how org consent works#67currents-codebot wants to merge 2 commits into
Conversation
The AI features send test data — errors, stack traces, the error-context snapshot, source snippets — to an LLM provider, and that now requires an organization admin's consent. Nothing documented what leaves the account or who controls it. Describes the payload, the default provider (OpenAI, overridable to any OpenAI-compatible endpoint in self-hosted deployments), that admins hold the org-wide switch, and that revoking fails in-flight requests but does not recall data already sent. Says plainly that test output is sent as captured: values a test logs into errors or console output are part of the payload. The PII guardrails in the codebase are not wired into the request path, so the page promises no filtering. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014phZyEHhP699kRwV4GRhQ1
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
| When an AI feature runs on a failed test, Currents sends the troubleshooting context it assembled for that test: | ||
|
|
||
| * The error message, stack trace, and code frame from the failed attempt | ||
| * The **error-context snapshot** captured at the moment of failure: the page accessibility tree, console output, and network logs | ||
| * Source code snippets referenced by the stack trace | ||
| * Test, spec, and run identifiers, along with the test's title and file path | ||
| * Historical pass/fail and flakiness data for that test | ||
|
|
||
| Nothing is sent on a schedule or in the background. A request is made only when someone opens an AI feature on a specific test, or when an automation you configured requests one. |
There was a problem hiding this comment.
What is sent describes only a single failed test's troubleshooting payload, while other supported AI entry points handle multi-test bundles, snapshots, metrics, and org/project-scoped run, history, and analytics data, so readers may consent without understanding most transfers. Should we document each entry point's payload (or a shared envelope with optional collections), distinguish MCP/API reads from data sent to an AI provider, and state when each is triggered?
Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`ai/data-usage-and-consent.md` around lines 16-24, revise the "What is sent" and
request-trigger descriptions so they cover every supported AI entry point instead of
presenting a single failed test as the complete data flow. Document the distinct payload
variants (or a common envelope with optional collections): single-test troubleshooting;
run-level `Fix all failures` (up to 25 failing tests and up to 5 snapshots); spec-level
`Fix spec failures` bundling all failed and flaky tests with their errors, identifiers,
histories, snapshots, and attachments; `Test Explorer` flaky/slow test metrics; `Analyze
with Currents` data for every test definition; and MCP/n8n queries and prompts for run
history, flakiness, and performance analytics. Distinguish which of these only
read/expose data directly via MCP/API versus which assemble a payload sent to an AI
provider, qualify when each is triggered, and cross-check the wording against the linked
IDE documentation and AI overview to ensure the consent scope is accurate — or
alternatively, explicitly scope this section to only the dashboard/Slack/IDE Fix-with-AI
payloads and add a separate description of the broader data MCP/n8n integrations can
retrieve and forward.
| ## Where it is sent | ||
|
|
||
| Currents sends the context to an LLM provider over an API call. The default provider is OpenAI; self-hosted and enterprise deployments can point Currents at a different OpenAI-compatible endpoint — an LLM gateway, Azure OpenAI, or another provider — via configuration. | ||
|
|
||
| Your test data is not used to train models. Requests are made through the provider's API, which is covered by the provider's API data-handling terms rather than its consumer-product terms. |
There was a problem hiding this comment.
The no-training/provider guarantee conflates Currents-managed backend calls with customer-controlled agents and providers, while currents.apiBaseUrl points to the Currents REST API rather than an LLM endpoint, so customers may assume external AI payloads are covered by Currents’ Service Agreement. Should we scope the guarantee to covered paths and either add a real provider endpoint/contract or remove the configurability claim tied to currents.apiBaseUrl?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`ai/data-usage-and-consent.md` around lines 32-36, rewrite the "Where it is sent"
section to accurately reflect the released architecture: (1) distinguish any
Currents-managed backend requests to a configured LLM endpoint from the dashboard,
Slack, IDE, MCP, and n8n flows that hand prompts/context to customer-controlled agents
or providers (e.g. Cursor, Copilot, Claude Code) — do not imply Currents transmits
every AI payload or that the OpenAI/provider API terms govern those external handoffs;
(2) remove or qualify the blanket guarantee that data is never used for training and is
covered by the linked Currents Service Agreement, since that agreement cannot govern an
independently selected provider — scope it by endpoint/provider or state clearly that
Currents cannot guarantee those settings for self-selected providers; (3) do not
reference `currents.apiBaseUrl` as a way to configure an OpenAI-compatible provider,
since it is documented elsewhere as the Currents REST API base URL — either implement
and document a real separate provider endpoint/contract, or remove the configurability
claim entirely.
There was a problem hiding this comment.
Commit 3cd35ef addressed this comment by removing the currents.apiBaseUrl provider-configuration reference and distinguishing provider API terms from consumer-product terms. However, the no-training guarantee and broader provider configurability remain insufficiently scoped.
|
|
||
| ## Who can grant consent | ||
|
|
||
| Only **organization administrators** can change the consent setting. The decision applies to the entire organization: every project, every user, every AI entry point. |
There was a problem hiding this comment.
Skill is misclassified as consent-controlled AI
The page treats the instruction-only Playwright SKILL.md as a data-sending AI entry point, while an external agent using the MCP server can retrieve Currents data and send it to a provider Currents cannot control — should we distinguish the Skill from data-sending features and state that limitation, per CLAUDE.md and AGENTS.md?
Want Baz to fix this for you? Activate Fixer You can also update your AI coding guidelines based on this comment by apply pr to [branch name]
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
ai/data-usage-and-consent.md around line 40, revise the organization-wide consent scope
statement to distinguish Currents-controlled, data-sending AI features from the
instruction-only Playwright Skill, whose SKILL.md guidance does not itself send test
data. Also explain that an external agent using the separately documented MCP server may
retrieve Currents data and choose its own provider, so Currents cannot control that
agent’s subsequent data handling or treat it as covered by Currents’ consent
boundary.
| ### Revoking | ||
|
|
||
| Revoking takes effect immediately: | ||
|
|
||
| * AI features stop working for everyone in the organization | ||
| * AI requests already in flight fail rather than completing | ||
| * Data already sent to the provider is not recalled — revoking stops future requests | ||
|
|
||
| Because in-flight work fails, the dashboard asks you to confirm before revoking. |
There was a problem hiding this comment.
Revocation lacks a client error contract
Revoked consent has no documented HTTP status, error code, or payload, so deployed IDE, Slack, dashboard, n8n, and MCP clients cannot distinguish it from provider or network failures or show the promised unavailable state. Could we define a stable consent-required/revoked response and document each consumer’s handling, including whether in-flight API/MCP retrieval is rejected?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
`ai/data-usage-and-consent.md` around lines 50-58, the Revoking section promises that AI
features and in-flight requests fail but does not define a stable, distinguishable
failure contract. Document the consent-revoked response, including its HTTP status,
error code, payload, and whether in-flight API/MCP retrievals are rejected, then explain
how the IDE, Slack, dashboard, n8n, and MCP clients should handle and present it. Align
the related consumer documentation so clients can distinguish revoked consent from
ordinary provider or network failures.
There was a problem hiding this comment.
Commit 3cd35ef addressed this comment by documenting that revocation takes effect immediately and that in-flight AI requests fail rather than completing. It does not define an HTTP status, error code, payload, or consumer-specific handling.
The page said consent was off by default and granted during signup. What merged puts a ticked "Allow AI analysis" box on the org creation form, and the signup screen carries no consent UI at all. Clearing the box records nothing, which is a third state from refusing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014phZyEHhP699kRwV4GRhQ1
2 participants
User description
Documents the data-handling side of the AI features, and gives the AI usage
consent UI in the dashboard a "Read more" destination.
New page
ai/data-usage-and-consent.md, linked from the AI section ofSUMMARY.mdand fromai/overview.md.What it covers
snapshot (accessibility tree, console, network), source snippets, identifiers,
and the test's history.
or background.
enterprise deployments can point at any OpenAI-compatible endpoint.
recall data already sent.
Two things stated deliberately
The page says test output is sent as captured — if a test logs a credential
into console output, that value is in the payload.
packages/ai/src/guardrailsexists but is not wired into the request path, so the page promises no PII
filtering. Claiming otherwise on a data-handling page would be a false
assurance.
It also names no specific model, and describes the provider as configurable,
so the page does not go stale when the gateway or model changes.
Depends on
Pairs with the dashboard AI usage consent stack (#3463 → #3476). The consent
copy in the dashboard links here, so this should merge before that copy ships.
🤖 Generated with Claude Code
https://claude.ai/code/session_014phZyEHhP699kRwV4GRhQ1
Generated description
Below is a concise technical summary of the changes proposed in this PR:
Document what Currents AI features send, when requests occur, where data is processed, and how organization administrators grant or revoke consent. Link the new consent guidance from the AI overview and documentation navigation.
Modified files (1)
Latest Contributors(1)
Modified files (2)
Latest Contributors(2)
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.