Skip to content

feat: ad hoc filter finishing touches - #3658

Open
knudtty wants to merge 3 commits into
mainfrom
aaron/adhoc-follow-up
Open

knudtty wants to merge 3 commits into
mainfrom
aaron/adhoc-follow-up

Conversation

@knudtty

@knudtty knudtty commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Finishing touches for ad hoc dashboard filters, which are still behind NEXT_PUBLIC_ENABLE_ADHOC_FILTERS.

  • The name is an identifier, not a variable. You can't reference an ad hoc filter as $name, but its name still has to be unique because it keys the filter's conditions in the filters= URL param. The edit form now says so. The filter-bar tooltip and the filters list no longer present it as a variable. getDashboardVariableFilters skips ad hoc filters, so checkAlerts, MCP validation and the alert chart stop declaring them. isVariableEnabled: true stays, because name-uniqueness validation reads it.
  • Broadcast is always on. With no variable, broadcasting is the only way the conditions apply. The schema now requires isBroadcastEnabled: z.literal(true), the same way STATIC_LIST and PROMETHEUS_LABEL use z.literal(false). The edit form drops the broadcast checkbox and always shows "Applies to sources".
  • Overlap warning. The edit form shows a warning when another ad hoc filter of the same source type already applies to any of the same sources. A click-to-filter on such a tile updates every one of those filters, which is confusing. It is a warning only; saving still works.
  • Collapsed chips. The filter input shows the first 3 conditions plus a "+N more" toggle that expands in place. Series with many labels, which is common in PromQL, made the input grow very tall.
  • IN (SQL) and =| (PromQL) operators. Each condition still holds one key and one value. When rendered, conditions sharing a key are OR'd: SQL renders toString(key) IN ('a', 'b') and PromQL renders key=~"a|b", with regex characters escaped. These are two separate stored values, each valid only for its source type. A condition the filter's source type can't express is skipped and its chip is marked invalid.

Compatibility: an ad hoc filter already stored with isBroadcastEnabled: false, or without the field, will no longer parse. The feature is behind a flag, so there is no migration.

Screenshots or video

I couldn't capture screenshots from my environment. @aaronknudtson, please add before/after shots of the ad hoc filter edit modal and of a filter input holding more than 3 conditions.

Before After

How to reproduce

Starting point: yarn dev. packages/app/.env.development sets NEXT_PUBLIC_ENABLE_ADHOC_FILTERS=true.

On main:

  1. Open a dashboard, click "Filters", then "Add filter", and choose the type "Ad hoc keys and values".
  2. The form has a "Broadcast filter conditions" checkbox, and the "Variable name" tooltip says it's how the variable is referenced.
  3. Save a filter over the Logs source, then add a second one over Logs. No warning appears.
  4. Add 6 conditions to one filter. The input wraps onto several rows.
  5. Open the operator select in the condition editor. There is no in option.

On this branch:

  1. In the same form there is no broadcast checkbox, and "Applies to sources" is always visible. Under "Variable name" it reads "Unique name for this filter. Ad hoc filters can't be referenced as variables in tile queries."
  2. Adding a second ad hoc filter over Logs shows "Another ad hoc filter (…) already applies to these sources."
  3. With 6 conditions, the input shows 3 chips and "+3 more". Clicking it shows every chip and "Show less".
  4. Add ServiceName in api and ServiceName in web. Tiles on Logs filter with toString(ServiceName) IN ('api', 'web'). On a Prometheus ad hoc filter, =| conditions on the same label render as label=~"(a|b)".
  5. Hovering the filter label shows "Filters N sources", without "available as variable".

How to test on Vercel preview

N/A. The preview builds without NEXT_PUBLIC_ENABLE_ADHOC_FILTERS, so ad hoc filters aren't offered there.

Preview routes: N/A

Steps:

  1. N/A

References

  • Linear Issue: Closes HDX-5761

- Add IN (SQL) and =| (PromQL) operators; conditions on the same key are OR'd
  into one IN (...) or =~"a|b" condition.
- Ad hoc filters always broadcast (isBroadcastEnabled: z.literal(true)), and
  their variable name is a unique identifier only, no longer declared as a
  $variable.
- Warn when another ad hoc filter already applies to the same sources.
- Collapse the filter input to three conditions behind a "+N more" toggle.
@knudtty
knudtty requested a review from a team as a code owner November 13, 2026 21:17
@changeset-bot

changeset-bot Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 61e898a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/api Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 13, 2026 10:00pm UTC
hyperdx-storybook Ready Ready Preview Nov 13, 2026 10:00pm UTC

Request Review

@github-actions github-actions Bot added the review/tier-3 Standard — full human review required label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Tier 3 — Standard

Introduces new logic, modifies core functionality, or touches areas with non-trivial risk.

Why this tier:

  • Diff size: 384 production lines changed (Tier 2 max: < 250)
  • Cross-layer change: touches frontend (packages/app) + shared utils (packages/common-utils)

Review process: Full human review — logic, architecture, edge cases.
SLA: First-pass feedback within 1 business day.

Stats
  • Production files changed: 9
  • Production lines changed: 384 (+ 338 in test files, excluded from tier calculation)
  • Branch: aaron/adhoc-follow-up
  • Author: knudtty

To override this classification, remove the review/tier-3 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.18%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

);
})}
{conditions
.slice(0, conditions.length - hiddenCount)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — New and invalid conditions are hidden while the filter is collapsed

New conditions are appended to the end of the list, and the list is cut at the first three. Once a filter holds three or more conditions, a condition the user adds with "+" or by clicking a tile value is hidden right away; the only sign of it is the "+N more" count going up. An invalid (data-invalid) chip past the third position is hidden the same way, so its "Filter not applied" warning never shows. Fix: expand the list when a condition is added (set isExpanded in the add and onChange paths), or always show invalid chips. At minimum, mark the "+N more" button when a hidden chip is invalid.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

)}
<Separator size="xs" />
<VariableNameInput control={control} otherFilters={otherFilters} />
<VariableNameInput

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Ad hoc form still labels the field "Variable name" while its own description says it isn't a variable

VariableNameInput hardcodes label="Variable name" in VariableNameInput.tsx:30, so the ad hoc form shows "Variable name" directly above "Ad hoc filters can't be referenced as variables". Fix: make the label configurable as well as the description (or replace description with a variant/isReferenceable prop that sets both), and pass something like "Filter ID" from AdhocFilterEditForm.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

3 finding(s): 🔴 0 critical · 🟠 0 major · 🔵 3 minor

2 posted as inline comment(s) on the changed lines. 1 listed below.

Findings outside the changed lines

1 minor
  • 🔵 packages/app/src/components/DashboardFiltersModal/VariableNameInput.tsx:45 — "Unique name" field still reports errors as "Variable name…" → The ad hoc form relabels the field "Unique name" and says the filter is not a variable. Validation still goes through validateVariableName (packages/common-utils/src/filters.ts:1444), which returns 'Variable name is required', 'Variable names must start with a letter…' and 'This variable name is used by another filter'. Fix: pass a noun (e.g. label) into validateVariableName, or map its messages in VariableNameInput, so the errors match the label.

Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Deep Review

Scope: PR #3658 — ad hoc dashboard filter finishing touches (behind NEXT_PUBLIC_ENABLE_ADHOC_FILTERS). 9 production files + tests across packages/app and packages/common-utils, diffed against c220e164.

Intent: Treat an ad hoc filter's name as a unique identifier (not a $variable), make broadcast always-on, add an overlap warning, collapse the condition input, and add IN (SQL) / =| (PromQL) operators whose same-key conditions render as one OR'd expression.

The core logic is well built and thoroughly unit-tested. Correctness verified the chip-slice index alignment, mixed-operator grouping, the =| double-escape round-trip, the unreachable defensive throws, and the getDashboardVariableFilters ADHOC exclusion (all callers pass type). Security could not construct an injection path: SQL values go through escapeSqlString (backslash-first ordering), PromQL values survive escapeRegexValue + escapePromqlStringValue, and keys are verbatim by the same trust model as the rest of the query builder.

✅ No critical issues found. Recommendations below.

🟡 P2 — recommended

  • packages/common-utils/src/types.ts:1348 — SQL_ADHOC_FILTER_OPERATORS and PROMQL_ADHOC_FILTER_OPERATORS changed from the derived AdhocFilterOperatorSchema.options to two hand-maintained partitions with no guard that they jointly cover the enum, so a future operator added to the enum and the render switches but omitted from both lists compiles cleanly yet never appears in the dropdown and makes isAdhocConditionSupported return false for both source types.
    • Fix: Add a test asserting every AdhocFilterOperatorSchema option appears in at least one of the two lists, and comment the lists as enum-synced.
    • maintainability, testing
  • packages/app/tests/e2e/features/dashboard-adhoc-filters.spec.ts:59 — CI reported 2 failed + 1 flaky e2e tests on the reviewed HEAD (61e898ab0, itself the "fix ad hoc e2e specs" commit), and the specific failing specs could not be confirmed from the available data; the heaviest assertion churn (removed broadcast checkbox, new Unique name exact-text check, $conds variable-rendering change) is in this spec.
    • Fix: Confirm which specs fail in the CI run and resolve them before merge rather than relying on the flaky retry.
    • previous-comments, testing
🔵 P3 nitpicks (2)
  • .changeset/adhoc-filter-finishing-touches.md:4 — declares @hyperdx/api: patch, but the only packages/api changes in this diff are test files, which per the repo's PR-hygiene rule do not warrant a release.
    • Fix: Remove the @hyperdx/api line (the dependency bump via common-utils is auto-handled if needed); keep common-utils and app.
  • packages/common-utils/src/types.ts:2452 — isBroadcastEnabled: z.literal(true) makes any ad hoc filter stored with the field false/missing fail DashboardFilterSchema.parse; the read path (fetchDashboards) casts JSON without re-parsing so a dashboard load won't crash, but a save or template-conversion of such a legacy flag-gated filter would be rejected.
    • Fix: Confirm no dev-stored ad hoc filters predate the invariant, or accept undefined/false and coerce to true; otherwise the already-documented no-migration trade-off is acceptable given the feature flag.

Reviewers (7): correctness, security, testing, maintainability, project-standards, learnings-researcher, previous-comments.

Coverage notes:

  • 4 selected reviewers (adversarial, api-contract, kieran-typescript, agent-native) had not returned at synthesis time; their primary concern areas were checked directly by the orchestrator: the breaking-schema blast radius is contained (read path does not zod-parse; the external API's discriminated union does not expose the ADHOC variant at all), the =| escaping round-trip is correct, and no injection path was found.
  • No relevant prior learnings exist in docs/solutions/ (the only such tree belongs to the bundled plugin, not ClickStack).

Testing gaps:

  • SQL operator option list (in present, =| absent) is unasserted — only the PromQL list test was updated (FilterConditionEditor.test.tsx).
  • The now-dynamic invalid-condition reason (SQL has no … vs PromQL has no …) is unasserted for the SQL case.
  • The multi-filter overlap-warning name join (overlappingFilters.map(f => f.name).join(', ')) is exercised with only one overlapping filter.
  • Collapsed-chip remove/replace by index is safe only because the slice starts at index 0; a future non-zero offset would silently corrupt edits with no test to catch it.
  • No test asserts the operator lists jointly cover AdhocFilterOperatorSchema (enum/list drift).

Residual note (pre-existing/by-design): a click-to-filter include emits = conditions, which are AND'd (only IN/=| same-key conditions are OR'd), so including two values of one key yields zero rows — an easier user trap now that IN exists alongside =.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 467 passed • 1 skipped • 1362s

Status Count
✅ Passed 467
❌ Failed 0
⚠️ Flaky 1
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

Comment thread packages/app/src/DashboardFilters.tsx Outdated
const variableName = isFilterVariableEnabled(filter)
? getFilterVariableName(filter)
: undefined;
// An ad hoc filter's variable name can't be referenced.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — The check for "is this name a referenceable variable" now lives in three places

The rule isFilterVariableEnabled(filter) && !isAdhocFilter(filter) is now written out in getFilterEffect, in DashboardFiltersList.tsx:166, and again as filter.type === 'ADHOC' inside getDashboardVariableFilters (common-utils/src/filters.ts:1290). If another non-referenceable type is added, all three have to be updated by hand. Fix: export one predicate, such as isFilterReferenceableVariable, from common-utils/src/filters.ts and use it in all three places.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

One predicate decides whether a filter's name is a referenceable variable,
used by getDashboardVariableFilters, getFilterEffect, and the filters list.
Updates the ad hoc filter e2e specs for the removed broadcast checkbox and
the list no longer showing an ad hoc filter's name as a variable.
);
})}
{conditions
.slice(0, conditions.length - hiddenCount)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — New conditions and invalid conditions get hidden behind the "+N more" toggle

When the filter is collapsed and already has 3 or more conditions, a new condition from the add popover or from click-to-filter is appended at the end. The slice hides it at once, so the user only sees the counter go up and never sees the chip they just added. Conditions past the first three that are marked invalid (=| on SQL, IN/LIKE on PromQL) are hidden in the same way, along with their 'Filter not applied' marker. Fix: expand when conditions.length grows, or always render the newest and invalid chips. As a fallback, show the invalid count on the toggle.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

case '!~':
return `NOT match(${target}, ${literal})`;
case 'IN':
return `${target} IN (${values.map(toLiteral).join(', ')})`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — SQL IN list hand-rolls formatVariableValues(…, 'sqlstring')

The =| branch already reuses formatVariableValues({ values }, 'regex'). The IN branch rebuilds the same quoted, escapeSqlString-escaped, comma-joined list that formatVariableValues produces for the 'sqlstring' format (packages/common-utils/src/variables.ts:96-99). Fix: use ${target} IN (${formatVariableValues({ values }, 'sqlstring')}) and drop the local toLiteral mapping.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

This branch was successfully deployed

2 active deployments
Preview – hyperdx-oss — 61e898ab Deployed Nov 13, 2026 by vercel[bot]
Preview – hyperdx-storybook — 61e898ab Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-3 Standard — full human review required

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant