Skip to content

Conversation

@pulpdrew
Copy link
Contributor

@pulpdrew pulpdrew commented Jan 8, 2026

Closes HDX-3124

Summary

This PR makes the following changes

  1. Date ranges for all MV queries are now aligned to the MV Granularity
  2. Each chart type now has an indicator when the date range has been adjusted to align with either the MV Granularity or (in the case of Line/Bar charts) the Chart Granularity.
  3. The useQueriedChartConfig, useRenderedSqlChartConfig, and useOffsetPaginatedQuery hooks have been updated to get the MV-optimized chart configuration from the useMVOptimizationExplanation, which allows us to share the EXPLAIN ESTIMATE query results between the MV Optimization Indicator (the lightning bolt icon on each chart) and the chart itself. This roughly halves the number of EXPLAIN ESTIMATE queries that are made.

Demo

Screenshot 2026-01-08 at 11 42 39 AM Screenshot 2026-01-08 at 11 40 54 AM

Testing

To test locally with an MV, you can use the following DDL

DDL For an MV
CREATE TABLE default.metrics_rollup_1m
(
   `Timestamp` DateTime,
   `ServiceName` LowCardinality(String),
   `SpanKind` LowCardinality(String),
   `StatusCode` LowCardinality(String),
   `count` SimpleAggregateFunction(sum, UInt64),
   `sum__Duration` SimpleAggregateFunction(sum, UInt64),
   `avg__Duration` AggregateFunction(avg, UInt64),
   `quantile__Duration` AggregateFunction(quantileTDigest(0.5), UInt64),
   `min__Duration` SimpleAggregateFunction(min, UInt64),
   `max__Duration` SimpleAggregateFunction(max, UInt64)
)
ENGINE = AggregatingMergeTree
PARTITION BY toDate(Timestamp)
ORDER BY (Timestamp, StatusCode, SpanKind, ServiceName)
SETTINGS index_granularity = 8192;

CREATE MATERIALIZED VIEW default.metrics_rollup_1m_mv TO default.metrics_rollup_1m
(
    `Timestamp` DateTime,
    `ServiceName` LowCardinality(String),
    `SpanKind` LowCardinality(String),
    `version` LowCardinality(String),
    `StatusCode` LowCardinality(String),
    `count` UInt64,
    `sum__Duration` Int64,
    `avg__Duration` AggregateFunction(avg, UInt64),
    `quantile__Duration` AggregateFunction(quantileTDigest(0.5), UInt64),
    `min__Duration` SimpleAggregateFunction(min, UInt64),
    `max__Duration` SimpleAggregateFunction(max, UInt64)
)
AS SELECT
    toStartOfMinute(Timestamp) AS Timestamp,
    ServiceName,
    SpanKind,
    StatusCode,
    count() AS count,
    sum(Duration) AS sum__Duration,
    avgState(Duration) AS avg__Duration,
    quantileTDigestState(0.5)(Duration) AS quantile__Duration,
    minSimpleState(Duration) AS min__Duration,
    maxSimpleState(Duration) AS max__Duration
FROM default.otel_traces
GROUP BY
    Timestamp,
    ServiceName,
    SpanKind,
    StatusCode;

@changeset-bot
Copy link

changeset-bot bot commented Jan 8, 2026

🦋 Changeset detected

Latest commit: 1829c1f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/common-utils Patch
@hyperdx/app Patch
@hyperdx/api Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link

vercel bot commented Jan 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
hyperdx-v2-oss-app Ready Ready Preview, Comment Jan 9, 2026 4:07pm

@claude
Copy link

claude bot commented Jan 8, 2026

PR Review

Critical Issues

None found. The implementation looks solid.

Code Quality Observations

Strong Points:

  • ✅ Comprehensive test coverage added for all chart components
  • ✅ Proper use of useMVOptimizationExplanation hook to share EXPLAIN ESTIMATE queries
  • ✅ Date range alignment logic correctly moved to common-utils for reuse
  • ✅ Proper handling of the structuredClone polyfill in test setup

Minor Suggestions (Non-blocking):

  1. packages/app/src/hooks/useChartConfig.tsx:281 - Consider adding error handling if mvOptimizationData?.optimizedConfig causes unexpected query failures (currently falls back to original config, which is good)

  2. packages/app/src/components/DBTimeChart.tsx:598-606 - The MV indicator is now passed queriedConfig instead of config. Verify this doesn't break any existing MV optimization display logic (likely fine, but worth double-checking in staging)

  3. Test pattern - The new chart component tests follow a consistent pattern and properly mock dependencies. Consider adding integration tests that verify the date range alignment behavior end-to-end with actual ClickHouse queries (future work)

Summary

No critical issues found. The PR successfully:

  • Reduces EXPLAIN ESTIMATE query count by ~50%
  • Aligns date ranges to MV granularity for query optimization
  • Adds clear UI indicators for date range adjustments
  • Maintains backward compatibility with showDateRangeIndicator={false} flag

The code follows HyperDX patterns and TypeScript best practices. Ready to merge after any final manual testing with the DDL provided in the PR description.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

E2E Test Results

All tests passed • 59 passed • 4 skipped • 748s

Status Count
✅ Passed 59
❌ Failed 0
⚠️ Flaky 1
⏭️ Skipped 4

Tests ran across 4 shards in parallel.

View full report →

"@types/ungap__structured-clone": "^1.2.0",
"@typescript-eslint/eslint-plugin": "^8.48.1",
"@typescript-eslint/parser": "^8.48.1",
"@ungap/structured-clone": "^1.3.0",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this polyfill for our tests, since the existing JSON.parse(JSON.stringify(obj)) workaround fails to correctly clone Dates (and other non-primitive types)

return Object.values(Granularity).includes(value as Granularity);
};

export function getAlignedDateRange(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this and the related tests into common-utils, unchanged.

@pulpdrew pulpdrew marked this pull request as ready for review January 8, 2026 16:47
@pulpdrew pulpdrew requested review from a team and teeohhem and removed request for a team January 8, 2026 16:48
Copy link
Contributor

@teeohhem teeohhem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff. Thanks!

@kodiakhq kodiakhq bot merged commit 0c16a4b into main Jan 9, 2026
12 checks passed
@kodiakhq kodiakhq bot deleted the drew/mv-date-range-alignment branch January 9, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants