feat(datadog_agent): support v3 series metrics intake - #26264
Conversation
github-actions
Bot
added
the
domain: sources
There was a problem hiding this comment.
Pull request overview
Adds Datadog Agent v3 series metrics intake support to the datadog_agent source so Vector can ingest metrics from Agent versions that default to the v3 endpoint, and updates tests to exercise the new path.
Changes:
- Add Warp routing + decoding for
/api/intake/metrics/v3/seriesand/api/intake/metrics/v3beta/series, translating v3 payloads into Vector metrics. - Add unit test coverage for decoding a v3 series payload.
- Restore the floating Datadog Agent v7 leg in the Datadog metrics E2E test matrix and add the v3 proto to codegen.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/datadog-metrics/config/test.yaml | Unpins the v7 agent matrix leg to exercise current Agent behavior (v3 default). |
| src/sources/datadog_agent/tests.rs | Adds a v3 endpoint constant, a raw-bytes sender helper, and a v3 decode test. |
| src/sources/datadog_agent/mod.rs | Wires in the generated v3 metrics protobuf module. |
| src/sources/datadog_agent/metrics.rs | Adds v3 series route and decoding/translation logic. |
| proto/vector/dd_metric_v3.proto | Introduces the v3 intake proto definition used for decoding. |
| changelog.d/25790_datadog_agent_v3_series.feature.md | Adds a changelog fragment documenting the user-facing feature. |
| build.rs | Adds the v3 proto to the build-time proto compilation inputs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5330fba0d3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
github-actions
Bot
added
domain: sinks
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8be2d900a4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| tags.extend( | ||
| tagsets | ||
| .get(index) | ||
| .ok_or("invalid Datadog v3 tagset reference")? | ||
| .iter() |
There was a problem hiding this comment.
A small untrusted v3 payload can make each dictionary entry reference the preceding tagset multiple times (for example, two deltas resolving to the same negative reference), so each extend doubles the materialized tag count. A few dozen nested entries can therefore exhaust process memory even though the compressed request remains tiny and passes the body-size cap; reject tagsets whose cumulative expanded size exceeds a safe bound.
Useful? React with 👍 / 👎.
| timestamp = timestamp | ||
| .checked_add(data.timestamps[timestamp_idx]) | ||
| .ok_or("Datadog v3 timestamp overflow")?; |
There was a problem hiding this comment.
When a v3 point supplies a delta such as i64::MAX, this checked addition succeeds, but the resulting value is outside Chrono's supported timestamp range. The shared conversion later calls Utc.timestamp_opt(...).single().expect("invalid timestamp"), so a tiny malformed request panics its request task instead of returning 422; validate the accumulated timestamp before storing the point.
Useful? React with 👍 / 👎.
| interval: i64::from( | ||
| u32::try_from(data.intervals[index]).map_err(|_| "invalid Datadog v3 interval")?, | ||
| ), |
There was a problem hiding this comment.
For a v3 series whose interval exceeds u32::MAX / 1000 seconds, this conversion still accepts the value as u32, but the shared gauge/rate decoder later computes interval * 1000 in u32. Release builds silently wrap the emitted interval while overflow-checking builds panic the request task, so validate that the interval can be represented in Vector's millisecond field before constructing the temporary series.
Useful? React with 👍 / 👎.
Summary
Adds support for the Datadog Agent v3 series metrics intake format used by default by Datadog Agent 7.81.0 and newer.
The
datadog_agentsource now accepts the dictionary-encoded protobuf payload at/api/intake/metrics/v3/seriesand/api/intake/metrics/v3beta/series, translates it into Vector metrics, and preserves metric names, tags, resources, units, intervals, timestamps, values, and origin metadata.The Datadog metrics e2e matrix also restores the floating Agent
7test leg so current Agent versions exercise the v3 path.References
Closes: #25790
Vector configuration
No new configuration is required. Existing
datadog_agentsource configurations accept the new intake endpoint automatically.How did you test this PR?
cargo check --lib --no-default-features --features sources-datadog_agentcargo test --lib --no-default-features --features sources-datadog_agent datadog_agent::cargo clippy --lib --no-default-features --features sources-datadog_agent -- -D warningscargo fmt --all -- --checkIs this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.