Skip to content

in_node_exporter_metrics: add netdev ignore regex#11819

Open
stondo wants to merge 2 commits into
fluent:masterfrom
stondo:in_node_exporter_metrics-netdev-ignore-regex
Open

in_node_exporter_metrics: add netdev ignore regex#11819
stondo wants to merge 2 commits into
fluent:masterfrom
stondo:in_node_exporter_metrics-netdev-ignore-regex

Conversation

@stondo

@stondo stondo commented May 18, 2026

Copy link
Copy Markdown

Summary

Add netdev.ignore_device_regex to the node_exporter_metrics input so Linux network device metrics can be filtered by regular expression before samples are emitted.

This mirrors the existing regex filtering model used by the diskstats and filesystem collectors, and covers dynamic interface names such as veth0, veth1, etc. Exact label deletion after collection is not enough for this case because the interface names are runtime-generated.

Example configuration

[INPUT]
    Name                       node_exporter_metrics
    Tag                        node.metrics
    Scrape_Interval            1
    Metrics                    netdev
    netdev.ignore_device_regex ^(lo|podman|veth.*)$

With this configuration, node_network_*{device="lo"}, device="podman", and dynamic device="veth*" series are skipped while physical or host-relevant interfaces continue to be emitted.

Testing

  • Example configuration file for the change
  • Debug log output from testing the change
  • [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

Build verification:

cmake -S . -B build -DFLB_CONFIG_YAML=Off -DFLB_TESTS_RUNTIME=Off -DFLB_TESTS_INTERNAL=Off -DFLB_EXAMPLES=Off
cmake --build build -j8 --target flb-plugin-in_node_exporter_metrics fluent-bit-bin

Local runtime smoke test on Linux:

[SERVICE]
    Flush     1
    Log_Level info

[INPUT]
    Name                       node_exporter_metrics
    Tag                        node.metrics
    Scrape_Interval            1
    Metrics                    netdev
    netdev.ignore_device_regex ^lo$

[OUTPUT]
    Name  prometheus_exporter
    Match *
    Host  127.0.0.1
    Port  9927

Smoke test result from http://127.0.0.1:9927/metrics:

node_network_total=32
lo_count=0

Sample retained host interfaces:

node_network_receive_bytes_total{device="wlp0s20f3"} 2269105625
node_network_receive_bytes_total{device="zcctun0"} 3550762101
node_network_receive_packets_total{device="wlp0s20f3"} 2971213
node_network_receive_packets_total{device="zcctun0"} 5405127

Log excerpt from the same smoke test:

[ info] [input:node_exporter_metrics:node_exporter_metrics.0] initializing
[ info] [input:node_exporter_metrics:node_exporter_metrics.0] storage_strategy='memory' (memory only)
[ info] [input:node_exporter_metrics:node_exporter_metrics.0] path.rootfs = /
[ info] [input:node_exporter_metrics:node_exporter_metrics.0] path.procfs = /proc
[ info] [input:node_exporter_metrics:node_exporter_metrics.0] path.sysfs  = /sys
[ info] [input:node_exporter_metrics:node_exporter_metrics.0] thread instance initialized

Embedded-device validation was also done with a downstream Yocto build on an ARM64 Linux device. With netdev.ignore_device_regex: "^(lo|podman|veth.*)$":

node_network_total=80
lo_count=0
podman_count=0
veth_count=0
eth0_count=16
wan0_count=16
br_lan_count=16

Documentation

  • Documentation required for this feature

The plugin documentation is maintained in fluent/fluent-bit-docs. I can open a follow-up docs PR for this new option if the feature shape is accepted.

Backporting

  • Backport to latest stable release.

No backport requested yet. This is a small additive configuration option and should be low risk if maintainers want it on a stable branch.

Summary by CodeRabbit

  • New Features

    • Added netdev.ignore_device_regex option to filter out network device metrics via a regex.
  • Bug Fixes

    • Fixed cleanup on collector activation failures to remove leftover registrations and reset state when initialization fails.
    • Ensure ignored-device patterns are compiled and released cleanly during plugin init/exit.

@coderabbitai

coderabbitai Bot commented May 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0b72de4f-ec43-4b94-8836-c749e7f210b5

📥 Commits

Reviewing files that changed from the base of the PR and between c67cfed and 131cfdf.

📒 Files selected for processing (3)
  • plugins/in_node_exporter_metrics/ne.c
  • plugins/in_node_exporter_metrics/ne.h
  • plugins/in_node_exporter_metrics/ne_netdev_linux.c
✅ Files skipped from review due to trivial changes (1)
  • plugins/in_node_exporter_metrics/ne.h
🚧 Files skipped from review as they are similar to previous changes (2)
  • plugins/in_node_exporter_metrics/ne.c
  • plugins/in_node_exporter_metrics/ne_netdev_linux.c

📝 Walkthrough

Walkthrough

Adds an optional netdev.ignore_device_regex config (stored in struct flb_ne), compiles/checks/cleans up that regex in netdev init/update/exit, and fixes collector init error-path by deleting a scheduled collector and resetting coll_fd when cb_init fails.

Changes

Network device regex filtering in node exporter metrics

Layer / File(s) Summary
Data structure and configuration contract
plugins/in_node_exporter_metrics/ne.h, plugins/in_node_exporter_metrics/ne.c
struct flb_ne gains netdev_regex_skip_devices and netdev_regex_skip_devices_text. Configuration map registers netdev.ignore_device_regex (optional string, default NULL) mapped to netdev_regex_skip_devices_text.
Device filtering implementation and lifecycle
plugins/in_node_exporter_metrics/ne_netdev_linux.c
Adds netdev_skip_device helper; ne_netdev_init compiles the ignore-regex from config (fails init on compile error); netdev_update sanitizes device names and skips metric updates for devices matching the regex; ne_netdev_exit destroys the compiled regex.
Collector init error-path cleanup
plugins/in_node_exporter_metrics/ne.c
On cb_init failure, if a collector update was scheduled (cb_update set and coll_fd valid), deletes the collector (flb_input_collector_delete) and resets coll_fd to -1 before returning the error.

Sequence Diagram

sequenceDiagram
  participant Init as ne_netdev_init()
  participant Helper as netdev_skip_device()
  participant Update as netdev_update()
  participant Exit as ne_netdev_exit()
  Init->>Init: Compile regex from config text
  Update->>Helper: Check device name against compiled regex
  Helper-->>Update: Return skip=true/false
  Update->>Update: Continue (skip) if matched
  Exit->>Exit: Destroy compiled regex
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Suggested reviewers

  • edsiper
  • cosmo0920

Poem

🐰 I hop through code where interfaces hide,
I compile a net that filters the tide.
Skip noisy names with a regex bright,
Clean timers left in the fading light.
Metrics neat — the rabbit's delight.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and directly summarizes the main change: adding a network device ignore regex feature to the node_exporter_metrics input plugin.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7773cbde1e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread plugins/in_node_exporter_metrics/ne_netdev_linux.c
@stondo

stondo commented Jun 2, 2026

Copy link
Copy Markdown
Author

I checked the init/error path more closely and the concern was valid: activate_collector() can register the collector before cb_init() completes successfully. If cb_init() then fails, the collector callback may remain armed and later run against partially initialized netdev state.

I pushed a follow-up fix in:

  • 4920043e6 fix: delete collector on init failure

That change removes the already-registered collector and resets coll_fd when cb_init() fails, so the failure path does not leave a live callback behind.

Please take another look.

@stondo stondo force-pushed the in_node_exporter_metrics-netdev-ignore-regex branch 2 times, most recently from 53917e6 to c67cfed Compare June 2, 2026 13:49
@cosmo0920

Copy link
Copy Markdown
Contributor

We need to suppress our commit linter's complain:

Run python .github/scripts/commit_prefix_check.py

❌ Commit c67cfedb6e failed:
Subject prefix 'fix:' does not match files changed.
Expected one of: in_node_exporter_metrics:

stondo added 2 commits June 11, 2026 22:38
Add netdev.ignore_device_regex so Linux network device metrics can be skipped by regular expression before metric samples are emitted.

This mirrors the existing regex filtering model used by diskstats and filesystem collectors, and covers dynamic interface names such as veth0 where exact label deletion is not sufficient.

Tested with:

cmake -S . -B build -DFLB_CONFIG_YAML=Off -DFLB_TESTS_RUNTIME=Off -DFLB_TESTS_INTERNAL=Off -DFLB_EXAMPLES=Off

cmake --build build -j8 --target flb-plugin-in_node_exporter_metrics fluent-bit-bin

Also smoke-tested locally with node_exporter_metrics netdev collection and netdev.ignore_device_regex ^lo$, confirming device="lo" was absent while other network interfaces remained present.

Signed-off-by: Stefano Tondo <stondo@gmail.com>
Signed-off-by: Stefano Tondo <stondo@gmail.com>
@stondo stondo force-pushed the in_node_exporter_metrics-netdev-ignore-regex branch from c67cfed to 131cfdf Compare June 11, 2026 20:38
@stondo

stondo commented Jun 12, 2026

Copy link
Copy Markdown
Author

Thanks @cosmo0920, fixed in the force-push. Both commits now use the in_node_exporter_metrics: prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants