in_tail: fix file append cleanup on failure

[sandy2008]

This fixes a tail setup failure path where a file could already be linked into the static/event tracking lists before a later setup step failed. In low-resource cases that left list, hash, and accounting state partly registered, which can make the next scan behave oddly.

What changed:

• roll back listed files through the normal remove path
• clean up partially initialized files by hand before they are listed
• check static/event hash registration failures
• keep the static file count in sync when removing static files
• avoid moving a static file to the event list if event hash registration fails

Testing:

• cmake -S . -B build -DFLB_TESTS_RUNTIME=On -DFLB_TESTS_INTERNAL=On
• cmake --build build -j8
• ctest --test-dir build -R in_tail --output-on-failure
• tests/integration/.venv/bin/python -m pytest tests/integration/scenarios/in_tail/tests/test_in_tail_001.py -q -k "copytruncate_with_stale_writer or follows_rename_rotation or handles_multiple_rename_rotations or multi_file_rapid_rotation or rotate_wait_keeps_old_inode_then_purges_it"
• Linux Docker: FLUENT_BIT_BINARY=/build/bin/fluent-bit VALGRIND=1 VALGRIND_STRICT=1 /tmp/flb-it-venv/bin/python -m pytest tests/integration/scenarios/in_tail/tests/test_in_tail_001.py -q -k "copytruncate_with_stale_writer or follows_rename_rotation or handles_multiple_rename_rotations or multi_file_rapid_rotation or rotate_wait_keeps_old_inode_then_purges_it"

Summary by CodeRabbit

• Bug Fixes
  • Improved error handling and resource cleanup in file tail monitoring to prevent resource leaks and ensure graceful failure recovery.
  • More reliable file tracking and registration logic for static and event monitoring modes, including rollback on failures and corrected bookkeeping so file counts and monitoring state remain accurate.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 141e45bb-52e1-47a8-82ec-38f36658c2cf

📥 Commits

Reviewing files that changed from the base of the PR and between 3c7d8f7 and 1f032cc.

📒 Files selected for processing (2)

• plugins/in_tail/tail_file.c
• plugins/in_tail/tail_fs_stat.c

---

📝 Walkthrough

Walkthrough

Refactors error handling and cleanup in the tail plugin: flb_tail_file_append now uses staged cleanup labels and explicit rollback of registrations/resources; flb_tail_file_remove decrements static-file counters; flb_tail_file_to_event inserts into event hash before list swap; flb_tail_fs_stat_remove always frees the fs backend when present.

Changes

Cohort / File(s)	Summary
Tail file append & lifecycle plugins/in_tail/tail_file.c	Rewrote flb_tail_file_append error paths into multiple err_* labels with staged resource release (fd, hash/list registrations, fs monitoring, ML streams, msgpack/dmode buffers, decompression/encoders). Added allocation checks for docker/multiline/encoder buffers, moved files-opened metric increment to post-encoder creation, and adjusted flb_tail_file_remove/flb_tail_file_to_event to correctly handle static→event transitions and rollback on hash registration failure.
FS stat backend cleanup plugins/in_tail/tail_fs_stat.c	Changed flb_tail_fs_stat_remove to free the file->fs_backend whenever non-NULL and clear the pointer, making stat backend cleanup independent of file->tail_mode.

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant TailFile as Tail File
    participant Hash as Event/Static Hash
    participant FSWatch as FS Watcher
    participant Decoder
    participant Encoder
    participant Metrics

    Caller->>TailFile: flb_tail_file_append(file)
    TailFile->>TailFile: open fd (fd = -1 -> set)
    TailFile->>Hash: register file in hash (static/event)
    alt hash registration fails
        Hash-->>TailFile: error
        TailFile->>FSWatch: rollback fs monitor (if registered)
        TailFile->>Decoder: free decompression ctx (if created)
        TailFile->>Encoder: free encoders/buffers (if created)
        TailFile-->>Caller: return error
    else hash succeeds
        TailFile->>FSWatch: add fs monitoring (if event mode)
        TailFile->>Decoder: create decompression/hash/name contexts
        TailFile->>Encoder: create encoder & buffers
        alt encoder creation fails
            Encoder-->>TailFile: error
            TailFile->>Hash: unregister from hash/list
            TailFile->>FSWatch: rollback fs monitor
            TailFile->>Decoder: free decompression ctx
            TailFile-->>Caller: return error
        else success
            TailFile->>Metrics: increment files-opened metric
            TailFile-->>Caller: success
        end
    end

Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested Reviewers

• edsiper
• koleini

Poem

🐰 A rabbit guards each opened file,
Hops back on errors with a tidy smile,
Hashes unmade, monitors rewound,
Buffers freed, no leak left found,
I nibble crumbs of cleanup, one hop at a time.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly and accurately describes the primary change: fixing file append cleanup on failure paths in the in_tail plugin.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c7d8f752a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@plugins/in_tail/tail_file.c`:
- Around line 2157-2165: flb_hash_table_add failure leaves fs_backend leaked for
the stat backend because flb_tail_fs_stat_remove only frees fs_backend when
file->tail_mode == FLB_TAIL_EVENT; update flb_tail_fs_stat_remove to free
file->fs_backend unconditionally (or null it after free) so resources allocated
by flb_tail_fs_stat_add are always released on cleanup, and keep existing guards
for other event-specific cleanup; reference functions: flb_tail_fs_stat_add,
flb_tail_fs_stat_remove, flb_tail_fs_remove, symbols: fs_backend, tail_mode,
FLB_TAIL_EVENT, FLB_TAIL_STATIC, and event_hash/flb_hash_table_add to locate the
related code paths.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0de8aca1-40fa-4a59-b4c3-a72bc02c83ac

📥 Commits

Reviewing files that changed from the base of the PR and between 29deec9 and 3c7d8f7.

📒 Files selected for processing (1)

• plugins/in_tail/tail_file.c