Pipe: Cache table-model pattern matches by table

[shuwenwei]

Summary

• Cache table-model source matches by database and table name instead of device ID.
• Match each table only once per event.
• Keep table-model TsFile table names complete for privilege checks.

Validation

• git diff --check
• No compile/build/test run per repository preference.

[codecov]

Codecov Report

❌ Patch coverage is 64.70588% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.96%. Comparing base (a05c63e) to head (d71c963).
⚠️ Report is 24 commits behind head on master.

Files with missing lines	Patch %	Lines
...ava/org/apache/iotdb/db/auth/AuthorityChecker.java	0.00%	4 Missing ⚠️
...n/realtime/matcher/CachedSchemaPatternMatcher.java	84.61%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18648      +/-   ##
============================================
+ Coverage     42.82%   42.96%   +0.14%     
- Complexity      442      486      +44     
============================================
  Files          5451     5469      +18     
  Lines        395425   396628    +1203     
  Branches      51805    52003     +198     
============================================
+ Hits         169349   170424    +1075     
- Misses       226076   226204     +128     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Caideyipi]

Thanks for this optimization. The table-level cache is a useful direction. I left two inline comments, including one authorization-cache invalidation race that can affect correctness.

Could you add focused coverage for the changed matching behavior as well?

• Multiple devices from one table should invoke table matching only once.
• A multi-table TsFile should still record every table after all sources have matched.
• Ideally, a cache-invalidation/concurrent-refill test should demonstrate that stale authorization results cannot survive an invalidation.

[Caideyipi]

[P2] Prevent stale authorization results from refilling this cache

AuthorityChecker.invalidateCache() currently clears this matcher before invalidating the authority cache. That leaves a stale-refill window: a concurrent event can miss this cache, read the old authorization result, and cache a table-wide denial here. After the grant invalidation completes, later events can keep hitting that stale entry, remain unmatched, and advance progress until another invalidation or source change.

Please invalidate the authority cache before this matcher (also in invalidateAllCache()), or coordinate the two caches with a generation/version so that an entry computed before invalidation cannot be installed afterward.

[Caideyipi]

[P3] Resolve the table-model database name only for table events

getTableModelDatabaseName() is evaluated before the model check, so every insertion event, including tree-model events, now performs its lazy substring/toLowerCase initialization on the realtime write path even though the value is only used for table-model matching below. Please move this lookup into the table-model branch.

[shuwenwei]

Addressed in 77731ba.

• P2: reordered cache invalidation so the authority cache is invalidated before the pipe matcher cache in both invalidateCache() and invalidateAllCache(), preventing stale authorization results from being cached back into the matcher.
• P3: tableModelDatabaseName is now resolved only inside the table-model matching branch, so tree-model insertion events no longer trigger the lazy substring/lowercase work.
• Added focused matcher tests covering per-table matching deduplication and complete tableNames collection for multi-table table-model TsFile events after all sources have matched.

The testCachedMatcher performance test is unchanged from master; its runtime is pre-existing.

[Caideyipi]

[P1] Guard authority-cache refills with an invalidation generation

Reordering these invalidations only removes matcher entries produced by an in-flight match; it does not prevent the underlying authority cache from being refilled after its invalidation. A matcher miss holds the matcher read lock while checkCanSelectFromTable4Pipe() may issue a ConfigNode RPC. If a pre-revocation successful response returns after the author cache is cleared, ClusterAuthorityFetcher.checkPrivilegeFromConfigNode() can call putUserCache() with the old User while this thread is waiting for the matcher write lock. The following matcher invalidation then clears only the matcher entry, leaving the stale authority entry behind; the next event repopulates the matcher and a busy pipe can continue passing a revoked user.

Please add a generation/epoch to authority-cache loads (capture it before the RPC and only install the response if unchanged), or otherwise coordinate refills atomically with both invalidations. A latch-based test for this exact interleaving would prevent regression.

[shuwenwei]

Thanks for the detailed analysis. We understand the concern about authority-cache refill after invalidation.

This refill race is pre-existing and orthogonal to this PR: the unconditional putUserCache() path in ClusterAuthorityFetcher / BasicAuthorityCache is unchanged, and this PR is scoped to table-level matcher caching and its invalidation behavior. We do not plan to expand this PR into an authority-cache generation/epoch redesign.

If the maintainers consider it necessary, a separate PR can be opened later to handle authority-cache refill coordination and add the latch-based interleaving test there. For this PR, we would prefer to keep the matcher-related invalidation ordering fix and focused matcher tests.

[shuwenwei]

Respectfully, we are not going to fix the authority-cache refill race in this PR. This PR is scoped to the table-level matcher cache change, while the refill race originates in the pre-existing authority-cache / ConfigNode load path and is independent of the modifications made here. Fixing it requires an authority-cache generation/epoch redesign plus dedicated concurrency tests, which is outside the objective of this PR.

We will keep this PR focused on the table-level matcher behavior and its matcher-side invalidation ordering. If the maintainers consider the authority-cache refill race mandatory for merge, it should be handled in a separate PR.

The separate test concern about making the deduplication test fail without the new table-level guard is addressed in d71c963.

[Caideyipi]

The table-level cache changes are directionally correct, but the authorization-cache refill race from thread #4032472791 remains a correctness blocker. An in-flight pre-invalidation ConfigNode response can still call putUserCache() after the invalidation has cleared the authority cache; a following pipe match can then cache that stale authorization result in the new per-table matcher. Reordering the two clears only narrows the timing and cannot prevent this interleaving. Please guard cache loads with an invalidation generation/epoch (or otherwise coordinate response installation with invalidation) and add the latch-based regression test before merge.

[Caideyipi]

[P2] Make this test fail without table deduplication

With one registered source, matchedSources.addAll(sources) fills the set on the first table, so the matcher exits before it reaches the second device. The pre-change implementation therefore also reports tableMatchCount == 1, and this test would pass without the new tableNames.add(...) guard. Please register at least two sources and add only one (or otherwise keep the match set incomplete) so a duplicate invocation is observable.

[shuwenwei]

@Caideyipi Thanks for the re-review.

We acknowledge the authority-cache refill race, but we are not going to fix it in this PR because it is outside this PR change objective. The objective here is the table-level matcher cache and matcher-side invalidation behavior. The refill race originates in the pre-existing authority-cache / ConfigNode load path, which this PR does not modify; fixing it requires an authority-cache generation/epoch design and dedicated concurrency coverage.

We are therefore not expanding this PR to include that redesign or the latch-based authority-cache test. If the maintainers consider the P1 issue mandatory, it should be handled in a separate PR.

The separate test-review concern is addressed in d71c963.

[Caideyipi]

Re-reviewed at d71c963. The original matcher-side invalidation ordering concern and the focused matcher coverage are addressed. The remaining authority-cache refill race is a pre-existing limitation in the shared authority-cache/ConfigNode load path; this reorder changes the observable interleaving but does not make that underlying issue part of this scoped Pipe optimization. The strengthened deduplication test now exercises the intended behavior. LGTM. The unrelated failed checks should still be rerun before merge.