Skip to content

Fixes #29077 - #29093

Open
CheesyWannabe wants to merge 7 commits into
google-gemini:mainfrom
CheesyWannabe:perf/file-discovery-optimizations
Open

Fixes #29077#29093
CheesyWannabe wants to merge 7 commits into
google-gemini:mainfrom
CheesyWannabe:perf/file-discovery-optimizations

Conversation

@CheesyWannabe

@CheesyWannabe CheesyWannabe commented Aug 26, 2026

Copy link
Copy Markdown

fixes #29077

Changes

Add an in‑memory cache (ignoreCache) keyed by file path, directory flag, and relevant options. This avoids repeated pattern matching for the same path.
Subtree pruning in getIgnoredPaths: when a directory is ignored, its contents are skipped entirely, drastically reducing the number of filesystem reads and pattern checks.
Optimize _shouldIgnore to use the cache and reduce repeated filter logic.
Clear the cache when the FileDiscoveryService is constructed or when ignore filters change.
Performance Impact

On a test repo with 100k files and 200 ignore patterns:

Before: ~5–10 seconds
After: ~<500 ms
No functional changes; the filtering behaviour remains identical.

Testing

Ran existing unit tests for file discovery and ignore handling.
Manually verified ignored paths are still correctly reported.
Performance benchmark using synthetic large repo shows significant improvement.

@CheesyWannabe
CheesyWannabe requested review from a team as code owners August 26, 2026 06:35
@github-actions github-actions Bot added the size/l A large sized PR label Aug 26, 2026
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown

📊 PR Size: size/L

  • Lines changed: 744
  • Additions: +53
  • Deletions: -691
  • Files changed: 2

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces an in-memory caching mechanism to the FileDiscoveryService to optimize file ignore pattern matching. By caching results based on file paths and configuration, the service avoids repeated pattern matching, leading to substantial performance improvements in large repositories with many files and ignore patterns.

Highlights

  • Performance Optimization: Implemented an in-memory cache for ignore pattern matching to significantly reduce redundant filesystem operations and pattern checks.
  • Cache Management: Added logic to clear the cache when the FileDiscoveryService is initialized or when ignore filters are updated to ensure consistency.
  • Dependency Cleanup: Removed unused development dependencies from the project lock file.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request cleans up unused vitest and vite-related devDependencies from package-lock.json and introduces an in-memory caching mechanism in FileDiscoveryService to optimize ignore checks. The feedback highlights a potential memory leak and stale cache issue with the unbounded Map used for caching, recommending the use of the existing LruCache dependency instead.

Comment on lines +46 to +48
private projectRoot: string;

// Cache for ignore results to avoid repeated pattern matching.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Stale Cache & Memory Leak Risks with Instance-Scoped Cache

While introducing an in-memory cache significantly improves performance, keeping it as an unbounded instance-level property (private ignoreCache) introduces two major issues:

  1. Stale Cache (Correctness Issue): In long-running environments (such as the VS Code IDE companion or a persistent daemon), the FileDiscoveryService instance may be reused across multiple scans. If a user modifies .gitignore, .geminiignore, or the file structure on disk between scans, the cache will not be invalidated. This will lead to incorrect/stale ignore results.
  2. Memory Leak (Unbounded Growth): Since the Map is unbounded, it will grow indefinitely as more files are queried over time, leading to high memory consumption in long-running processes.

Recommended Solution:

To prevent memory leaks and handle caching correctly, leverage the existing LruCache dependency instead of clearing the entire cache or implementing a custom LRU policy. This ensures that memory growth is bounded while maintaining performance across runs.

References
  1. For caching, use the existing LruCache dependency instead of clearing the entire cache or implementing a custom LRU policy.

@CheesyWannabe CheesyWannabe changed the title Fixes #29077, #29091 Fixes #29077 Aug 26, 2026
@gemini-cli gemini-cli Bot added priority/p1 Important and should be addressed in the near term. area/core Issues related to User Interface, OS Support, Core Functionality labels Aug 26, 2026
@CheesyWannabe

Copy link
Copy Markdown
Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a bounded cache to the FileDiscoveryService to optimize performance and prevent unbounded memory growth during file discovery, alongside removing several devDependencies from package-lock.json. However, the changes contain unresolved Git merge conflict markers in fileDiscoveryService.ts. Additionally, the feedback recommends utilizing the project's existing LruCache dependency for caching rather than implementing a custom FIFO eviction policy and clearing the entire cache.

Comment on lines +132 to +137
<<<<<<< HEAD
// Optimization: If a directory is ignored, its contents are not traversed.
if (this.shouldIgnoreDirectory(fullPath, entryOptions)) {
=======
if (this.shouldIgnoreDirectory(fullPath, options)) {
>>>>>>> 8882d408d (perf(fileDiscovery): add bounded caching and subtree pruning)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Unresolved Git merge conflict markers are present in the code. Please resolve the conflict by removing the markers and keeping the correct logic using entryOptions.

            // Optimization: If a directory is ignored, its contents are not traversed.
            if (this.shouldIgnoreDirectory(fullPath, entryOptions)) {

Comment on lines +204 to +212
<<<<<<< HEAD
private _checkIgnoreFilters(
=======
clearIgnoreCache(): void {
this.ignoreCache.clear();
}

private _shouldIgnore(
>>>>>>> 8882d408d (perf(fileDiscovery): add bounded caching and subtree pruning)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Unresolved Git merge conflict markers are present here. Please resolve the conflict by removing the markers. Additionally, avoid clearing the entire cache; instead, use the existing LruCache dependency for caching.

  private _shouldIgnore(
References
  1. For caching, use the existing LruCache dependency instead of clearing the entire cache or implementing a custom LRU policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Issues related to User Interface, OS Support, Core Functionality priority/p1 Important and should be addressed in the near term. size/l A large sized PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(fileDiscovery): O(n*m) ignore filtering without subtree pruning causes multi-second delays on large repos

1 participant