Indexing works in parallel instead of series.

[Takitxt]

Addressed Issues:

Fixes #1436 : Indexing works in series in the application while folder Uploads.

• Fixes the issue where newly uploaded folders wait for a previous folder's AI Tagging pass before their own indexing can start.

Screenshots/Recordings:

PictoPy Before:

Screen.Recording.2026-08-06.at.10.24.28.AM.mov

PictoPy After:

Screen.Recording.2026-08-06.at.10.25.29.AM.mov

Additional Notes:

Files changed: 5

backend/main.py:

Added a dedicated indexing process pool that is initialized and shut down alongside the existing executor.

from app.config.settings import (
    DATABASE_PATH,
    THUMBNAIL_IMAGES_PATH,
    INDEXING_MAX_WORKERS,
)

app.state.indexing_executor = ProcessPoolExecutor(max_workers=INDEXING_MAX_WORKERS)

backend/app/config/settings.py:

Added a configurable INDEXING_MAX_WORKERS setting to control the size of the dedicated indexing pool.

# Separate pool for folder indexing so it never queues behind AI tagging.
INDEXING_MAX_WORKERS = _get_env_int("INDEXING_MAX_WORKERS", 2, min_value=1, max_value=8)

backend/app/routes/folders.py:

• Updated add_folder to submit indexing jobs to the dedicated indexing pool instead of the shared executor.

• Added a follow-up tagging sweep that is queued only after folder indexing completes, ensuring images indexed after an earlier tagging pass are not missed.

def _queue_post_index_tagging_sweep(app_state: State) -> None:
    """
    Runs after folder indexing completes to trigger a follow-up tagging sweep.
    Prevents images indexed after an earlier tagging pass from being missed.
    """
    try:
        app_state.executor.submit(post_AI_tagging_enabled_sequence)
    except Exception as e:
        logger.error(f"Failed to queue post-index tagging sweep: {e}")

backend/tests/test_folders.py:

• mocks the new pool; asserts indexing runs on it and the original pool is untouched; new test proves the catch-up sweep is queued only once the indexing future resolves

Conclusion:

• Now, indexing process works in parallel instead of waiting for another folder to complete AI tagging process, and if a folder is empty it shows up instantly instead of waiting.

(Still AI tagging process works in series but it is alright because it is in the need to go through each and every image, and changing it requires a lot of redesigning.)

AI Usage Disclosure:

We encourage contributors to use AI tools responsibly when creating Pull Requests. While AI can be a valuable aid, it is essential to ensure that your contributions meet the task requirements, build successfully, include relevant tests, and pass all linters. Submissions that do not meet these standards may be closed without warning to maintain the quality and integrity of the project. Please take the time to understand the changes you are proposing and their impact. AI slop is strongly discouraged and may lead to banning and blocking. Do not spam our repos with AI slop.

Check one of the checkboxes below:

• This PR does not contain AI-generated code at all.
• This PR contains AI-generated code. I have read the AI Usage Policy and this PR complies with this policy. I have tested the code locally and I am responsible for it.

I have used the following AI models and tools: Claude Sonnet - 5

Checklist

• My PR addresses a single issue, fixes a single bug or makes a single improvement.
• My code follows the project's code style and conventions
• If applicable, I have made corresponding changes or additions to the documentation
• If applicable, I have made corresponding changes or additions to tests
• My changes generate no new warnings or errors
• I have joined the Discord server and I will share a link to this PR with the project maintainers there
• I have read the Contribution Guidelines
• Once I submit my PR, CodeRabbit AI will automatically review it and I will address CodeRabbit's comments.
• I have filled this PR template completely and carefully, and I understand that my PR may be closed without review otherwise.

Summary by CodeRabbit

New Features

• Added configurable parallel processing for folder indexing, with a supported range of 1–8 workers.
• Added automatic AI-tagging sweeps after successful folder indexing.

Performance

• Folder indexing now runs independently from other background work, improving workload isolation and responsiveness.

Bug Fixes

• Ensured post-index tagging starts only after indexing completes successfully.

Tests

• Added coverage for separate indexing and deferred tagging behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9a60cc7f-fcff-45c8-b38e-edcc233ac447

📥 Commits

Reviewing files that changed from the base of the PR and between df248b6 and 50a950e.

📒 Files selected for processing (2)

• backend/app/config/settings.py
• backend/main.py

🚧 Files skipped from review as they are similar to previous changes (2)

• backend/main.py
• backend/app/config/settings.py

---

Walkthrough

The application runs folder indexing in a dedicated, configurable process pool. After successful indexing, it queues the AI-tagging sweep on the general executor. Tests verify executor separation and callback timing.

Changes

Folder indexing execution

Layer / File(s)	Summary
Indexing executor lifecycle backend/app/config/settings.py, backend/main.py, backend/app/routes/dependencies.py	Adds the INDEXING_MAX_WORKERS setting. Creates and shuts down a dedicated indexing executor. Updates shared executor documentation.
Folder indexing and tagging flow backend/app/routes/folders.py	Submits folder indexing to indexing_executor. Queues the AI-tagging sweep after successful indexing. Updates type syntax and error formatting.
Asynchronous indexing validation backend/tests/test_folders.py	Verifies executor separation and confirms that tagging starts only after indexing completes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant FolderRoute
  participant indexing_executor
  participant tagging_executor
  participant post_AI_tagging_enabled_sequence
  FolderRoute->>indexing_executor: Submit folder indexing
  indexing_executor-->>FolderRoute: Complete indexing future
  FolderRoute->>tagging_executor: Queue tagging sweep
  tagging_executor->>post_AI_tagging_enabled_sequence: Run tagging sequence

Loading

Possibly related PRs

• AOSSIE-Org/PictoPy#1380: Both changes coordinate folder indexing completion with AI-tagging behavior.

Suggested labels: Python

Poem

I’m a rabbit watching futures run,
Indexing workers hop in parallel fun.
When indexing completes,
Tagging follows its beat.
Folder queues now move as one. 🐇

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: folder indexing now runs in parallel instead of in series.
Linked Issues check	✅ Passed	The changes add a dedicated configurable indexing process pool and verify parallel folder indexing, fulfilling issue #1436.
Out of Scope Changes check	✅ Passed	The configuration, executor lifecycle, route updates, and tests directly support parallel indexing and follow-up tagging behavior.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/app/routes/folders.py`:
- Around line 289-291: Update the done callback attached to index_future in
post_folder_add_sequence to inspect the completed future’s result and queue
_queue_post_index_tagging_sweep(app_state) only when it returns True. Handle
exceptions, cancellation, and False results by logging the failure and returning
without scheduling the tagging sweep, and add coverage for a failed-index
result.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 1236a559-e8e5-417f-93a3-1f9f451a286a

📥 Commits

Reviewing files that changed from the base of the PR and between 5b4c41d and 62b0d59.

📒 Files selected for processing (5)

• backend/app/config/settings.py
• backend/app/routes/dependencies.py
• backend/app/routes/folders.py
• backend/main.py
• backend/tests/test_folders.py

[Takitxt]

@rohan-pandeyy can you please review this PR.

[github-actions]

⚠️ This PR has merge conflicts.

Please resolve the merge conflicts before review.

Your PR will only be reviewed by a maintainer after all conflicts have been resolved.

📺 Watch this video to understand why conflicts occur and how to resolve them:
https://www.youtube.com/watch?v=Sqsz1-o7nXk

[Takitxt]

@rohan-pandeyy can you please review this pr. 🙃