Skip to content

feat: Google Drive HITL tools, team management UI, Daytona sandboxes, indexing pipeline hardening & test infrastructure#840

Merged
MODSetter merged 131 commits intomainfrom
dev
Feb 27, 2026
Merged

feat: Google Drive HITL tools, team management UI, Daytona sandboxes, indexing pipeline hardening & test infrastructure#840
MODSetter merged 131 commits intomainfrom
dev

Conversation

@MODSetter
Copy link
Owner

@MODSetter MODSetter commented Feb 27, 2026

  • Google Drive HITL tools – Added create/delete file agent tools with UI components, OAuth reauth flow for insufficient permissions, and KB deletion support (@CREDO23)
  • Team roles & UI polish – Team roles management with pagination/skeletons, chat rename dialog, mobile sidebar improvements, updated icons/styling, select-none across interactive elements, and file-upload tooltip enhancements (@AnishSarkar22)
  • Code sandbox integration – Integrated Daytona sandboxes (replacing microsandbox) with file handling, local persistence, and enhanced code-execution system prompt (@MODSetter)
  • Indexing pipeline hardening – Structured error handling & logging, per-document error isolation, IntegrityError recovery, persistence helpers extraction, file-upload adapter, and fallback document summary (@CREDO23)
  • Test infrastructure – Bootstrapped pytest environment, added unit tests for document hashing/indexing, end-to-end upload & searchability tests, task dispatcher, dynamic embedding dimension config, and testing docs (@CREDO23, @AnishSarkar22)
  • Misc – Performance logging & caching improvements, dependency updates, linting & formatter cleanup (@MODSetter, @AnishSarkar22)

Description

Motivation and Context

FIX #

Screenshots

API Changes

  • This PR includes API changes

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactoring
  • Documentation
  • Dependency/Build system
  • Breaking change
  • Other (specify):

Testing Performed

  • Tested locally
  • Manual/QA verification

Checklist

  • Follows project coding standards and conventions
  • Documentation updated as needed
  • Dependencies updated as needed
  • No lint/build errors or new warnings
  • All relevant tests are passing

High-level PR Summary

This PR introduces five major feature areas: Google Drive HITL tools for creating and deleting files with OAuth reauth flow and UI components; Daytona sandbox integration replacing microsandbox to provide isolated code execution with local file persistence; indexing pipeline hardening with structured error handling, per-document error isolation, IntegrityError recovery, and opt-in LLM-based summary generation via enable_summary flag; comprehensive pytest test infrastructure with unit and integration tests for document hashing, indexing, upload workflows, and page-limit enforcement; and team management UI improvements including role management with pagination, chat rename dialogs, mobile sidebar enhancements, and updated icons/styling. Additional changes include performance logging, MCP tool caching, dependency updates, and linting cleanup.

⏱️ Estimated Review Time: 3+ hours

💡 Review Order Suggestion
Order File Path
1 surfsense_backend/.env.example
2 surfsense_backend/pyproject.toml
3 Dockerfile.allinone
4 docker-compose.yml
5 scripts/docker/entrypoint-allinone.sh
6 surfsense_backend/alembic/versions/102_add_enable_summary_to_connectors.py
7 surfsense_backend/app/db.py
8 surfsense_backend/app/schemas/search_source_connector.py
9 surfsense_web/contracts/types/connector.types.ts
10 surfsense_backend/app/indexing_pipeline/connector_document.py
11 surfsense_backend/app/indexing_pipeline/document_hashing.py
12 surfsense_backend/app/indexing_pipeline/exceptions.py
13 surfsense_backend/app/indexing_pipeline/document_summarizer.py
14 surfsense_backend/app/indexing_pipeline/document_chunker.py
15 surfsense_backend/app/indexing_pipeline/document_embedder.py
16 surfsense_backend/app/indexing_pipeline/document_persistence.py
17 surfsense_backend/app/indexing_pipeline/pipeline_logger.py
18 surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
19 surfsense_backend/app/indexing_pipeline/adapters/file_upload_adapter.py
20 surfsense_backend/app/utils/document_converters.py
21 surfsense_backend/app/tasks/connector_indexers/google_drive_indexer.py
22 surfsense_backend/app/tasks/connector_indexers/airtable_indexer.py
23 surfsense_backend/app/tasks/connector_indexers/bookstack_indexer.py
24 surfsense_backend/app/tasks/connector_indexers/clickup_indexer.py
25 surfsense_backend/app/tasks/connector_indexers/confluence_indexer.py
26 surfsense_backend/app/tasks/connector_indexers/github_indexer.py
27 surfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.py
28 surfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.py
29 surfsense_backend/app/tasks/connector_indexers/jira_indexer.py
30 surfsense_backend/app/tasks/connector_indexers/linear_indexer.py
31 surfsense_backend/app/tasks/connector_indexers/luma_indexer.py
32 surfsense_backend/app/tasks/connector_indexers/notion_indexer.py
33 surfsense_backend/app/tasks/connector_indexers/obsidian_indexer.py
34 surfsense_backend/app/tasks/connector_indexers/slack_indexer.py
35 surfsense_backend/app/tasks/connector_indexers/teams_indexer.py
36 surfsense_backend/app/tasks/connector_indexers/webcrawler_indexer.py
37 surfsense_backend/app/tasks/document_processors/file_processors.py
38 surfsense_backend/app/tasks/surfsense_docs_indexer.py
39 surfsense_backend/app/connectors/composio_gmail_connector.py
40 surfsense_backend/app/connectors/composio_google_calendar_connector.py
41 surfsense_backend/app/connectors/composio_google_drive_connector.py
42 surfsense_backend/app/services/notion/kb_sync_service.py
43 surfsense_backend/app/services/linear/kb_sync_service.py
44 surfsense_backend/app/connectors/google_drive/client.py
45 surfsense_backend/app/services/google_drive/tool_metadata_service.py
46 surfsense_backend/app/agents/new_chat/tools/google_drive/create_file.py
47 surfsense_backend/app/agents/new_chat/tools/google_drive/trash_file.py
48 surfsense_backend/app/routes/google_drive_add_connector_route.py
49 surfsense_web/components/tool-ui/google-drive/create-file.tsx
50 surfsense_web/components/tool-ui/google-drive/trash-file.tsx
51 surfsense_backend/app/agents/new_chat/sandbox.py
52 surfsense_backend/app/routes/sandbox_routes.py
53 surfsense_backend/app/agents/new_chat/system_prompt.py
54 surfsense_backend/app/agents/new_chat/chat_deepagent.py
55 surfsense_backend/app/tasks/chat/stream_new_chat.py
56 surfsense_web/components/tool-ui/sandbox-execute.tsx
57 surfsense_backend/app/agents/new_chat/tools/mcp_tool.py
58 surfsense_backend/app/agents/new_chat/tools/registry.py
59 surfsense_backend/app/routes/search_source_connectors_routes.py
60 surfsense_web/components/assistant-ui/connector-popup/components/summary-config.tsx
61 surfsense_web/components/assistant-ui/connector-popup/connector-configs/views/connector-edit-view.tsx
62 surfsense_web/components/assistant-ui/connector-popup/connector-configs/views/indexing-configuration-view.tsx
63 surfsense_web/components/assistant-ui/connector-popup/hooks/use-connector-dialog.ts
64 surfsense_backend/app/services/task_dispatcher.py
65 surfsense_backend/app/routes/documents_routes.py
66 surfsense_web/components/assistant-ui/document-upload-popup.tsx
67 surfsense_web/components/assistant-ui/thread.tsx
68 surfsense_web/components/sources/DocumentUploadTab.tsx
69 surfsense_backend/app/agents/new_chat/tools/search_surfsense_docs.py
70 surfsense_backend/app/agents/new_chat/tools/shared_memory.py
71 surfsense_backend/app/agents/new_chat/tools/user_memory.py
72 surfsense_backend/app/services/llm_router_service.py
73 surfsense_backend/app/services/connector_service.py
74 surfsense_backend/app/app.py
75 surfsense_backend/main.py
76 surfsense_backend/app/routes/new_chat_routes.py
77 surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx
78 .cursor/skills/tdd/SKILL.md
79 .cursor/skills/tdd/deep-modules.md
80 .cursor/skills/tdd/interface-design.md
81 .cursor/skills/tdd/mocking.md
82 .cursor/skills/tdd/refactoring.md
83 .cursor/skills/tdd/tests.md
84 surfsense_backend/tests/conftest.py
85 surfsense_backend/tests/unit/indexing_pipeline/conftest.py
86 surfsense_backend/tests/unit/indexing_pipeline/test_connector_document.py
87 surfsense_backend/tests/unit/indexing_pipeline/test_document_hashing.py
88 surfsense_backend/tests/unit/indexing_pipeline/test_document_chunker.py
89 surfsense_backend/tests/unit/indexing_pipeline/test_document_summarizer.py
90 surfsense_backend/tests/integration/conftest.py
91 surfsense_backend/tests/integration/indexing_pipeline/test_prepare_for_indexing.py
92 surfsense_backend/tests/integration/indexing_pipeline/test_index_document.py
93 surfsense_backend/tests/integration/indexing_pipeline/adapters/test_file_upload_adapter.py
94 surfsense_backend/tests/integration/document_upload/conftest.py
95 surfsense_backend/tests/integration/document_upload/test_document_upload.py
96 surfsense_backend/tests/integration/document_upload/test_page_limits.py
97 surfsense_backend/tests/integration/document_upload/test_upload_limits.py
98 surfsense_backend/tests/utils/helpers.py
99 surfsense_web/components/settings/roles-manager.tsx
100 surfsense_web/app/dashboard/[search_space_id]/team/page.tsx
101 surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx
102 surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx
103 surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsTableShell.tsx
104 surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/PaginationControls.tsx
105 surfsense_web/components/assistant-ui/connector-popup.tsx
106 surfsense_web/components/layout/ui/sidebar/AllPrivateChatsSidebar.tsx
107 surfsense_web/components/layout/ui/sidebar/AllSharedChatsSidebar.tsx
108 surfsense_web/components/dashboard-breadcrumb.tsx
109 surfsense_web/components/layout/ui/sidebar/ChatListItem.tsx
110 surfsense_web/components/layout/ui/sidebar/MobileSidebar.tsx
111 surfsense_web/components/layout/ui/sidebar/Sidebar.tsx
112 surfsense_web/components/layout/ui/sidebar/SidebarHeader.tsx
113 surfsense_web/components/layout/ui/sidebar/SidebarSlideOutPanel.tsx
114 surfsense_web/components/layout/ui/sidebar/SidebarUserProfile.tsx
115 surfsense_web/components/new-chat/model-selector.tsx
116 surfsense_web/app/dashboard/[search_space_id]/onboard/page.tsx
117 surfsense_web/app/dashboard/[search_space_id]/more-pages/page.tsx
118 surfsense_web/components/layout/providers/LayoutDataProvider.tsx
119 surfsense_web/components/new-chat/chat-share-button.tsx
120 surfsense_web/app/docs/layout.tsx
121 surfsense_web/app/docs/sidebar-separator.tsx
122 surfsense_web/app/layout.config.tsx
123 surfsense_web/content/docs/index.mdx
124 surfsense_web/content/docs/installation.mdx
125 surfsense_web/content/docs/docker-installation.mdx
126 surfsense_web/content/docs/manual-installation.mdx
127 surfsense_web/content/docs/connectors/meta.json
128 surfsense_web/content/docs/how-to/meta.json
129 surfsense_web/content/docs/meta.json
130 surfsense_web/content/docs/testing.mdx
131 surfsense_web/lib/source.ts
⚠️ Inconsistent Changes Detected
File Path Warning
.cursor/skills/tdd/SKILL.md TDD skill documentation appears unrelated to the main PR features (Google Drive, Daytona, indexing, team management)
.cursor/skills/tdd/deep-modules.md Software design philosophy documentation is unrelated to the PR's feature additions
.cursor/skills/tdd/interface-design.md Interface design guidelines documentation is unrelated to the PR's feature additions
.cursor/skills/tdd/mocking.md Mocking guidelines documentation is unrelated to the PR's feature additions
.cursor/skills/tdd/refactoring.md Refactoring guidelines documentation is unrelated to the PR's feature additions
.cursor/skills/tdd/tests.md Testing philosophy documentation is unrelated to the PR's feature additions
surfsense_web/components/icons/providers/xai.svg Icon update appears unrelated to the main PR features
surfsense_web/lib/provider-icons.tsx Icon color change appears unrelated to the main PR features

Need help? Join our Discord

Analyze latest changes

CREDO23 and others added 30 commits February 20, 2026 16:05
…n behavior in team management and breadcrumb components
… and enhancing UI components for better user experience
…ct-none class for improved text selection behavior
…d components for improved user information display
… improving layout for shared and private chats
AnishSarkar22 and others added 23 commits February 26, 2026 02:22
- Introduced a new test class to verify that uploaded documents appear in search results once their status is ready.
- Implemented assertions to ensure the uploaded document's ID is present in the search response.
- Introduced a TaskDispatcher abstraction to decouple the upload endpoint from Celery, allowing for easier testing with synchronous implementations.
- Updated the create_documents_file_upload function to utilize the new dispatcher for task management.
- Removed direct Celery task imports from the upload function, enhancing modularity.
- Added integration tests for document upload, including page limit enforcement and file size restrictions.
- Removed commented-out testing configuration from .env.example to streamline the file.
- Updated markers in pyproject.toml to remove the e2e test marker, clarifying the purpose of the remaining markers.
…ation

- Updated the embedding dimension in test configurations to use the value from the application config, enhancing maintainability and consistency across tests.
- Introduced slow callback logging in FastAPI to identify blocking calls.
- Added performance logging for agent creation and tool loading processes.
- Implemented caching for MCP tools to reduce redundant server calls.
- Enhanced sandbox management with in-process caching for improved efficiency.
- Refactored several functions for better readability and performance tracking.
- Updated tests to ensure proper functionality of new features and optimizations.
… layout

- Introduced SidebarSeparator component for improved sidebar organization.
- Updated layout configuration to utilize the new SidebarSeparator in the documentation layout.
fix: enhanced document upload, page limit, upload limit tests
- Replaced direct embedding calls with a utility function across various components to streamline embedding logic.
- Added enable_summary flag to several models and routes to control summary generation behavior.
@vercel
Copy link

vercel bot commented Feb 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
surf-sense-frontend Ready Ready Preview, Comment Feb 27, 2026 4:48am

Request Review

Copy link

@recurseml recurseml bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by RecurseML

🔍 Review performed on af3810b..6f4bf11

  Severity     Location     Issue     Delete  
High surfsense_backend/app/routes/documents_routes.py:305 Missing parameter causes TypeError
✅ Files analyzed, no issues (49)

.cursor/skills/tdd/SKILL.md
.cursor/skills/tdd/deep-modules.md
.cursor/skills/tdd/interface-design.md
.cursor/skills/tdd/mocking.md
.cursor/skills/tdd/refactoring.md
.cursor/skills/tdd/tests.md
Dockerfile.allinone
docker-compose.yml
scripts/docker/entrypoint-allinone.sh
surfsense_backend/.env.example
surfsense_backend/.gitignore
surfsense_backend/alembic/versions/102_add_enable_summary_to_connectors.py
surfsense_backend/app/agents/new_chat/chat_deepagent.py
surfsense_backend/app/agents/new_chat/sandbox.py
surfsense_backend/app/agents/new_chat/system_prompt.py
surfsense_backend/app/agents/new_chat/tools/google_drive/__init__.py
surfsense_backend/app/agents/new_chat/tools/google_drive/create_file.py
surfsense_backend/app/agents/new_chat/tools/google_drive/trash_file.py
surfsense_backend/app/agents/new_chat/tools/mcp_tool.py
surfsense_backend/app/agents/new_chat/tools/registry.py
surfsense_backend/app/agents/new_chat/tools/search_surfsense_docs.py
surfsense_backend/app/agents/new_chat/tools/shared_memory.py
surfsense_backend/app/agents/new_chat/tools/user_memory.py
surfsense_backend/app/app.py
surfsense_backend/app/connectors/composio_gmail_connector.py
surfsense_backend/app/connectors/composio_google_calendar_connector.py
surfsense_backend/app/connectors/composio_google_drive_connector.py
surfsense_backend/app/connectors/google_drive/client.py
surfsense_backend/app/db.py
surfsense_backend/app/indexing_pipeline/adapters/file_upload_adapter.py
surfsense_backend/app/indexing_pipeline/connector_document.py
surfsense_backend/app/indexing_pipeline/document_chunker.py
surfsense_backend/app/indexing_pipeline/document_embedder.py
surfsense_backend/app/indexing_pipeline/document_hashing.py
surfsense_backend/app/indexing_pipeline/document_persistence.py
surfsense_backend/app/indexing_pipeline/document_summarizer.py
surfsense_backend/app/indexing_pipeline/exceptions.py
surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
surfsense_backend/app/indexing_pipeline/pipeline_logger.py
surfsense_backend/app/routes/__init__.py
surfsense_backend/app/routes/google_drive_add_connector_route.py
surfsense_backend/app/routes/new_chat_routes.py
surfsense_backend/app/routes/reports_routes.py
surfsense_backend/app/routes/sandbox_routes.py
surfsense_backend/app/routes/search_source_connectors_routes.py
surfsense_backend/app/schemas/documents.py
surfsense_backend/app/schemas/podcasts.py
surfsense_backend/app/schemas/search_source_connector.py
surfsense_backend/app/services/connector_service.py

⏭️ Files skipped (132)
  Locations  
surfsense_backend/app/indexing_pipeline/__init__.py
surfsense_backend/app/indexing_pipeline/adapters/__init__.py
surfsense_backend/app/services/google_drive/__init__.py
surfsense_backend/app/services/google_drive/tool_metadata_service.py
surfsense_backend/app/services/linear/kb_sync_service.py
surfsense_backend/app/services/llm_router_service.py
surfsense_backend/app/services/notion/kb_sync_service.py
surfsense_backend/app/services/task_dispatcher.py
surfsense_backend/app/tasks/celery_tasks/document_tasks.py
surfsense_backend/app/tasks/chat/stream_new_chat.py
surfsense_backend/app/tasks/connector_indexers/airtable_indexer.py
surfsense_backend/app/tasks/connector_indexers/bookstack_indexer.py
surfsense_backend/app/tasks/connector_indexers/clickup_indexer.py
surfsense_backend/app/tasks/connector_indexers/confluence_indexer.py
surfsense_backend/app/tasks/connector_indexers/discord_indexer.py
surfsense_backend/app/tasks/connector_indexers/github_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_drive_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.py
surfsense_backend/app/tasks/connector_indexers/jira_indexer.py
surfsense_backend/app/tasks/connector_indexers/linear_indexer.py
surfsense_backend/app/tasks/connector_indexers/luma_indexer.py
surfsense_backend/app/tasks/connector_indexers/notion_indexer.py
surfsense_backend/app/tasks/connector_indexers/obsidian_indexer.py
surfsense_backend/app/tasks/connector_indexers/slack_indexer.py
surfsense_backend/app/tasks/connector_indexers/teams_indexer.py
surfsense_backend/app/tasks/connector_indexers/webcrawler_indexer.py
surfsense_backend/app/tasks/document_processors/file_processors.py
surfsense_backend/app/tasks/surfsense_docs_indexer.py
surfsense_backend/app/utils/document_converters.py
surfsense_backend/main.py
surfsense_backend/pyproject.toml
surfsense_backend/tests/__init__.py
surfsense_backend/tests/conftest.py
surfsense_backend/tests/fixtures/empty.pdf
surfsense_backend/tests/fixtures/sample.md
surfsense_backend/tests/fixtures/sample.txt
surfsense_backend/tests/integration/__init__.py
surfsense_backend/tests/integration/conftest.py
surfsense_backend/tests/integration/document_upload/__init__.py
surfsense_backend/tests/integration/document_upload/conftest.py
surfsense_backend/tests/integration/document_upload/test_document_upload.py
surfsense_backend/tests/integration/document_upload/test_page_limits.py
surfsense_backend/tests/integration/document_upload/test_upload_limits.py
surfsense_backend/tests/integration/indexing_pipeline/__init__.py
surfsense_backend/tests/integration/indexing_pipeline/adapters/__init__.py
surfsense_backend/tests/integration/indexing_pipeline/adapters/test_file_upload_adapter.py
surfsense_backend/tests/integration/indexing_pipeline/test_index_document.py
surfsense_backend/tests/integration/indexing_pipeline/test_prepare_for_indexing.py
surfsense_backend/tests/unit/__init__.py
surfsense_backend/tests/unit/adapters/__init__.py
surfsense_backend/tests/unit/indexing_pipeline/__init__.py
surfsense_backend/tests/unit/indexing_pipeline/conftest.py
surfsense_backend/tests/unit/indexing_pipeline/test_connector_document.py
surfsense_backend/tests/unit/indexing_pipeline/test_document_chunker.py
surfsense_backend/tests/unit/indexing_pipeline/test_document_hashing.py
surfsense_backend/tests/unit/indexing_pipeline/test_document_summarizer.py
surfsense_backend/tests/utils/__init__.py
surfsense_backend/tests/utils/helpers.py
surfsense_backend/uv.lock
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentTypeIcon.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsTableShell.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/PaginationControls.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/types.ts
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/page.tsx
surfsense_web/app/dashboard/[search_space_id]/logs/(manage)/page.tsx
surfsense_web/app/dashboard/[search_space_id]/more-pages/page.tsx
surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx
surfsense_web/app/dashboard/[search_space_id]/onboard/page.tsx
surfsense_web/app/dashboard/[search_space_id]/settings/page.tsx
surfsense_web/app/dashboard/[search_space_id]/team/page.tsx
surfsense_web/app/docs/layout.tsx
surfsense_web/app/docs/sidebar-separator.tsx
surfsense_web/app/globals.css
surfsense_web/app/layout.config.tsx
surfsense_web/components/assistant-ui/connector-popup.tsx
surfsense_web/components/assistant-ui/connector-popup/components/summary-config.tsx
surfsense_web/components/assistant-ui/connector-popup/connector-configs/views/connector-edit-view.tsx
surfsense_web/components/assistant-ui/connector-popup/connector-configs/views/indexing-configuration-view.tsx
surfsense_web/components/assistant-ui/connector-popup/hooks/use-connector-dialog.ts
surfsense_web/components/assistant-ui/document-upload-popup.tsx
surfsense_web/components/assistant-ui/thread.tsx
surfsense_web/components/assistant-ui/tooltip-icon-button.tsx
surfsense_web/components/dashboard-breadcrumb.tsx
surfsense_web/components/icons/providers/xai.svg
surfsense_web/components/layout/providers/LayoutDataProvider.tsx
surfsense_web/components/layout/ui/dialogs/CreateSearchSpaceDialog.tsx
surfsense_web/components/layout/ui/shell/LayoutShell.tsx
surfsense_web/components/layout/ui/sidebar/AllPrivateChatsSidebar.tsx
surfsense_web/components/layout/ui/sidebar/AllSharedChatsSidebar.tsx
surfsense_web/components/layout/ui/sidebar/ChatListItem.tsx
surfsense_web/components/layout/ui/sidebar/InboxSidebar.tsx
surfsense_web/components/layout/ui/sidebar/MobileSidebar.tsx
surfsense_web/components/layout/ui/sidebar/Sidebar.tsx
surfsense_web/components/layout/ui/sidebar/SidebarHeader.tsx
surfsense_web/components/layout/ui/sidebar/SidebarSlideOutPanel.tsx
surfsense_web/components/layout/ui/sidebar/SidebarUserProfile.tsx
surfsense_web/components/new-chat/chat-share-button.tsx
surfsense_web/components/new-chat/model-selector.tsx
surfsense_web/components/public-chat-snapshots/public-chat-snapshot-row.tsx
surfsense_web/components/report-panel/report-panel.tsx
surfsense_web/components/settings/roles-manager.tsx
surfsense_web/components/sources/DocumentUploadTab.tsx
surfsense_web/components/tool-ui/google-drive/create-file.tsx
surfsense_web/components/tool-ui/google-drive/index.ts
surfsense_web/components/tool-ui/google-drive/trash-file.tsx
surfsense_web/components/tool-ui/index.ts
surfsense_web/components/tool-ui/sandbox-execute.tsx
surfsense_web/components/ui/expanded-gif-overlay.tsx
surfsense_web/components/ui/hero-carousel.tsx
surfsense_web/content/docs/connectors/meta.json
surfsense_web/content/docs/docker-installation.mdx
surfsense_web/content/docs/how-to/meta.json
surfsense_web/content/docs/index.mdx
surfsense_web/content/docs/installation.mdx
surfsense_web/content/docs/manual-installation.mdx
surfsense_web/content/docs/meta.json
surfsense_web/content/docs/testing.mdx
surfsense_web/contracts/types/connector.types.ts
surfsense_web/contracts/types/document.types.ts
surfsense_web/hooks/use-documents.ts
surfsense_web/lib/apis/documents-api.service.ts
surfsense_web/lib/electric/client.ts
surfsense_web/lib/provider-icons.tsx
surfsense_web/lib/source.ts
surfsense_web/messages/en.json
surfsense_web/messages/es.json
surfsense_web/messages/hi.json
surfsense_web/messages/pt.json
surfsense_web/messages/zh.json
surfsense_web/pnpm-lock.yaml

filename=filename,
search_space_id=search_space_id,
user_id=str(user.id),
should_summarize=should_summarize,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runtime TypeError: dispatch_file_processing() got an unexpected keyword argument 'should_summarize'

The documents upload endpoint at line 305 passes should_summarize=should_summarize to dispatcher.dispatch_file_processing(), but the TaskDispatcher protocol and CeleryTaskDispatcher implementation in app/services/task_dispatcher.py do NOT include this parameter in their signatures (lines 13-21 and 27-46).

The protocol defines:

class TaskDispatcher(Protocol):
    async def dispatch_file_processing(
        self,
        *,
        document_id: int,
        temp_path: str,
        filename: str,
        search_space_id: int,
        user_id: str,
    ) -> None: ...  # NO should_summarize

And the implementation doesn't accept it either:

class CeleryTaskDispatcher:
    async def dispatch_file_processing(
        self,
        *,
        document_id: int,
        temp_path: str,
        filename: str,
        search_space_id: int,
        user_id: str,
    ) -> None:  # NO should_summarize

Even though the underlying Celery task process_file_upload_with_document_task accepts should_summarize, the dispatcher layer blocks it from being passed through.

This will crash on EVERY file upload when users try to upload documents with the new should_summarize parameter (line 122 adds it as a Form parameter).

Error that will occur:

TypeError: dispatch_file_processing() got an unexpected keyword argument 'should_summarize'

Fix needed: Add should_summarize: bool = False parameter to both the TaskDispatcher protocol and CeleryTaskDispatcher.dispatch_file_processing() method, and pass it through to the Celery task.


React with 👍 to tell me that this comment was useful, or 👎 if not (and I'll stop posting more comments like this in the future)

- Introduced should_summarize parameter in TaskDispatcher and CeleryTaskDispatcher to control summary generation.
- Updated InlineTaskDispatcher to support the new parameter for document processing.
- Introduced dynamic character budget calculation for document formatting based on model's context window.
- Updated `format_documents_for_context` to respect character limits and improve output quality.
- Added `max_input_tokens` parameter to various functions to facilitate context-aware processing.
- Enhanced error handling for context overflow in LLM router service.
@MODSetter MODSetter merged commit 3ca401c into main Feb 27, 2026
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants