Skip to content

update the timeout for running embeddings#2049

Merged
shixiao-coder merged 6 commits into
datacommonsorg:masterfrom
shixiao-coder:update-embedding-timeout
May 29, 2026
Merged

update the timeout for running embeddings#2049
shixiao-coder merged 6 commits into
datacommonsorg:masterfrom
shixiao-coder:update-embedding-timeout

Conversation

@shixiao-coder
Copy link
Copy Markdown
Contributor

No description provided.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a configurable timeout parameter for Spanner queries in the ingestion helper. It updates the get_updated_nodes and generate_embeddings_partitioned functions to accept a timeout parameter, defaulting to 300 seconds if not provided. In main.py, a new command-line flag timeout is defined with a default of 3600 seconds and passed to these functions. Feedback suggests reducing the default timeout flag value from 3600 to 1800 seconds to align with Google Cloud Workflows' maximum HTTP request timeout limit.

Comment thread import-automation/workflow/ingestion-helper/main.py Outdated
@shixiao-coder shixiao-coder requested a review from gmechali May 29, 2026 14:12
Comment thread import-automation/workflow/ingestion-helper/embedding_utils.py Outdated
Comment thread import-automation/workflow/ingestion-helper/embedding_utils.py Outdated
Comment thread import-automation/workflow/ingestion-helper/main.py Outdated
Comment thread import-automation/workflow/ingestion-helper/embedding_utils.py Outdated
@shixiao-coder shixiao-coder merged commit 2b23535 into datacommonsorg:master May 29, 2026
13 checks passed
@shixiao-coder shixiao-coder deleted the update-embedding-timeout branch May 29, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants