Skip to content

fix(server): add --workers flag to prevent single blocking request stalling HTTP server#465

Closed
r266-tech wants to merge 1 commit intovolcengine:mainfrom
r266-tech:fix/uvicorn-workers
Closed

fix(server): add --workers flag to prevent single blocking request stalling HTTP server#465
r266-tech wants to merge 1 commit intovolcengine:mainfrom
r266-tech:fix/uvicorn-workers

Conversation

@r266-tech
Copy link
Contributor

Summary

Fixes #464

OpenViking started uvicorn with no workers argument (defaulting to 1), which means one slow or blocking request can stall the entire HTTP server process — including lightweight probes like /health.

This PR adds a workers configuration knob so operators can run multiple uvicorn worker processes and prevent this availability issue.

Changes

openviking/server/config.py

  • Add workers: int = 1 field to ServerConfig
  • Load server.workers from ov.conf
  • Add validation in validate_server_config: reject workers < 1 with a clear error and sys.exit(1)

openviking/server/bootstrap.py

  • Add --workers N CLI argument
  • When workers == 1 (default): existing behaviour unchanged, app object passed directly to uvicorn.run()
  • When workers > 1: use uvicorn.run('openviking.server.app:create_app', factory=True, workers=N) so each worker process independently imports and initialises its own OpenVikingService. The config file path is propagated via the OPENVIKING_CONFIG_FILE env var already set in main()

tests/server/test_server_workers_config.py (new)

  • ServerConfig defaults to workers=1
  • Custom workers value stored correctly
  • validate_server_config rejects workers=0 and workers=-1
  • validate_server_config accepts workers=1 and workers=4
  • load_server_config reads workers from config file
  • load_server_config defaults workers to 1 when absent

Usage

# CLI flag
openviking-server --workers 4

# ov.conf
{
  "server": {
    "workers": 4
  }
}

Limitations

  • --with-bot is not supported with --workers > 1 (warning emitted, bot proxy disabled). Bot mode requires a separate single-worker instance.
  • Multiple workers each hold independent in-memory state. For local SQLite backends, concurrent writes may cause contention; a networked backend is recommended for production multi-worker deployments.

Backward Compatibility

Default is workers=1, so all existing deployments are unaffected.

…alling HTTP server

Fixes #464

## Problem

OpenViking started uvicorn with a single worker (no `workers=` argument),
meaning one slow or blocking request could stall the entire server process,
including lightweight endpoints such as /health.

## Solution

Add a `workers` configuration knob with three touch-points:

1. `ServerConfig.workers` (default 1) - new field in the server
   dataclass, loaded from ov.conf under `server.workers`.
2. `--workers N` CLI flag - lets operators set the worker count at
   startup without editing ov.conf.
3. `validate_server_config` - rejects workers < 1 with a clear
   error message and a non-zero exit code.

### Multi-worker launch strategy

When workers > 1, the app is passed to uvicorn.run() as an import
string with factory=True so each forked worker process independently
imports and initialises its own OpenVikingService instance. The
config file path is propagated via the OPENVIKING_CONFIG_FILE
environment variable that is already set earlier in main().

Single-worker mode (workers=1, the default) is fully backward-
compatible and continues to support the vikingbot gateway subprocess.

### Limitations / known constraints

* --with-bot is not supported together with --workers > 1; a
  warning is emitted and the bot proxy is skipped. Bot mode requires
  a separate single-worker server instance.
* Multiple workers each hold their own in-memory state. If OpenViking
  is configured with a local SQLite backend, concurrent writes from
  multiple workers may cause contention; using a networked storage
  backend is recommended for production multi-worker deployments.

## Tests

Added tests/server/test_server_workers_config.py covering:
- ServerConfig defaults to workers=1
- Custom workers value is stored correctly
- validate_server_config rejects workers=0 and workers=-1
- validate_server_config accepts workers=1 and workers=4
- load_server_config reads workers from the config file
- load_server_config defaults workers to 1 when absent
@MaojiaSheng
Copy link
Collaborator

你测过能 work 吗?我也 vibe 过类似的版本,但有些问题

@r266-tech
Copy link
Contributor Author

Superseded by #470 with cleaner implementation approach.

@r266-tech r266-tech closed this Mar 6, 2026
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: OpenViking runs with a single uvicorn worker, allowing one blocking request to stall the entire HTTP server

2 participants