From 8534be3ed74d123670e9a16748e2ec9eb61d499d Mon Sep 17 00:00:00 2001 From: Cody Maffucci <46459665+Maffooch@users.noreply.github.com> Date: Fri, 30 Oct 2026 23:15:34 -0600 Subject: [PATCH 1/3] docs(onprem): document the web-request statement_timeout and its settings Co-Authored-By: Claude Opus 5.5 --- docs/content/get_started/pro/onprem/hardware_sizing.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/content/get_started/pro/onprem/hardware_sizing.md b/docs/content/get_started/pro/onprem/hardware_sizing.md index 730892672f2..8ea44b34762 100644 --- a/docs/content/get_started/pro/onprem/hardware_sizing.md +++ b/docs/content/get_started/pro/onprem/hardware_sizing.md @@ -102,6 +102,16 @@ These four decide how many database connections the application holds open, so t Change either one with `dojo-compose-cli environment add` and restart, as above. A harakiri restart frees the process but does not remove the cause. If one shows up in the log, send the stack traces that come with it to support, since they show where the process was stuck. +**Slow database statements.** A harakiri restart stops the web process, but not the database query it was running: PostgreSQL keeps working on that query until it finishes. From 3.5.0, PostgreSQL itself stops any single statement of a web request that runs 30 seconds longer than that service's harakiri allows (870 seconds on `dojo`, 1740 seconds on `dojo-import-scan`), and the request answers with a 503 that says it took too long. Imports and exports get 1740 seconds on either service. Background tasks, migrations and management commands are never limited. + +| Setting | Service | Default | Notes | +| --- | --- | --- | --- | +| `DD_DATABASE_WEB_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under `DD_UWSGI_HARAKIRI` | Seconds; `0` turns the limit off. | +| `DD_DATABASE_IMPORT_STATEMENT_TIMEOUT` | `dojo-import-scan` (imports) | 30 seconds under `DD_UWSGI_IMPORT_HARAKIRI` | Seconds; `0` turns the limit off. | +| `DD_DATABASE_WEB_LONG_STATEMENT_TIMEOUT` | `dojo` (web) | 1740 | The limit for imports and exports served by the web service. `0` removes it for them. | + +On Kubernetes, the limit follows each uWSGI pool's harakiri from the chart values; set `DD_DATABASE_WEB_STATEMENT_TIMEOUT` through the chart's extra environment to change it. If a request answers with this 503, the log names the request: send it to support, since a statement that runs this long is a performance problem to fix. + **Import memory and shutdown.** uWSGI also replaces a process whose memory grows too large. `DD_UWSGI_RELOAD_ON_RSS` (MB) replaces it after a request leaves it above the limit: it stops taking requests, and the ones still running on its other threads get `DD_UWSGI_WORKER_RELOAD_MERCY` seconds before uWSGI kills it, failing them with a 502 (the log says `NO MERCY`). `DD_UWSGI_EVIL_RELOAD_ON_RSS` (MB) is a safety limit: a process that passes it is killed within about a second, and every request it is serving fails with a 502. The log says `EVIL RELOAD`, and Docker shows no restart. Imports need far more memory than pages, so from 3.4.300 the import service can be set on its own: | Setting | Service | Default | Notes | From a3db692d6cb7ce4cedb05c1bd688eefe9acb1874 Mon Sep 17 00:00:00 2001 From: Cody Maffucci <46459665+Maffooch@users.noreply.github.com> Date: Sat, 31 Oct 2026 05:35:01 -0600 Subject: [PATCH 2/3] docs(onprem): statement timeout on Kubernetes, poolers and the capped long bound Co-Authored-By: Claude Opus 5.5 --- .../get_started/pro/onprem/hardware_sizing.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/content/get_started/pro/onprem/hardware_sizing.md b/docs/content/get_started/pro/onprem/hardware_sizing.md index 8ea44b34762..755ea75208c 100644 --- a/docs/content/get_started/pro/onprem/hardware_sizing.md +++ b/docs/content/get_started/pro/onprem/hardware_sizing.md @@ -102,15 +102,17 @@ These four decide how many database connections the application holds open, so t Change either one with `dojo-compose-cli environment add` and restart, as above. A harakiri restart frees the process but does not remove the cause. If one shows up in the log, send the stack traces that come with it to support, since they show where the process was stuck. -**Slow database statements.** A harakiri restart stops the web process, but not the database query it was running: PostgreSQL keeps working on that query until it finishes. From 3.5.0, PostgreSQL itself stops any single statement of a web request that runs 30 seconds longer than that service's harakiri allows (870 seconds on `dojo`, 1740 seconds on `dojo-import-scan`), and the request answers with a 503 that says it took too long. Imports and exports get 1740 seconds on either service. Background tasks, migrations and management commands are never limited. +**Slow database statements.** A harakiri restart stops the web process, but not the database query it was running: PostgreSQL keeps working on that query until it finishes. From 3.5.0, PostgreSQL itself stops any single statement of a web request that runs past 30 seconds under that service's harakiri (870 seconds on `dojo`, 1740 seconds on `dojo-import-scan`), and the request answers with a 503 that says it took too long. Where harakiri is unset or `0`, the limit is 1740 seconds. Imports and exports may get a higher limit than other requests, but never more than 30 seconds under their own service's harakiri, because harakiri ends the request at that point anyway: with the defaults they get the same limit as everything else on that service. Background tasks, migrations and management commands are never limited. + +The limit is sent to PostgreSQL as a connection startup option. If DefectDojo reaches PostgreSQL through PgBouncer or another connection pooler, the pooler may refuse those connections or drop the option; set the limits below to `0` there. | Setting | Service | Default | Notes | | --- | --- | --- | --- | -| `DD_DATABASE_WEB_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under `DD_UWSGI_HARAKIRI` | Seconds; `0` turns the limit off. | -| `DD_DATABASE_IMPORT_STATEMENT_TIMEOUT` | `dojo-import-scan` (imports) | 30 seconds under `DD_UWSGI_IMPORT_HARAKIRI` | Seconds; `0` turns the limit off. | -| `DD_DATABASE_WEB_LONG_STATEMENT_TIMEOUT` | `dojo` (web) | 1740 | The limit for imports and exports served by the web service. `0` removes it for them. | +| `DD_DATABASE_WEB_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under `DD_UWSGI_HARAKIRI`, or 1740 when that is unset or `0` | Seconds; `0` turns the limit off. | +| `DD_DATABASE_IMPORT_STATEMENT_TIMEOUT` | `dojo-import-scan` (imports) | 30 seconds under `DD_UWSGI_IMPORT_HARAKIRI`, or 1740 when that is unset or `0` | Docker Compose only. Seconds; `0` turns the limit off. | +| `DD_DATABASE_WEB_LONG_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under the service's harakiri | The limit for imports and exports. A value above the default is capped at it; `0` means the default. | -On Kubernetes, the limit follows each uWSGI pool's harakiri from the chart values; set `DD_DATABASE_WEB_STATEMENT_TIMEOUT` through the chart's extra environment to change it. If a request answers with this 503, the log names the request: send it to support, since a statement that runs this long is a performance problem to fix. +On Kubernetes, the limit follows each uWSGI pool's harakiri from the chart values: 870 seconds on the web pool and 1740 seconds on the import pool, or 1740 seconds on the web pool when `django.uwsgiImport.enabled` is `false`. To change it, set `DD_DATABASE_WEB_STATEMENT_TIMEOUT` in `config.extra` or `django.extraEnv`. The chart has no separate setting for the import pool, so a value set there applies to the web and import pools alike. If a request answers with this 503, the log names the request: send it to support, since a statement that runs this long is a performance problem to fix. **Import memory and shutdown.** uWSGI also replaces a process whose memory grows too large. `DD_UWSGI_RELOAD_ON_RSS` (MB) replaces it after a request leaves it above the limit: it stops taking requests, and the ones still running on its other threads get `DD_UWSGI_WORKER_RELOAD_MERCY` seconds before uWSGI kills it, failing them with a 502 (the log says `NO MERCY`). `DD_UWSGI_EVIL_RELOAD_ON_RSS` (MB) is a safety limit: a process that passes it is killed within about a second, and every request it is serving fails with a 502. The log says `EVIL RELOAD`, and Docker shows no restart. Imports need far more memory than pages, so from 3.4.300 the import service can be set on its own: From e196797c5bd7458c4601ac70d4707ddad83e167b Mon Sep 17 00:00:00 2001 From: Cody Maffucci <46459665+Maffooch@users.noreply.github.com> Date: Sat, 31 Oct 2026 12:58:01 -0600 Subject: [PATCH 3/3] docs(onprem): long statement timeout default when harakiri is unset Co-Authored-By: Claude Opus 5.5 --- docs/content/get_started/pro/onprem/hardware_sizing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/get_started/pro/onprem/hardware_sizing.md b/docs/content/get_started/pro/onprem/hardware_sizing.md index 755ea75208c..348cd40d13c 100644 --- a/docs/content/get_started/pro/onprem/hardware_sizing.md +++ b/docs/content/get_started/pro/onprem/hardware_sizing.md @@ -110,7 +110,7 @@ The limit is sent to PostgreSQL as a connection startup option. If DefectDojo re | --- | --- | --- | --- | | `DD_DATABASE_WEB_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under `DD_UWSGI_HARAKIRI`, or 1740 when that is unset or `0` | Seconds; `0` turns the limit off. | | `DD_DATABASE_IMPORT_STATEMENT_TIMEOUT` | `dojo-import-scan` (imports) | 30 seconds under `DD_UWSGI_IMPORT_HARAKIRI`, or 1740 when that is unset or `0` | Docker Compose only. Seconds; `0` turns the limit off. | -| `DD_DATABASE_WEB_LONG_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under the service's harakiri | The limit for imports and exports. A value above the default is capped at it; `0` means the default. | +| `DD_DATABASE_WEB_LONG_STATEMENT_TIMEOUT` | `dojo` (web) | 30 seconds under the service's harakiri, or 1740 when that is unset or `0` | The limit for imports and exports. A value above the default is capped at it; `0` means the default. | On Kubernetes, the limit follows each uWSGI pool's harakiri from the chart values: 870 seconds on the web pool and 1740 seconds on the import pool, or 1740 seconds on the web pool when `django.uwsgiImport.enabled` is `false`. To change it, set `DD_DATABASE_WEB_STATEMENT_TIMEOUT` in `config.extra` or `django.extraEnv`. The chart has no separate setting for the import pool, so a value set there applies to the web and import pools alike. If a request answers with this 503, the log names the request: send it to support, since a statement that runs this long is a performance problem to fix.