Summary
On load-balancing Serverless endpoints, two documented settings don't behave as described in Load balancing endpoints:
HEALTH_CHECK_PATH is ignored. The load balancer always polls GET /ping. The docs say: "By default the load balancer polls /ping, but you can point it at any path by setting the HEALTH_CHECK_PATH environment variable", and they give llama.cpp's /health as the motivating example.
PORT_HEALTH doesn't default to PORT. With PORT=8000 set and PORT_HEALTH unset, the saved template gets PORT_HEALTH=80 injected. The env var table says PORT_HEALTH defaults to "Same as PORT".
Together, these mean the documented use case (stock ghcr.io/ggml-org/llama.cpp:server-cuda, health on /health) never becomes healthy. The worker loads the model and listens, but it's never routed to, and it's killed and restarted after a few minutes.
Reproduction (2026-09-23)
Endpoints created with the GraphQL saveEndpoint mutation, type: "LB", scalerType: "REQUEST_COUNT", workersMin: 0, workersMax: 1, with an inline template. The console "Health check endpoint" field was not used.
1. HEALTH_CHECK_PATH ignored. A request-logging worker:
imageName: "python:3.12-slim"
dockerArgs: "python -u -m http.server 80"
env: [{key: "HEALTH_CHECK_PATH", value: "/health"}]
The platform then injected PORT=80 and PORT_HEALTH=80. The worker log (runpodctl serverless logs <id>) shows only /ping probes, never /health:
100.64.0.76 - - [23/Sep/2026 08:28:53] "GET /ping HTTP/1.1" 404 -
100.64.0.76 - - [23/Sep/2026 08:28:54] "GET /ping HTTP/1.1" 404 -
100.64.0.76 - - [23/Sep/2026 08:28:55] "GET /ping HTTP/1.1" 404 -
...
2. PORT_HEALTH default. The template env was sent as LLAMA_CACHE=/models, PORT=8000, HEALTH_CHECK_PATH=/health. Reading it back with podTemplate(id) returned:
env: LLAMA_CACHE=/models, HEALTH_CHECK_PATH=/health, PORT=8000, PORT_HEALTH=80
ports: 80/http,8000/http
Expected
- The load balancer polls the path in
HEALTH_CHECK_PATH when it's set in the template env.
PORT_HEALTH defaults to the value of PORT.
If either setting is intentionally only honored through the console field or the v2 REST API, the docs should say so.
Impact
Any image whose server doesn't answer /ping (llama.cpp, many OpenAI-compatible servers) needs a custom wrapper image, which is exactly what the docs say HEALTH_CHECK_PATH avoids.
Summary
On load-balancing Serverless endpoints, two documented settings don't behave as described in Load balancing endpoints:
HEALTH_CHECK_PATHis ignored. The load balancer always pollsGET /ping. The docs say: "By default the load balancer polls/ping, but you can point it at any path by setting theHEALTH_CHECK_PATHenvironment variable", and they givellama.cpp's/healthas the motivating example.PORT_HEALTHdoesn't default toPORT. WithPORT=8000set andPORT_HEALTHunset, the saved template getsPORT_HEALTH=80injected. The env var table saysPORT_HEALTHdefaults to "Same asPORT".Together, these mean the documented use case (stock
ghcr.io/ggml-org/llama.cpp:server-cuda, health on/health) never becomes healthy. The worker loads the model and listens, but it's never routed to, and it's killed and restarted after a few minutes.Reproduction (2026-09-23)
Endpoints created with the GraphQL
saveEndpointmutation,type: "LB",scalerType: "REQUEST_COUNT",workersMin: 0,workersMax: 1, with an inline template. The console "Health check endpoint" field was not used.1.
HEALTH_CHECK_PATHignored. A request-logging worker:imageName: "python:3.12-slim"dockerArgs: "python -u -m http.server 80"env: [{key: "HEALTH_CHECK_PATH", value: "/health"}]The platform then injected
PORT=80andPORT_HEALTH=80. The worker log (runpodctl serverless logs <id>) shows only/pingprobes, never/health:2.
PORT_HEALTHdefault. The template env was sent asLLAMA_CACHE=/models,PORT=8000,HEALTH_CHECK_PATH=/health. Reading it back withpodTemplate(id)returned:Expected
HEALTH_CHECK_PATHwhen it's set in the template env.PORT_HEALTHdefaults to the value ofPORT.If either setting is intentionally only honored through the console field or the v2 REST API, the docs should say so.
Impact
Any image whose server doesn't answer
/ping(llama.cpp, many OpenAI-compatible servers) needs a custom wrapper image, which is exactly what the docs sayHEALTH_CHECK_PATHavoids.