fix(GH-2284): Put docker:dind in proper cgroup parent#4480
Conversation
There was a problem hiding this comment.
Pull request overview
This PR aims to address GH-2284 by configuring the Docker-in-Docker (docker:dind) daemon to place nested Docker containers under the runner Pod’s cgroup via dockerd --cgroup-parent, improving cgroupv2 resource accounting/limits behavior.
Changes:
- Add a
--cgroup-parent=...flag to the dinddockerdargs. - Inject the Pod UID into the dind container via the Downward API (
metadata.uid) for use in the cgroup parent path.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - --cgroup-parent=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod${POD_UID//-/_}.slice | ||
| env: | ||
| - name: DOCKER_GROUP_GID | ||
| value: "123" | ||
| - name: POD_UID | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.uid |
| - dockerd | ||
| - --host=unix:///var/run/docker.sock | ||
| - --group=$(DOCKER_GROUP_GID) | ||
| - --cgroup-parent=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod${POD_UID//-/_}.slice |
| - dockerd | ||
| - --host=unix:///var/run/docker.sock | ||
| - --group=$(DOCKER_GROUP_GID) | ||
| - --cgroup-parent=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod${POD_UID//-/_}.slice |
|
This patch works, but the containers started via the dind docker process are only able to read the docker sidecar container cgroupv2 slice (or pod level resource if you run on Kubernetes 1.34+ and set pod memory limits) if This allows the dind created nested cotainer to see the full cgroup tree of the host and find its parent cgroup (the pod or container cgroup). Using |
|
After deeper investigation this will not work properly. There are 2 reasons:
I don't see much option rather than github arc going the same direction: to have own dind image with modified dockerd-entrypoint and dind. After modifications, all OOM issues on nodes ended in our company. Previously it was "WTF" all the time. |
Hello,
This PR is fix for ticket openeded since 2023: #2284 .
The idea is to use dockerd cgroup-parent flag to put dind containers in proper control groups.