From 55fb2669f1c1e7e276dd9283dad993496fadb67b Mon Sep 17 00:00:00 2001 From: Takwa Date: Tue, 3 Nov 2026 13:38:41 +0100 Subject: [PATCH 1/2] Send needle's traces to the box's collector Exports over OTLP/HTTP to alloy-traces on the host's loopback, and trusts the traceparent Envoy sends, so needle samples exactly the requests Envoy sampled and its spans join Envoy's traces. Needs needle v1.1.0. Co-Authored-By: Claude Opus 5.5 --- k8s/needle/kustomization.yaml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/k8s/needle/kustomization.yaml b/k8s/needle/kustomization.yaml index 693280d..e737f83 100644 --- a/k8s/needle/kustomization.yaml +++ b/k8s/needle/kustomization.yaml @@ -140,6 +140,21 @@ configMapGenerator: # which is the only reason the per-peer lock contention was ever visible. - NEEDLE_PPROF=true + # Traces (ipni/infra#163), to the trace collector on the box's loopback + # (ipni/infra kubernetes/platform/alloy-traces) over OTLP/HTTP. Under + # hostNetwork this is the HOST's loopback, which is where it listens. + # Setting the endpoint alone is enough for needle to export. + - OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318 + + # Join the trace Envoy starts and follow its sampling decision (5%, see + # tracing in k8s/route-origin/envoy.yaml.j2) rather than deciding again. + # Safe only because needle is reachable solely through that Envoy: the + # API is loopback-only, and Envoy drops a client's traceparent and + # tracestate before reading them, so a client cannot choose what needle + # samples. NEEDLE_SAMPLING_FRACTION stays at its default of 0, so a + # request without a traceparent, from a tool on the box, is not traced. + - NEEDLE_TRACING_TRUST_TRACEPARENT=true + # The libp2p resource manager defaults to 85% of HOST RAM and ignores the # cgroup limit, which would let it plan past the pod's 21Gi and get # OOM-killed. 16 GiB, in bytes: a quarter of the 64 GiB the single From 4afbefb23d55a4ea9761e319fbf9daf9ae5970d7 Mon Sep 17 00:00:00 2001 From: Takwa Date: Tue, 3 Nov 2026 13:39:00 +0100 Subject: [PATCH 2/2] Say in Envoy's config that needle joins its traces The comment still described v1.0.0. It is rendered into envoy.yaml, so even this changes the config hash and restarts Envoy on the next apply. Co-Authored-By: Claude Opus 5.5 --- k8s/route-origin/envoy.yaml.j2 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/k8s/route-origin/envoy.yaml.j2 b/k8s/route-origin/envoy.yaml.j2 index b0f7282..d8cc432 100644 --- a/k8s/route-origin/envoy.yaml.j2 +++ b/k8s/route-origin/envoy.yaml.j2 @@ -87,10 +87,10 @@ static_resources: # request and, with spawn_upstream_span, one for the call to # needle (which instance, how long). The span names are the # route decorators below, never the path: a path carries a CID, - # and every span name becomes a metric series. needle does not - # join these traces yet: v1.0.0 drops the traceparent Envoy - # sends (NEEDLE_TRACING_AUTH is unset), and its own sampling - # never fires (ipni/infra#163). + # and every span name becomes a metric series. needle joins + # these traces: it trusts the traceparent Envoy sends + # (NEEDLE_TRACING_TRUST_TRACEPARENT in k8s/needle), so it samples + # exactly these requests and adds its own spans below them. # # Only this sampling decides. A client cannot ask to be traced: # client_sampling ignores x-client-trace-id, and the early