Skip to content

Route actor hosts by optional "<port>-" prefix#296

Open
rogeroger (rogeroger-yu) wants to merge 2 commits into
agent-substrate:mainfrom
rogeroger-yu:port-prefixed-actor-routing
Open

Route actor hosts by optional "<port>-" prefix#296
rogeroger (rogeroger-yu) wants to merge 2 commits into
agent-substrate:mainfrom
rogeroger-yu:port-prefixed-actor-routing

Conversation

@rogeroger-yu

Copy link
Copy Markdown

What

Encode an optional target service port in the left-most DNS label of an actor host, so a single actor can expose multiple in-actor services without dynamic Envoy listeners or per-actor reverse proxies.

<actor-id>.<suffix>            -> <worker-ip>:80   (unchanged default)
<port>-<actor-id>.<suffix>     -> <worker-ip>:<port>

Examples:

my-actor.actors.resources.substrate.ate.dev        -> my-actor on port 80
8080-my-actor.actors.resources.substrate.ate.dev   -> my-actor on port 8080
49983-sbx1.actors.resources.substrate.ate.dev      -> sbx1 on port 49983

This is the design proposed in #265.

Why this shape

  • No dynamic listener creation in Envoy or atenet.
  • Works with the existing wildcard DNS shape: the host still ends in actors.resources.substrate.ate.dev.
  • Backwards compatible: hosts without a numeric prefix keep routing to port 80.
  • Enables generic multi-service actors (command/filesystem API, code execution, VNC, user HTTP servers, debugging endpoints, ...) rather than one specific SDK/compat layer.

Changes

Two layers had to change for this to work end to end:

  1. atenet routing (cmd/atenet/internal/router): parseActorID now returns (actorID, port). A left-most label matching ^([0-9]+)-(.+)$ is parsed as <port>-<actorID>; the port is validated to be in 1..65535 and otherwise fails routing with a 404 rather than silently falling back. handleRequestHeaders forwards to <workerIP>:<port> instead of a hard-coded :80.

  2. ateom-gvisor actor DNAT (cmd/ateom-gvisor/main.go): the actor prerouting DNAT previously matched only TCP/80 and rewrote both address and port to the actor veth on 80, which made port-prefixed routing a no-op end to end. It now matches all inbound TCP to the worker pod IP and DNATs only the destination address to the actor veth, preserving the original port. (ateom serves its own RPCs over a unix socket, not a pod-IP TCP port, so this catch-all does not shadow the control path.)

Ambiguity

Actor IDs that themselves start with <digits>- are interpreted as port-prefixed. DNS-1123 permits such labels, so this is a documented, deterministic rule: tools that mint actor IDs for multi-port access must avoid IDs beginning with <digits>-. IDs that start with digits but have no <digits>- prefix (e.g. 123abc) keep routing to the default port.

Tests

  • Unit tests in extproc_in_test.go cover default-port routing, prefixed-port routing, min/max ports, port 0, out-of-range ports, leading-zero normalization, and the existing host/connection-port parsing.
  • Manually verified end to end on a local kind cluster with a counter actor listening on both 80 and 8080:
    • my-actor.<suffix> -> 200, served from port 80
    • 8080-my-actor.<suffix> -> 200, served from port 8080 (distinct response body)
    • 0-my-actor.<suffix> / 99999-my-actor.<suffix> -> 404 (invalid port)
    • atenet logs confirm targetAddr of <ip>:8080 vs <ip>:80.

Notes

  • UDP DNAT for actors exposing QUIC/UDP is left as a follow-up (existing TODO retained).

@google-cla

google-cla Bot commented Jun 24, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Encode an optional target service port in the left-most DNS label of an
actor host, so a single actor can expose multiple in-actor services
without dynamic Envoy listeners or per-actor reverse proxies:

    <actor-id>.<suffix>            -> <worker-ip>:80   (unchanged default)
    <port>-<actor-id>.<suffix>     -> <worker-ip>:<port>

parseActorID now returns the parsed port alongside the actor ID. A
left-most label matching ^([0-9]+)-(.+)$ is treated as port-prefixed;
the port is validated to be in 1..65535 and otherwise fails routing
rather than silently falling back. Hosts without a numeric prefix keep
routing to port 80, so existing actors are unaffected.

The deterministic rule means actor IDs minted for multi-port access must
not themselves begin with "<digits>-"; this is the one documented
ambiguity, since DNS-1123 permits such labels.

Refs agent-substrate#265
The actor prerouting DNAT previously matched only TCP/80 and rewrote both
the destination address and port to the actor veth on port 80. That made
port-prefixed actor routing a no-op end to end: atenet would forward to
<podIP>:<port> for a "<port>-<actor>" host, but any port other than 80
was dropped before reaching the actor.

Match all inbound TCP destined for the worker pod IP and DNAT only the
destination address to the actor veth, leaving the port unchanged
(RegProtoMin unset). The actor decides which ports it listens on. ateom
serves its own control RPCs over a unix socket, not a pod-IP TCP port,
so this catch-all does not shadow the control path.

This is the actor-network half of port-prefixed routing; the routing
half lives in atenet's parseActorID.

Refs agent-substrate#265
@rogeroger-yu rogeroger (rogeroger-yu) force-pushed the port-prefixed-actor-routing branch from 21b6202 to 22fd06c Compare June 24, 2026 04:12
@bowei Bowei Du (bowei) self-assigned this Jun 25, 2026
@bowei

Copy link
Copy Markdown
Collaborator

Would be good to resolve the conversations on the linked issue before we get too far along in the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants