From ae481b04a923d16c741d07736877ff77ef78b3b7 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 14:26:17 -0500 Subject: [PATCH 01/22] docs: propose private DNS names for Compute instances --- docs/enhancements/internal-dns/README.md | 327 +++++++++++++++++++++++ 1 file changed, 327 insertions(+) create mode 100644 docs/enhancements/internal-dns/README.md diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md new file mode 100644 index 00000000..7b160816 --- /dev/null +++ b/docs/enhancements/internal-dns/README.md @@ -0,0 +1,327 @@ +--- +status: provisional +stage: alpha +latest-milestone: "TBD" +--- + +# Internal DNS for Compute instances + +Tracking issue: [Internal DNS for Galactic VPC](https://github.com/datum-cloud/enhancements/issues/921). + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User stories](#user-stories) + - [Risks and mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Integration boundaries](#integration-boundaries) + - [Resolver configuration](#resolver-configuration) + - [Record publication](#record-publication) + - [Lifecycle and health](#lifecycle-and-health) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed](#infrastructure-needed) + +## Summary + +Compute instances attached to a virtual private cloud (VPC) receive private DNS +names and resolver settings automatically. Compute publishes each instance's +private addresses to the internal DNS platform. Users can reach instances by +name without selecting a DNS zone or managing address records. + +## Motivation + +Instance addresses change as workloads scale, restart, or move. Automatic names +let users identify individual instances without maintaining address lists. +Resolution must remain isolated to the authorized VPC, including when VPCs use +overlapping addresses or private zone names. + +### Goals + +- Assign an instance identity name within each attached VPC's managed namespace. +- Configure guest resolvers from verified network-interface settings. +- Publish IPv4 and IPv6 host addresses only while their network attachment is valid. +- Bound stale answers when instances disappear or publishers lose connectivity. + +### Non-Goals + +- Workload-level service discovery and application health policies. +- Public DNS publication, custom per-instance zone selection, and DNS serving engines. +- Supporting several managed DNS contexts within one guest in the initial release. + +## Proposal + +For a VPC with managed DNS enabled, Compute reserves an instance name in the +namespace allocated by DNS. The name remains stable for that instance's +lifetime. A replacement instance receives a distinct identity. Additional names +can follow the VPC's DNS naming policy without adding a zone choice to each +Compute resource. + +Workload providers apply the resolver settings supplied by network services. +DNS publication and guest resolver configuration have separate status from +Compute readiness so users can distinguish naming problems from runtime failures. + +### User stories + +A user attaches a workload to a VPC and queries an instance's assigned name +from another instance in that VPC: + +```console +$ dig +short A web-01.instances.vpc-a7c9.project-p4e2.internal +10.20.0.10 +$ dig +short AAAA web-01.instances.vpc-a7c9.project-p4e2.internal +fd20::10 +``` + +Names, suffixes, and addresses are illustrative. DNS supplies the canonical name; +consumers do not construct it from a workload display name. + +When an address is released, subsequent answers omit it after withdrawal and +client-cache expiry. An application failing its readiness check does not remove +the instance identity name, which can still support diagnostics. + +### Risks and mitigations + +- Cross-VPC publication: pin project, VPC, context, and registration lifetimes; + authorize publishers through scoped grants. +- Stale addresses: require fresh network eligibility observations and preserve + their deadlines through replication, retry, and restart. +- Incorrect guest resolution: apply only verified, unexpired settings and reject + unsupported combinations of managed contexts across interfaces. + +## Design Details + +### Integration boundaries + +```mermaid +flowchart LR + N[Network services] -->|Context and access| P[Consumer project API] + C[Compute publisher] -->|Registration, addresses, eligibility| P + I[Trusted grant issuer] -->|Scoped contribution grant| P + P -->|DNS intent| D[Shared DNS platform] + P -->|Workload and network desired state| K[Karmada] + K --> E[Edge interfaces and workload providers] + E -->|Verified instance and interface state| C + E -->|Resolver settings| G[Guest] + G -->|VPC-authorized private service path| D +``` + +Compute uses authenticated project clients for publication. It identifies the +VPC's context through trusted project and Network UIDs, matching the +`DNSResolverContext.spec.consumerID` contract. DNS does not read Compute objects +to determine addresses or health. + +Network services own resolver contexts, regional access, and interface DNS +settings. Galactic provides the authorized private service path. DNS owns zones, +name validation, publication status, and shared query serving. Compute does not +create access bindings or select serving replicas. + +See the [DNS architecture proposal](https://github.com/datum-cloud/dns-operator/pull/229), +[VPC DNS integration proposal](https://github.com/datum-cloud/network-services-operator/pull/589), +and [Private Service Connect design](https://github.com/datum-cloud/galactic/blob/main/docs/enhancements/networking/private-service-connect/README.md) +for these contracts. + +### Resolver configuration + +Providers consume `NetworkInterface.spec.dns` at the edge. This selected-field +example shows generated settings, not a manifest users submit: + +```yaml +apiVersion: networking.datumapis.com/v1alpha +kind: NetworkInterface +metadata: + name: web-01-eth0 +spec: + network: + name: application + interfaceName: eth0 + # Network services own inherited resolver settings. + dns: + # Pin the source DNS context and access lifetimes. + contextRef: + name: application + uid: 33333333-3333-4333-8333-333333333333 + accessBindingRef: + name: application-us-central-1 + uid: 44444444-4444-4444-8444-444444444444 + # Reject stale settings and preserve the original authorization deadline. + writerEpoch: 3 + sequence: 27 + validUntil: "2026-10-09T20:05:00Z" + # Galactic exposes this well-known address inside the authorized VPC. + nameservers: ["fd53::53"] + # DNS allocates the managed suffix. + searches: [vpc-a7c9.project-p4e2.internal] +``` + +Providers apply current settings to the guest and report `DNSConfigured` only +after successful application. They reconcile updates and expiry, retain source +identity across federation, and reject stale revisions. Resolver failure must +not send private names to a public resolver. The guest configuration mechanism +requires qualification for each supported runtime. + +### Record publication + +Compute creates one registration per instance and VPC, using the managed zone +reported by the matching DNS context. Allocation ties the name to the immutable +Instance UID; name reuse cannot adopt another instance's registration. + +A trusted grant issuer authorizes the Compute publisher's authenticated subject +and source-cluster identity for the registration, record types, and owner name. +The runtime publisher cannot issue its own `DNSContributionGrant` or authorize +resolver access. Project authorization and DNS admission enforce both boundaries. + +These examples use the proposed DNS API. UIDs, generations, and times are +illustrative. Controllers write the resources; `status` shows controller output. + +```yaml +apiVersion: dns.networking.miloapis.com/v1alpha1 +kind: DNSRegistration +metadata: + name: instance-web-01 +spec: + # Select the context's DNS-allocated managed zone and pin its lifetime. + dnsZoneRef: + name: managed-application + uid: 55555555-5555-4555-8555-555555555555 + # Reserve the allocated instance identity name relative to the zone. + name: web-01.instances + recordTypes: [A, AAAA] + # Publish addresses only under a current eligible contribution. + publicationPolicy: EligibleContributions + ttlSeconds: 30 +status: + # DNS-owned output used to display the assigned name to consumers. + canonicalFQDN: web-01.instances.vpc-a7c9.project-p4e2.internal +--- +apiVersion: dns.networking.miloapis.com/v1alpha1 +kind: DNSRecordContribution +metadata: + name: web-01-eth0 +spec: + # Pin the reserved name and its publication policy generation. + registrationRef: + name: instance-web-01 + uid: 66666666-6666-4666-8666-666666666666 + generation: 1 + # Reference authorization issued separately from the runtime publisher. + grantRef: + name: web-01-compute + uid: 77777777-7777-4777-8777-777777777777 + generation: 1 + # Include only private host addresses allocated and programmed for this VPC. + recordSets: + - recordType: A + records: + - name: web-01.instances + ttl: 30 + a: + content: 10.20.0.10 + - recordType: AAAA + records: + - name: web-01.instances + ttl: 30 + aaaa: + content: fd20::10 +status: + # Compute-owned observation of this contribution's current spec. + observedGeneration: 1 + # Echo the DNS-issued grant epoch and advance the observation sequence. + writerEpoch: 3 + sequence: 18 + eligible: true + reason: NetworkInterfaceProgrammed + # DNS preserves this deadline and withdraws expired contributions locally. + validUntil: "2026-10-09T20:01:00Z" +``` + +Compute renews observations from the authoritative project Instance and verified +interface state. Federation and edge copies retain source lifetime identity; +their locally assigned UIDs cannot establish publication authority. Status writes +preserve DNS-owned fields. Retries cannot extend an observation's deadline. + +### Lifecycle and health + +Instance identity eligibility requires a live Instance and allocated, programmed +private host addresses. Public addresses and delegated prefixes are excluded. +Each interface contributes only addresses from its own VPC. + +Address release, network deprogramming, or instance deletion triggers a +higher-sequence ineligible observation and cleanup. Project-side owner references +and bounded leases cover interrupted cleanup. Serving agents enforce expiry +during publisher outages; client caches remain bounded by the advertised TTL. + +Workload service discovery needs a separate registration and explicit application +eligibility policy. It must withdraw unhealthy endpoints and accept recovery +only from a fresh observation. That capability requires a separate Compute API +proposal before implementation. + +## Production Readiness Review Questionnaire + +### Feature Enablement and Rollback + +Use the default-off `InternalDNSPublishing` feature gate for the publisher. +Provider application of managed resolver settings requires a separate rollout +control. Disabling publication stops renewals; records expire, and applications +using their names can lose resolution. Reenabling requires fresh observations. +Test both controls independently. + +### Rollout, Upgrade and Rollback Planning + +Start with opted-in projects and supported single-context guests. Extend the +shared Kubernetes end-to-end environment with actual Compute publishers and +providers. Require two isolated VPCs with overlapping names and addresses, UDP +and TCP queries from guests, renewal stability, address changes, deletion, +publisher outages, grant revocation, stale replay, and restart coverage. +Qualify guest configuration per runtime and mixed controller versions before +enabling production. These are release criteria, not validation claims. + +### Monitoring Requirements + +Expose assigned names and DNS publication conditions through Instance status; +the exact Compute status schema requires API review. Track publication lag, +renewal failures, withdrawal delay, and guest configuration failures without +per-instance metric labels. Set latency and withdrawal targets before release. + +### Dependencies + +Project APIs and scoped authorization support publication. Network services, +Galactic, and the DNS fleet support resolver access. API outages stop fresh +observations; installed records and access remain subject to their original +deadlines. DNS errors requeue independently of Compute runtime readiness. + +### Scalability + +Budget registrations per instance/VPC and contributions per interface. Measure +project API renewal load, controller queues, and grant provisioning at fleet +scale. Spread renewals with jitter and bound concurrent project operations. + +### Troubleshooting + +Trace the Instance and VPC lifetimes through context, registration, grant, and +contribution status. Check current network eligibility and deadlines, then +compare inherited settings with guest configuration and query results. + +## Implementation History + +2026-10-09: Compute integration enhancement proposed. + +## Drawbacks + +Publication adds project API traffic and freshness coordination. DNS outages can +interrupt applications that depend on assigned names despite a healthy runtime. + +## Alternatives + +User-managed records require consumers to track instance address changes. +Automatic publication keeps that lifecycle with Compute. + +## Infrastructure Needed + +Provision project-scoped publisher credentials, a separate trusted grant issuer, +VPC resolver access, shared DNS capacity, and the local Kubernetes validation +environment. This integration uses shared serving infrastructure across VPCs. From 3e693221993e01d75b85adc3d132fd70658baa74 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 14:54:39 -0500 Subject: [PATCH 02/22] docs: separate public DNS settings from provider authorization --- docs/enhancements/internal-dns/README.md | 44 ++++++++++++++---------- 1 file changed, 25 insertions(+), 19 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 7b160816..4ba9c609 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -105,6 +105,7 @@ flowchart LR P -->|DNS intent| D[Shared DNS platform] P -->|Workload and network desired state| K[Karmada] K --> E[Edge interfaces and workload providers] + N -->|Internal resolver delivery contract| E E -->|Verified instance and interface state| C E -->|Resolver settings| G[Guest] G -->|VPC-authorized private service path| D @@ -127,8 +128,10 @@ for these contracts. ### Resolver configuration -Providers consume `NetworkInterface.spec.dns` at the edge. This selected-field -example shows generated settings, not a manifest users submit: +`NetworkInterface` is a public API. Its proposed read-only DNS status exposes +resolver addresses, search domains, and configuration readiness. Customers +select the network; they do not supply DNS authorization or renewal values. +This selected-field example includes proposed controller-written status: ```yaml apiVersion: networking.datumapis.com/v1alpha @@ -139,30 +142,33 @@ spec: network: name: application interfaceName: eth0 - # Network services own inherited resolver settings. +status: + # Report effective settings without exposing internal authorization state. dns: - # Pin the source DNS context and access lifetimes. - contextRef: - name: application - uid: 33333333-3333-4333-8333-333333333333 - accessBindingRef: - name: application-us-central-1 - uid: 44444444-4444-4444-8444-444444444444 - # Reject stale settings and preserve the original authorization deadline. - writerEpoch: 3 - sequence: 27 - validUntil: "2026-10-09T20:05:00Z" # Galactic exposes this well-known address inside the authorized VPC. nameservers: ["fd53::53"] # DNS allocates the managed suffix. searches: [vpc-a7c9.project-p4e2.internal] + conditions: + # The provider reports success after applying authorized settings to the guest. + - type: DNSConfigured + status: "True" + reason: ResolverConfigured + lastTransitionTime: "2026-10-09T20:00:00Z" ``` -Providers apply current settings to the guest and report `DNSConfigured` only -after successful application. They reconcile updates and expiry, retain source -identity across federation, and reject stale revisions. Resolver failure must -not send private names to a public resolver. The guest configuration mechanism -requires qualification for each supported runtime. +Network services deliver authorization through a separate, access-restricted +provider contract bound to the project, interface, and VPC lifetimes. That +contract carries source references, fencing epochs, renewal sequences, and +expiry deadlines. These fields do not belong in the public interface's spec or +status. Public DNS status reports observed configuration; it cannot authorize +provider actions. + +Providers apply current, authorized settings and report `DNSConfigured` only +after successful application. They reconcile updates and expiry and reject +stale revisions. Resolver failure must not send private names to a public +resolver. The internal delivery schema and runtime-specific guest configuration +mechanism require joint review with network services before implementation. ### Record publication From 69a9908ccc3724c4111bc43e3b6649755d969c03 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 15:26:19 -0500 Subject: [PATCH 03/22] docs: define namespace-aware instance names and search domains --- docs/enhancements/internal-dns/README.md | 51 +++++++++++++++++++----- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 4ba9c609..7feff31d 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -17,6 +17,7 @@ Tracking issue: [Internal DNS for Galactic VPC](https://github.com/datum-cloud/e - [Risks and mitigations](#risks-and-mitigations) - [Design Details](#design-details) - [Integration boundaries](#integration-boundaries) + - [Names and search domains](#names-and-search-domains) - [Resolver configuration](#resolver-configuration) - [Record publication](#record-publication) - [Lifecycle and health](#lifecycle-and-health) @@ -43,6 +44,7 @@ overlapping addresses or private zone names. ### Goals - Assign an instance identity name within each attached VPC's managed namespace. +- Resolve short instance names within the resource's namespace. - Configure guest resolvers from verified network-interface settings. - Publish IPv4 and IPv6 host addresses only while their network attachment is valid. - Bound stale answers when instances disappear or publishers lose connectivity. @@ -71,14 +73,21 @@ A user attaches a workload to a VPC and queries an instance's assigned name from another instance in that VPC: ```console -$ dig +short A web-01.instances.vpc-a7c9.project-p4e2.internal +$ cat /etc/resolv.conf +nameserver fd53::53 +search production.vpc-a7c9.project-p4e2.internal vpc-a7c9.project-p4e2.internal +$ dig +search +short A web-01-k7m2 10.20.0.10 -$ dig +short AAAA web-01.instances.vpc-a7c9.project-p4e2.internal +$ dig +short AAAA web-01-k7m2.production.vpc-a7c9.project-p4e2.internal. fd20::10 ``` Names, suffixes, and addresses are illustrative. DNS supplies the canonical name; consumers do not construct it from a workload display name. +The example resolver file describes effective settings; runtimes can apply them +through their native resolver configuration. `dig` requires +[`+search`](https://bind9.readthedocs.io/en/stable/manpages.html#dig-dns-lookup-utility) +to use the configured search list. When an address is released, subsequent answers omit it after withdrawal and client-cache expiry. An application failing its readiness check does not remove @@ -126,6 +135,25 @@ See the [DNS architecture proposal](https://github.com/datum-cloud/dns-operator/ and [Private Service Connect design](https://github.com/datum-cloud/galactic/blob/main/docs/enhancements/networking/private-service-connect/README.md) for these contracts. +### Names and search domains + +Use `..` for +instance identity. The resource namespace is the customer-visible project +namespace, not a mapped federation or edge namespace. + +For a guest in `production`, try `production.` before +``. This resolves short instance names locally and supports +VPC-scoped names without a product prefix. Use a fully qualified name to select +another namespace unambiguously. Namespace names organize resolution; VPC access +authorization provides the network isolation boundary. + +Network services supply the VPC suffix and approved additional search domains. +Compute providers prepend the guest's resource namespace and report the effective +ordered list. Associating a private zone does not automatically add it to the +search list. Bound list size and qualify resolver search order across runtimes. +The namespace-aware naming and search contract requires joint DNS and network +services API review. + ### Resolver configuration `NetworkInterface` is a public API. Its proposed read-only DNS status exposes @@ -138,6 +166,7 @@ apiVersion: networking.datumapis.com/v1alpha kind: NetworkInterface metadata: name: web-01-eth0 + namespace: production spec: network: name: application @@ -147,8 +176,10 @@ status: dns: # Galactic exposes this well-known address inside the authorized VPC. nameservers: ["fd53::53"] - # DNS allocates the managed suffix. - searches: [vpc-a7c9.project-p4e2.internal] + # Resolve resource-namespace names before VPC-scoped names. + searches: + - production.vpc-a7c9.project-p4e2.internal + - vpc-a7c9.project-p4e2.internal conditions: # The provider reports success after applying authorized settings to the guest. - type: DNSConfigured @@ -194,15 +225,15 @@ spec: dnsZoneRef: name: managed-application uid: 55555555-5555-4555-8555-555555555555 - # Reserve the allocated instance identity name relative to the zone. - name: web-01.instances + # Reserve the allocated instance name within its resource namespace. + name: web-01-k7m2.production recordTypes: [A, AAAA] # Publish addresses only under a current eligible contribution. publicationPolicy: EligibleContributions ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01.instances.vpc-a7c9.project-p4e2.internal + canonicalFQDN: web-01-k7m2.production.vpc-a7c9.project-p4e2.internal --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordContribution @@ -223,13 +254,13 @@ spec: recordSets: - recordType: A records: - - name: web-01.instances + - name: web-01-k7m2.production ttl: 30 a: content: 10.20.0.10 - recordType: AAAA records: - - name: web-01.instances + - name: web-01-k7m2.production ttl: 30 aaaa: content: fd20::10 @@ -283,6 +314,8 @@ shared Kubernetes end-to-end environment with actual Compute publishers and providers. Require two isolated VPCs with overlapping names and addresses, UDP and TCP queries from guests, renewal stability, address changes, deletion, publisher outages, grant revocation, stale replay, and restart coverage. +Test equal short names in different namespaces, search precedence, fully +qualified queries, and explicit additional-zone search configuration. Qualify guest configuration per runtime and mixed controller versions before enabling production. These are release criteria, not validation claims. From a8636affce01d84f6f74ef312a9af8c0283365f6 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 15:30:38 -0500 Subject: [PATCH 04/22] docs: use a consistent internal domain across VPCs --- docs/enhancements/internal-dns/README.md | 48 ++++++++++++++---------- 1 file changed, 29 insertions(+), 19 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 7feff31d..38ee9c14 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -57,9 +57,10 @@ overlapping addresses or private zone names. ## Proposal -For a VPC with managed DNS enabled, Compute reserves an instance name in the -namespace allocated by DNS. The name remains stable for that instance's -lifetime. A replacement instance receives a distinct identity. Additional names +Every VPC with managed DNS enabled uses `internal` as its default domain. DNS +manages a separate zone within each VPC's context. Compute reserves an instance +name beneath its resource namespace. The name remains stable for that instance's +lifetime; a replacement instance receives a distinct identity. Additional names can follow the VPC's DNS naming policy without adding a zone choice to each Compute resource. @@ -75,15 +76,16 @@ from another instance in that VPC: ```console $ cat /etc/resolv.conf nameserver fd53::53 -search production.vpc-a7c9.project-p4e2.internal vpc-a7c9.project-p4e2.internal +search production.internal internal $ dig +search +short A web-01-k7m2 10.20.0.10 -$ dig +short AAAA web-01-k7m2.production.vpc-a7c9.project-p4e2.internal. +$ dig +short AAAA web-01-k7m2.production.internal. fd20::10 ``` -Names, suffixes, and addresses are illustrative. DNS supplies the canonical name; -consumers do not construct it from a workload display name. +Instance names and addresses are illustrative; `internal` is the proposed shared +default domain. DNS supplies the canonical name; consumers do not construct it +from a workload display name. The example resolver file describes effective settings; runtimes can apply them through their native resolver configuration. `dig` requires [`+search`](https://bind9.readthedocs.io/en/stable/manpages.html#dig-dns-lookup-utility) @@ -137,22 +139,29 @@ for these contracts. ### Names and search domains -Use `..` for -instance identity. The resource namespace is the customer-visible project -namespace, not a mapped federation or edge namespace. +Use `..internal` for instance +identity in every VPC. The resource namespace is the customer-visible project +namespace, not a mapped federation or edge namespace. VPC and project identifiers +do not appear in the default domain. -For a guest in `production`, try `production.` before -``. This resolves short instance names locally and supports +For a guest in `production`, try `production.internal` before `internal`. +This resolves short instance names locally and supports VPC-scoped names without a product prefix. Use a fully qualified name to select another namespace unambiguously. Namespace names organize resolution; VPC access authorization provides the network isolation boundary. -Network services supply the VPC suffix and approved additional search domains. +Two VPCs can resolve `web-01-k7m2.production.internal` to different addresses. +The authorized DNS context selects the zone before lookup and cache access; +matching domain names do not share records or grant access across VPCs. An +instance attached to several VPCs uses the same allocated label in each context, +with only that VPC's addresses in its answers. + +Network services supply the managed domain and approved additional search domains. Compute providers prepend the guest's resource namespace and report the effective ordered list. Associating a private zone does not automatically add it to the search list. Bound list size and qualify resolver search order across runtimes. -The namespace-aware naming and search contract requires joint DNS and network -services API review. +The consistent default domain, namespace-aware naming, and search contract +require joint DNS and network services API review. ### Resolver configuration @@ -178,8 +187,8 @@ status: nameservers: ["fd53::53"] # Resolve resource-namespace names before VPC-scoped names. searches: - - production.vpc-a7c9.project-p4e2.internal - - vpc-a7c9.project-p4e2.internal + - production.internal + - internal conditions: # The provider reports success after applying authorized settings to the guest. - type: DNSConfigured @@ -233,7 +242,7 @@ spec: ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01-k7m2.production.vpc-a7c9.project-p4e2.internal + canonicalFQDN: web-01-k7m2.production.internal --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordContribution @@ -314,7 +323,8 @@ shared Kubernetes end-to-end environment with actual Compute publishers and providers. Require two isolated VPCs with overlapping names and addresses, UDP and TCP queries from guests, renewal stability, address changes, deletion, publisher outages, grant revocation, stale replay, and restart coverage. -Test equal short names in different namespaces, search precedence, fully +Test the same default domain and hostname with different answers across VPCs, +equal short names in different namespaces, search precedence, fully qualified queries, and explicit additional-zone search configuration. Qualify guest configuration per runtime and mixed controller versions before enabling production. These are release criteria, not validation claims. From a2b2c75dc5d2a78cb8537bb12dd8cea5a6d0525c Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 15:41:10 -0500 Subject: [PATCH 05/22] docs: use datum.internal as the default private domain --- docs/enhancements/internal-dns/README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 38ee9c14..e7c0642c 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -57,7 +57,7 @@ overlapping addresses or private zone names. ## Proposal -Every VPC with managed DNS enabled uses `internal` as its default domain. DNS +Every VPC with managed DNS enabled uses `datum.internal` as its default domain. DNS manages a separate zone within each VPC's context. Compute reserves an instance name beneath its resource namespace. The name remains stable for that instance's lifetime; a replacement instance receives a distinct identity. Additional names @@ -76,14 +76,14 @@ from another instance in that VPC: ```console $ cat /etc/resolv.conf nameserver fd53::53 -search production.internal internal +search production.datum.internal datum.internal $ dig +search +short A web-01-k7m2 10.20.0.10 -$ dig +short AAAA web-01-k7m2.production.internal. +$ dig +short AAAA web-01-k7m2.production.datum.internal. fd20::10 ``` -Instance names and addresses are illustrative; `internal` is the proposed shared +Instance names and addresses are illustrative; `datum.internal` is the proposed shared default domain. DNS supplies the canonical name; consumers do not construct it from a workload display name. The example resolver file describes effective settings; runtimes can apply them @@ -139,18 +139,18 @@ for these contracts. ### Names and search domains -Use `..internal` for instance +Use `..datum.internal` for instance identity in every VPC. The resource namespace is the customer-visible project namespace, not a mapped federation or edge namespace. VPC and project identifiers do not appear in the default domain. -For a guest in `production`, try `production.internal` before `internal`. +For a guest in `production`, try `production.datum.internal` before `datum.internal`. This resolves short instance names locally and supports VPC-scoped names without a product prefix. Use a fully qualified name to select another namespace unambiguously. Namespace names organize resolution; VPC access authorization provides the network isolation boundary. -Two VPCs can resolve `web-01-k7m2.production.internal` to different addresses. +Two VPCs can resolve `web-01-k7m2.production.datum.internal` to different addresses. The authorized DNS context selects the zone before lookup and cache access; matching domain names do not share records or grant access across VPCs. An instance attached to several VPCs uses the same allocated label in each context, @@ -187,8 +187,8 @@ status: nameservers: ["fd53::53"] # Resolve resource-namespace names before VPC-scoped names. searches: - - production.internal - - internal + - production.datum.internal + - datum.internal conditions: # The provider reports success after applying authorized settings to the guest. - type: DNSConfigured @@ -242,7 +242,7 @@ spec: ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01-k7m2.production.internal + canonicalFQDN: web-01-k7m2.production.datum.internal --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordContribution From 6a06533a977328b5b388643ba52cf3d121e689f3 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 15:50:46 -0500 Subject: [PATCH 06/22] docs: scope default instance names to the VPC DNS context --- docs/enhancements/internal-dns/README.md | 52 +++++++++++------------- 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index e7c0642c..15475756 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -44,7 +44,7 @@ overlapping addresses or private zone names. ### Goals - Assign an instance identity name within each attached VPC's managed namespace. -- Resolve short instance names within the resource's namespace. +- Resolve short instance names within the attached VPC. - Configure guest resolvers from verified network-interface settings. - Publish IPv4 and IPv6 host addresses only while their network attachment is valid. - Bound stale answers when instances disappear or publishers lose connectivity. @@ -59,7 +59,7 @@ overlapping addresses or private zone names. Every VPC with managed DNS enabled uses `datum.internal` as its default domain. DNS manages a separate zone within each VPC's context. Compute reserves an instance -name beneath its resource namespace. The name remains stable for that instance's +name in that zone. The name remains stable for that instance's lifetime; a replacement instance receives a distinct identity. Additional names can follow the VPC's DNS naming policy without adding a zone choice to each Compute resource. @@ -76,10 +76,10 @@ from another instance in that VPC: ```console $ cat /etc/resolv.conf nameserver fd53::53 -search production.datum.internal datum.internal +search datum.internal $ dig +search +short A web-01-k7m2 10.20.0.10 -$ dig +short AAAA web-01-k7m2.production.datum.internal. +$ dig +short AAAA web-01-k7m2.datum.internal. fd20::10 ``` @@ -139,29 +139,24 @@ for these contracts. ### Names and search domains -Use `..datum.internal` for instance -identity in every VPC. The resource namespace is the customer-visible project -namespace, not a mapped federation or edge namespace. VPC and project identifiers -do not appear in the default domain. +Use `.datum.internal` for instance identity in every +VPC and `datum.internal` as the default search domain. The authorized DNS context +scopes resolution to the attached network. Resource namespaces remain API +ownership boundaries; namespace, VPC, and project labels are unnecessary in the +default hostname. Reserve unique names across product publishers within each +context. -For a guest in `production`, try `production.datum.internal` before `datum.internal`. -This resolves short instance names locally and supports -VPC-scoped names without a product prefix. Use a fully qualified name to select -another namespace unambiguously. Namespace names organize resolution; VPC access -authorization provides the network isolation boundary. - -Two VPCs can resolve `web-01-k7m2.production.datum.internal` to different addresses. +Two VPCs can resolve `web-01-k7m2.datum.internal` to different addresses. The authorized DNS context selects the zone before lookup and cache access; matching domain names do not share records or grant access across VPCs. An instance attached to several VPCs uses the same allocated label in each context, with only that VPC's addresses in its answers. -Network services supply the managed domain and approved additional search domains. -Compute providers prepend the guest's resource namespace and report the effective -ordered list. Associating a private zone does not automatically add it to the -search list. Bound list size and qualify resolver search order across runtimes. -The consistent default domain, namespace-aware naming, and search contract -require joint DNS and network services API review. +Network services supply the managed domain and approved additional search +domains. Compute providers apply and report the effective ordered list. +Associating a private zone does not automatically add it to the search list. +Bound list size and qualify resolver search order across runtimes. The default +domain and naming contract require joint DNS and network services API review. ### Resolver configuration @@ -185,9 +180,8 @@ status: dns: # Galactic exposes this well-known address inside the authorized VPC. nameservers: ["fd53::53"] - # Resolve resource-namespace names before VPC-scoped names. + # Resolve short names within the attached VPC's DNS context. searches: - - production.datum.internal - datum.internal conditions: # The provider reports success after applying authorized settings to the guest. @@ -234,15 +228,15 @@ spec: dnsZoneRef: name: managed-application uid: 55555555-5555-4555-8555-555555555555 - # Reserve the allocated instance name within its resource namespace. - name: web-01-k7m2.production + # Reserve the allocated instance name within this VPC's managed zone. + name: web-01-k7m2 recordTypes: [A, AAAA] # Publish addresses only under a current eligible contribution. publicationPolicy: EligibleContributions ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01-k7m2.production.datum.internal + canonicalFQDN: web-01-k7m2.datum.internal --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordContribution @@ -263,13 +257,13 @@ spec: recordSets: - recordType: A records: - - name: web-01-k7m2.production + - name: web-01-k7m2 ttl: 30 a: content: 10.20.0.10 - recordType: AAAA records: - - name: web-01-k7m2.production + - name: web-01-k7m2 ttl: 30 aaaa: content: fd20::10 @@ -324,7 +318,7 @@ providers. Require two isolated VPCs with overlapping names and addresses, UDP and TCP queries from guests, renewal stability, address changes, deletion, publisher outages, grant revocation, stale replay, and restart coverage. Test the same default domain and hostname with different answers across VPCs, -equal short names in different namespaces, search precedence, fully +name collisions between product publishers, short-name resolution, fully qualified queries, and explicit additional-zone search configuration. Qualify guest configuration per runtime and mixed controller versions before enabling production. These are release criteria, not validation claims. From 217dceaefb50e4488031a5415c96fbf624ecafde Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 15:56:23 -0500 Subject: [PATCH 07/22] docs: propose qualified peer names with explicit DNS sharing --- docs/enhancements/internal-dns/README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 15475756..0fe19249 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -152,6 +152,15 @@ matching domain names do not share records or grant access across VPCs. An instance attached to several VPCs uses the same allocated label in each context, with only that VPC's addresses in its answers. +For peered networks, support optional qualified aliases such as +`web-01-k7m2.application.production.datum.internal`, using +`...datum.internal`. Explicit DNS +sharing makes the peer's qualified zone available in the local context. Keep +the local `datum.internal` zone and search domain; do not import a peer's default +zone or automatically add peer search domains. Cross-project peers require a +unique project qualifier or an explicitly chosen unique domain. Peer naming and +sharing authorization require review alongside the peering design. + Network services supply the managed domain and approved additional search domains. Compute providers apply and report the effective ordered list. Associating a private zone does not automatically add it to the search list. @@ -322,6 +331,8 @@ name collisions between product publishers, short-name resolution, fully qualified queries, and explicit additional-zone search configuration. Qualify guest configuration per runtime and mixed controller versions before enabling production. These are release criteria, not validation claims. +Before enabling peer DNS sharing, test qualified aliases, zone collisions, +sharing revocation, and unchanged local short-name resolution. ### Monitoring Requirements From 6610dccd19bc89ea36c945ee854044efedf8a47d Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Fri, 9 Oct 2026 21:43:21 -0500 Subject: [PATCH 08/22] Document instance DNS publication status --- docs/enhancements/internal-dns/README.md | 26 ++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 0fe19249..999022ae 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -224,6 +224,28 @@ and source-cluster identity for the registration, record types, and owner name. The runtime publisher cannot issue its own `DNSContributionGrant` or authorize resolver access. Project authorization and DNS admission enforce both boundaries. +Consumers read assigned names and publication readiness from `Instance.status.dns`: + +```yaml +status: + dns: + - network: + name: application + namespace: production + # Pin the attached network's lifetime. + networkUID: 55555555-5555-4555-8555-555555555555 + # Display names allocated by DNS within this network. + hostnames: [web-01-k7m2.datum.internal] + # Report publication independently of runtime readiness. + conditions: + - type: Ready + status: "True" + reason: Published + message: Private name is published + observedGeneration: 1 + lastTransitionTime: "2026-10-09T20:00:00Z" +``` + These examples use the proposed DNS API. UIDs, generations, and times are illustrative. Controllers write the resources; `status` shows controller output. @@ -336,8 +358,8 @@ sharing revocation, and unchanged local short-name resolution. ### Monitoring Requirements -Expose assigned names and DNS publication conditions through Instance status; -the exact Compute status schema requires API review. Track publication lag, +Expose assigned names and DNS publication conditions through `Instance.status.dns`. +Track publication lag, renewal failures, withdrawal delay, and guest configuration failures without per-instance metric labels. Set latency and withdrawal targets before release. From d4bd7386a6fe73ed98747a41cd149c66d55426bc Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Sun, 11 Oct 2026 14:06:11 -0500 Subject: [PATCH 09/22] Clarify network ownership of the default DNS domain --- docs/enhancements/internal-dns/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 999022ae..2cee97c2 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -57,9 +57,10 @@ overlapping addresses or private zone names. ## Proposal -Every VPC with managed DNS enabled uses `datum.internal` as its default domain. DNS -manages a separate zone within each VPC's context. Compute reserves an instance -name in that zone. The name remains stable for that instance's +Network services selects `datum.internal` as the default domain for VPCs with +managed DNS enabled and requests the zone from DNS. DNS manages a separate zone +within each VPC's context. Compute reads the allocated suffix and reserves an +instance name in that zone. The name remains stable for that instance's lifetime; a replacement instance receives a distinct identity. Additional names can follow the VPC's DNS naming policy without adding a zone choice to each Compute resource. From 72b627de869ceb23c3724b435211b372c7b3f350 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Mon, 12 Oct 2026 18:53:48 -0500 Subject: [PATCH 10/22] docs: Define stable Compute names and explicit DNS health updates --- docs/enhancements/internal-dns/README.md | 51 ++++++++++++------------ 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 2cee97c2..d8df6ad9 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -78,9 +78,9 @@ from another instance in that VPC: $ cat /etc/resolv.conf nameserver fd53::53 search datum.internal -$ dig +search +short A web-01-k7m2 +$ dig +search +short A web-01 10.20.0.10 -$ dig +short AAAA web-01-k7m2.datum.internal. +$ dig +short AAAA web-01.datum.internal. fd20::10 ``` @@ -140,21 +140,21 @@ for these contracts. ### Names and search domains -Use `.datum.internal` for instance identity in every +Use `.datum.internal` for instance identity in every VPC and `datum.internal` as the default search domain. The authorized DNS context scopes resolution to the attached network. Resource namespaces remain API ownership boundaries; namespace, VPC, and project labels are unnecessary in the default hostname. Reserve unique names across product publishers within each context. -Two VPCs can resolve `web-01-k7m2.datum.internal` to different addresses. +Two VPCs can resolve `web-01.datum.internal` to different addresses. The authorized DNS context selects the zone before lookup and cache access; matching domain names do not share records or grant access across VPCs. An -instance attached to several VPCs uses the same allocated label in each context, +instance attached to several VPCs uses the same label in each context, with only that VPC's addresses in its answers. For peered networks, support optional qualified aliases such as -`web-01-k7m2.application.production.datum.internal`, using +`web-01.application.production.datum.internal`, using `...datum.internal`. Explicit DNS sharing makes the peer's qualified zone available in the local context. Keep the local `datum.internal` zone and search domain; do not import a peer's default @@ -203,27 +203,28 @@ status: Network services deliver authorization through a separate, access-restricted provider contract bound to the project, interface, and VPC lifetimes. That -contract carries source references, fencing epochs, renewal sequences, and -expiry deadlines. These fields do not belong in the public interface's spec or +contract carries source references, fencing epochs, and sequences for explicit +access changes. These fields do not belong in the public interface's spec or status. Public DNS status reports observed configuration; it cannot authorize provider actions. Providers apply current, authorized settings and report `DNSConfigured` only -after successful application. They reconcile updates and expiry and reject -stale revisions. Resolver failure must not send private names to a public +after successful application. They reconcile explicit updates and reject stale revisions. Applied settings +persist during control plane outages. Resolver failure must not send private names to a public resolver. The internal delivery schema and runtime-specific guest configuration mechanism require joint review with network services before implementation. ### Record publication Compute creates one registration per instance and VPC, using the managed zone -reported by the matching DNS context. Allocation ties the name to the immutable -Instance UID; name reuse cannot adopt another instance's registration. +reported by the matching DNS context. The registration pins the immutable Instance lifetime; the hostname uses the +instance name. A recreated instance can use the same hostname after the previous +registration is withdrawn. A trusted grant issuer authorizes the Compute publisher's authenticated subject -and source-cluster identity for the registration, record types, and owner name. +for the registration, record types, and owner name. The runtime publisher cannot issue its own `DNSContributionGrant` or authorize -resolver access. Project authorization and DNS admission enforce both boundaries. +resolver access. Project RBAC controls resource access; DNS validates references and version fences. Consumers read assigned names and publication readiness from `Instance.status.dns`: @@ -236,7 +237,7 @@ status: # Pin the attached network's lifetime. networkUID: 55555555-5555-4555-8555-555555555555 # Display names allocated by DNS within this network. - hostnames: [web-01-k7m2.datum.internal] + hostnames: [web-01.datum.internal] # Report publication independently of runtime readiness. conditions: - type: Ready @@ -261,14 +262,14 @@ spec: name: managed-application uid: 55555555-5555-4555-8555-555555555555 # Reserve the allocated instance name within this VPC's managed zone. - name: web-01-k7m2 + name: web-01 recordTypes: [A, AAAA] # Publish addresses only under a current eligible contribution. publicationPolicy: EligibleContributions ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01-k7m2.datum.internal + canonicalFQDN: web-01.datum.internal --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordContribution @@ -289,13 +290,13 @@ spec: recordSets: - recordType: A records: - - name: web-01-k7m2 + - name: web-01 ttl: 30 a: content: 10.20.0.10 - recordType: AAAA records: - - name: web-01-k7m2 + - name: web-01 ttl: 30 aaaa: content: fd20::10 @@ -307,14 +308,12 @@ status: sequence: 18 eligible: true reason: NetworkInterfaceProgrammed - # DNS preserves this deadline and withdraws expired contributions locally. - validUntil: "2026-10-09T20:01:00Z" ``` Compute renews observations from the authoritative project Instance and verified interface state. Federation and edge copies retain source lifetime identity; their locally assigned UIDs cannot establish publication authority. Status writes -preserve DNS-owned fields. Retries cannot extend an observation's deadline. +preserve DNS-owned fields. Observation sequences advance when endpoint state changes. ### Lifecycle and health @@ -323,9 +322,11 @@ private host addresses. Public addresses and delegated prefixes are excluded. Each interface contributes only addresses from its own VPC. Address release, network deprogramming, or instance deletion triggers a -higher-sequence ineligible observation and cleanup. Project-side owner references -and bounded leases cover interrupted cleanup. Serving agents enforce expiry -during publisher outages; client caches remain bounded by the advertised TTL. +higher-sequence ineligible observation and cleanup. Project-side owner references support cleanup. Failed API observations and +unknown runtime health retain published state and report unknown readiness. +Confirmed unhealthy state or deletion withdraws records explicitly. DNS retains +last reported eligibility during publisher outages; client caches follow the +advertised TTL. Workload service discovery needs a separate registration and explicit application eligibility policy. It must withdraw unhealthy endpoints and accept recovery From e8410a55535a40875d7559e53a6cbdb62491567c Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Mon, 12 Oct 2026 18:58:59 -0500 Subject: [PATCH 11/22] docs: Align DNS outage and observation contracts --- docs/enhancements/internal-dns/README.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index d8df6ad9..41bf715b 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -100,9 +100,9 @@ the instance identity name, which can still support diagnostics. - Cross-VPC publication: pin project, VPC, context, and registration lifetimes; authorize publishers through scoped grants. -- Stale addresses: require fresh network eligibility observations and preserve - their deadlines through replication, retry, and restart. -- Incorrect guest resolution: apply only verified, unexpired settings and reject +- Stale addresses: publish explicit eligibility changes and retain the last + reported state when observations are unavailable. +- Incorrect guest resolution: apply verified settings and reject unsupported combinations of managed contexts across interfaces. ## Design Details @@ -369,19 +369,18 @@ per-instance metric labels. Set latency and withdrawal targets before release. Project APIs and scoped authorization support publication. Network services, Galactic, and the DNS fleet support resolver access. API outages stop fresh -observations; installed records and access remain subject to their original -deadlines. DNS errors requeue independently of Compute runtime readiness. +observations; installed records and access persist until explicit withdrawal. DNS errors requeue independently of Compute runtime readiness. ### Scalability Budget registrations per instance/VPC and contributions per interface. Measure -project API renewal load, controller queues, and grant provisioning at fleet -scale. Spread renewals with jitter and bound concurrent project operations. +project API publication load, controller queues, and grant provisioning at fleet +scale. Spread observation polls with jitter and bound concurrent project operations. ### Troubleshooting Trace the Instance and VPC lifetimes through context, registration, grant, and -contribution status. Check current network eligibility and deadlines, then +contribution status. Check network eligibility, observation health, and version fences, then compare inherited settings with guest configuration and query results. ## Implementation History @@ -390,7 +389,7 @@ compare inherited settings with guest configuration and query results. ## Drawbacks -Publication adds project API traffic and freshness coordination. DNS outages can +Publication adds project API traffic and endpoint observation coordination. DNS outages can interrupt applications that depend on assigned names despite a healthy runtime. ## Alternatives From b38ab2f3f9df2861598e84c13ab351bd18b4f663 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Mon, 12 Oct 2026 19:07:53 -0500 Subject: [PATCH 12/22] docs: Define explicit DNS publisher rollback --- docs/enhancements/internal-dns/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 41bf715b..2a55363d 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -339,16 +339,17 @@ proposal before implementation. Use the default-off `InternalDNSPublishing` feature gate for the publisher. Provider application of managed resolver settings requires a separate rollout -control. Disabling publication stops renewals; records expire, and applications -using their names can lose resolution. Reenabling requires fresh observations. -Test both controls independently. +control. Disabling publication stops updates and retains published records. +To remove published names, explicitly withdraw contributions before disabling +the publisher. Reenabling requires fresh observations. Test both controls +independently. ### Rollout, Upgrade and Rollback Planning Start with opted-in projects and supported single-context guests. Extend the shared Kubernetes end-to-end environment with actual Compute publishers and providers. Require two isolated VPCs with overlapping names and addresses, UDP -and TCP queries from guests, renewal stability, address changes, deletion, +and TCP queries from guests, outage continuity, address changes, deletion, publisher outages, grant revocation, stale replay, and restart coverage. Test the same default domain and hostname with different answers across VPCs, name collisions between product publishers, short-name resolution, fully From 3fff000f38cd7833e441b1e413efef04280cc081 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Tue, 13 Oct 2026 06:00:55 -0500 Subject: [PATCH 13/22] docs: identify instance DNS status by network reference --- docs/enhancements/internal-dns/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 2a55363d..4a477600 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -234,8 +234,6 @@ status: - network: name: application namespace: production - # Pin the attached network's lifetime. - networkUID: 55555555-5555-4555-8555-555555555555 # Display names allocated by DNS within this network. hostnames: [web-01.datum.internal] # Report publication independently of runtime readiness. From 75a1d88488ceb42c16f83a195de9e3fa9e637e18 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Tue, 13 Oct 2026 08:30:06 -0500 Subject: [PATCH 14/22] docs: Select managed DNS zones by name --- docs/enhancements/internal-dns/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 4a477600..3ef19708 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -255,10 +255,9 @@ kind: DNSRegistration metadata: name: instance-web-01 spec: - # Select the context's DNS-allocated managed zone and pin its lifetime. + # Select the context's DNS-allocated managed zone by name. dnsZoneRef: name: managed-application - uid: 55555555-5555-4555-8555-555555555555 # Reserve the allocated instance name within this VPC's managed zone. name: web-01 recordTypes: [A, AAAA] From e501a844f8c6a09cf7143510d5e8b1832525683f Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Tue, 13 Oct 2026 14:18:18 -0500 Subject: [PATCH 15/22] docs: Align Compute private DNS publication contract --- docs/enhancements/internal-dns/README.md | 67 ++++++++++-------------- 1 file changed, 28 insertions(+), 39 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 3ef19708..465573ee 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -99,7 +99,7 @@ the instance identity name, which can still support diagnostics. ### Risks and mitigations - Cross-VPC publication: pin project, VPC, context, and registration lifetimes; - authorize publishers through scoped grants. + scope publisher permissions through project RBAC. - Stale addresses: publish explicit eligibility changes and retain the last reported state when observations are unavailable. - Incorrect guest resolution: apply verified settings and reject @@ -113,7 +113,6 @@ the instance identity name, which can still support diagnostics. flowchart LR N[Network services] -->|Context and access| P[Consumer project API] C[Compute publisher] -->|Registration, addresses, eligibility| P - I[Trusted grant issuer] -->|Scoped contribution grant| P P -->|DNS intent| D[Shared DNS platform] P -->|Workload and network desired state| K[Karmada] K --> E[Edge interfaces and workload providers] @@ -201,12 +200,11 @@ status: lastTransitionTime: "2026-10-09T20:00:00Z" ``` -Network services deliver authorization through a separate, access-restricted -provider contract bound to the project, interface, and VPC lifetimes. That -contract carries source references, fencing epochs, and sequences for explicit -access changes. These fields do not belong in the public interface's spec or -status. Public DNS status reports observed configuration; it cannot authorize -provider actions. +Network services deliver resolver intent through an access-restricted provider +contract bound to the project, interface, and VPC lifetimes. Providers verify +the current Galactic network configuration before applying resolver settings. +Public DNS status reports observed configuration; it cannot authorize provider +actions. DNS manages publication ordering within its own control plane. Providers apply current, authorized settings and report `DNSConfigured` only after successful application. They reconcile explicit updates and reject stale revisions. Applied settings @@ -221,10 +219,9 @@ reported by the matching DNS context. The registration pins the immutable Instan instance name. A recreated instance can use the same hostname after the previous registration is withdrawn. -A trusted grant issuer authorizes the Compute publisher's authenticated subject -for the registration, record types, and owner name. -The runtime publisher cannot issue its own `DNSContributionGrant` or authorize -resolver access. Project RBAC controls resource access; DNS validates references and version fences. +Project RBAC permits the Compute publisher to manage registrations and +contributions. Network services manages resolver access separately. DNS +validates current registration references and orders publication internally. Consumers read assigned names and publication readiness from `Instance.status.dns`: @@ -261,8 +258,6 @@ spec: # Reserve the allocated instance name within this VPC's managed zone. name: web-01 recordTypes: [A, AAAA] - # Publish addresses only under a current eligible contribution. - publicationPolicy: EligibleContributions ttlSeconds: 30 status: # DNS-owned output used to display the assigned name to consumers. @@ -273,16 +268,14 @@ kind: DNSRecordContribution metadata: name: web-01-eth0 spec: - # Pin the reserved name and its publication policy generation. + # Pin the reserved name and its current generation. registrationRef: name: instance-web-01 uid: 66666666-6666-4666-8666-666666666666 generation: 1 - # Reference authorization issued separately from the runtime publisher. - grantRef: - name: web-01-compute - uid: 77777777-7777-4777-8777-777777777777 - generation: 1 + # Publish addresses and eligibility together in one update. + eligible: true + reason: NetworkInterfaceProgrammed # Include only private host addresses allocated and programmed for this VPC. recordSets: - recordType: A @@ -298,19 +291,15 @@ spec: aaaa: content: fd20::10 status: - # Compute-owned observation of this contribution's current spec. + # DNS-owned acknowledgement of the contribution generation. observedGeneration: 1 - # Echo the DNS-issued grant epoch and advance the observation sequence. - writerEpoch: 3 - sequence: 18 - eligible: true - reason: NetworkInterfaceProgrammed ``` -Compute renews observations from the authoritative project Instance and verified -interface state. Federation and edge copies retain source lifetime identity; -their locally assigned UIDs cannot establish publication authority. Status writes -preserve DNS-owned fields. Observation sequences advance when endpoint state changes. +Compute observes the authoritative project Instance and verified interface +state. It updates records and eligibility together using conditional API writes. +Federation and edge copies retain source lifetime identity; their locally +assigned UIDs cannot establish publication authority. DNS owns contribution +status. Unavailable observations retain the last reported contribution. ### Lifecycle and health @@ -318,8 +307,8 @@ Instance identity eligibility requires a live Instance and allocated, programmed private host addresses. Public addresses and delegated prefixes are excluded. Each interface contributes only addresses from its own VPC. -Address release, network deprogramming, or instance deletion triggers a -higher-sequence ineligible observation and cleanup. Project-side owner references support cleanup. Failed API observations and +Address release, network deprogramming, or instance deletion triggers +an atomic ineligible update and cleanup. Project-side owner references support cleanup. Failed API observations and unknown runtime health retain published state and report unknown readiness. Confirmed unhealthy state or deletion withdraws records explicitly. DNS retains last reported eligibility during publisher outages; client caches follow the @@ -347,7 +336,7 @@ Start with opted-in projects and supported single-context guests. Extend the shared Kubernetes end-to-end environment with actual Compute publishers and providers. Require two isolated VPCs with overlapping names and addresses, UDP and TCP queries from guests, outage continuity, address changes, deletion, -publisher outages, grant revocation, stale replay, and restart coverage. +publisher outages, publisher RBAC isolation, stale replay, and restart coverage. Test the same default domain and hostname with different answers across VPCs, name collisions between product publishers, short-name resolution, fully qualified queries, and explicit additional-zone search configuration. @@ -360,7 +349,7 @@ sharing revocation, and unchanged local short-name resolution. Expose assigned names and DNS publication conditions through `Instance.status.dns`. Track publication lag, -renewal failures, withdrawal delay, and guest configuration failures without +observation failures, withdrawal delay, and guest configuration failures without per-instance metric labels. Set latency and withdrawal targets before release. ### Dependencies @@ -372,13 +361,13 @@ observations; installed records and access persist until explicit withdrawal. DN ### Scalability Budget registrations per instance/VPC and contributions per interface. Measure -project API publication load, controller queues, and grant provisioning at fleet +project API publication load and controller queues at fleet scale. Spread observation polls with jitter and bound concurrent project operations. ### Troubleshooting -Trace the Instance and VPC lifetimes through context, registration, grant, and -contribution status. Check network eligibility, observation health, and version fences, then +Trace the Instance and VPC lifetimes through context, registration, and +contribution status. Check network eligibility and publication conditions, then compare inherited settings with guest configuration and query results. ## Implementation History @@ -397,6 +386,6 @@ Automatic publication keeps that lifecycle with Compute. ## Infrastructure Needed -Provision project-scoped publisher credentials, a separate trusted grant issuer, -VPC resolver access, shared DNS capacity, and the local Kubernetes validation +Provision project-scoped publisher credentials, VPC resolver access, shared DNS +capacity, and the local Kubernetes validation environment. This integration uses shared serving infrastructure across VPCs. From c227a097126fcb4aefbf365e6351b21b87299fee Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Tue, 13 Oct 2026 15:48:52 -0500 Subject: [PATCH 16/22] Document project discovery and shared staging rollout --- docs/enhancements/internal-dns/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 465573ee..15df9d8d 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -122,7 +122,8 @@ flowchart LR G -->|VPC-authorized private service path| D ``` -Compute uses authenticated project clients for publication. It identifies the +Compute discovers project identities through Milo, including newly created +projects, and uses authenticated project clients for publication. It identifies the VPC's context through trusted project and Network UIDs, matching the `DNSResolverContext.spec.consumerID` contract. DNS does not read Compute objects to determine addresses or health. @@ -332,9 +333,9 @@ independently. ### Rollout, Upgrade and Rollback Planning -Start with opted-in projects and supported single-context guests. Extend the -shared Kubernetes end-to-end environment with actual Compute publishers and -providers. Require two isolated VPCs with overlapping names and addresses, UDP +Enable staging for all projects and supported single-context guests. Extend +the shared Kubernetes end-to-end environment with actual Compute publishers +and providers. Require two isolated VPCs with overlapping names and addresses, UDP and TCP queries from guests, outage continuity, address changes, deletion, publisher outages, publisher RBAC isolation, stale replay, and restart coverage. Test the same default domain and hostname with different answers across VPCs, From 2124ed2096c2765da62f0c519b62027eebfe074c Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Wed, 21 Oct 2026 10:27:49 -0500 Subject: [PATCH 17/22] ci: refresh checks after the main update From 2882ed7f5150684a36a3b3e472fd6917d1578406 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Wed, 21 Oct 2026 10:29:58 -0500 Subject: [PATCH 18/22] test: stop after missing rendered values --- internal/cmd/compute/build/analysis_test.go | 1 + internal/workloadspec/render_test.go | 2 ++ 2 files changed, 3 insertions(+) diff --git a/internal/cmd/compute/build/analysis_test.go b/internal/cmd/compute/build/analysis_test.go index 0cb6623b..1ba52f02 100644 --- a/internal/cmd/compute/build/analysis_test.go +++ b/internal/cmd/compute/build/analysis_test.go @@ -932,6 +932,7 @@ func TestDefaultChecksAllowStaticETEXECEntrypoint(t *testing.T) { tc, elfFile := detectToolchain(data) if elfFile == nil { t.Fatal("expected ELF") + return } if elfFile.Type == elf.ET_DYN { t.Skip("test toolchain produced PIE by default") diff --git a/internal/workloadspec/render_test.go b/internal/workloadspec/render_test.go index c9f8e40d..49676132 100644 --- a/internal/workloadspec/render_test.go +++ b/internal/workloadspec/render_test.go @@ -197,6 +197,7 @@ func TestRenderPortsOpenIngress(t *testing.T) { policy := spec.NetworkInterfaces[0].NetworkPolicy if policy == nil { t.Fatal("declared ports must open matching ingress rules") + return } if len(policy.Ingress) != 2 { t.Fatalf("ingress rules = %d, want 2", len(policy.Ingress)) @@ -346,6 +347,7 @@ func TestRenderVM(t *testing.T) { vm := spec.Runtime.VirtualMachine if vm == nil { t.Fatal("virtualMachine not rendered") + return } if len(vm.Ports) != 1 { t.Errorf("vm ports = %d, want 1", len(vm.Ports)) From c59d902f472ce1f6512ca527b9677b3e2f36f831 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Wed, 21 Oct 2026 11:01:09 -0500 Subject: [PATCH 19/22] ci: raise watcher limits for Karmada tests --- .github/workflows/test-e2e.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/test-e2e.yml b/.github/workflows/test-e2e.yml index 195223e0..6d254ab5 100644 --- a/.github/workflows/test-e2e.yml +++ b/.github/workflows/test-e2e.yml @@ -48,6 +48,9 @@ jobs: with: go-version-file: go.mod + - name: Set file watcher limits + run: sudo sysctl -w fs.inotify.max_user_instances=8192 fs.inotify.max_user_watches=1048576 + # go-task drives the entire harness (task e2e:env:up / e2e:deploy / # e2e:test). The action publishes major refs as branches (v1/v2/v3); the # version input pins the go-task binary itself. repo-token avoids GitHub From b3c09c3d350b5a5adf8a0bb6908d71f14fa55381 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Wed, 21 Oct 2026 11:47:48 -0500 Subject: [PATCH 20/22] test: make CLI nil guards explicit for lint --- internal/cmd/compute/deploy/deploy_test.go | 1 + internal/cmd/compute/url/lookup_test.go | 2 ++ 2 files changed, 3 insertions(+) diff --git a/internal/cmd/compute/deploy/deploy_test.go b/internal/cmd/compute/deploy/deploy_test.go index f86a17ee..47a2847f 100644 --- a/internal/cmd/compute/deploy/deploy_test.go +++ b/internal/cmd/compute/deploy/deploy_test.go @@ -181,6 +181,7 @@ func TestPortFlagStaysRegistered(t *testing.T) { port := cmd.Flags().Lookup("port") if port == nil { t.Fatal("--port must stay registered for one release so it can error") + return } if !port.Hidden { t.Error("--port must not be advertised in help") diff --git a/internal/cmd/compute/url/lookup_test.go b/internal/cmd/compute/url/lookup_test.go index 36d8b758..4d2aec34 100644 --- a/internal/cmd/compute/url/lookup_test.go +++ b/internal/cmd/compute/url/lookup_test.go @@ -115,6 +115,7 @@ func TestForWorkloadPublished(t *testing.T) { } if info == nil { t.Fatal("ForWorkload returned nil for a published workload") + return } if info.URL != testCanonicalURL { @@ -344,6 +345,7 @@ func TestForWorkloadWithoutBackendsYet(t *testing.T) { } if info == nil { t.Fatal("info = nil, want a URL with no backends") + return } if info.Backends != (Backends{}) || info.Service != nil { t.Errorf("Backends = %+v, Service = %v, want empty", info.Backends, info.Service) From 6eb44077deb2da05d90e5e497baf719e5600c452 Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Thu, 22 Oct 2026 14:42:01 -0500 Subject: [PATCH 21/22] docs: Specify Compute DNS name conflicts and link the DNS types Review on #470 found instance-name uniqueness undesigned. Each VPC's managed zone is one flat namespace shared by every publisher, and Compute derives the label from an Instance name that is unique only within its namespace. The proposal now states the outcome DNS already implements: the oldest registration keeps a contested name, the loser reports NameConflict on its registration and in Instance.status.dns, and nothing is renamed. A recreated Instance does not reclaim its old name ahead of a registration that was already waiting; whether it should is recorded as an open question. It also notes that publication is limited today to Networks in the Instance's namespace. The copied DNSRegistration and DNSRecordContribution YAML is replaced by a link to the dns-operator types and a description of what Compute writes. Peered-network aliases are deferred to the DNS and network services designs. Retaining published state through a Compute outage with no maximum age is recorded as an accepted risk alongside the dns-operator decision. The proposal says what a guest on two managed VPCs gets, uses the DNSConfigured reasons the network services API defines, and describes publication from NetworkInterface allocation addresses with the runtime Pod and node as health gates, matching compute#518. Co-Authored-By: Claude Opus 5.5 --- docs/enhancements/internal-dns/README.md | 207 +++++++---------------- 1 file changed, 60 insertions(+), 147 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index 15df9d8d..bf3721bf 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -101,7 +101,10 @@ the instance identity name, which can still support diagnostics. - Cross-VPC publication: pin project, VPC, context, and registration lifetimes; scope publisher permissions through project RBAC. - Stale addresses: publish explicit eligibility changes and retain the last - reported state when observations are unavailable. + reported state when observations are unavailable. Retention has no maximum + age; see [Lifecycle and health](#lifecycle-and-health). +- Name collisions: the oldest registration keeps a contested name and others + report `NameConflict`; see [Name uniqueness](#name-uniqueness). - Incorrect guest resolution: apply verified settings and reject unsupported combinations of managed contexts across interfaces. @@ -140,89 +143,62 @@ for these contracts. ### Names and search domains -Use `.datum.internal` for instance identity in every -VPC and `datum.internal` as the default search domain. The authorized DNS context -scopes resolution to the attached network. Resource namespaces remain API -ownership boundaries; namespace, VPC, and project labels are unnecessary in the -default hostname. Reserve unique names across product publishers within each -context. - -Two VPCs can resolve `web-01.datum.internal` to different addresses. -The authorized DNS context selects the zone before lookup and cache access; -matching domain names do not share records or grant access across VPCs. An -instance attached to several VPCs uses the same label in each context, -with only that VPC's addresses in its answers. - -For peered networks, support optional qualified aliases such as -`web-01.application.production.datum.internal`, using -`...datum.internal`. Explicit DNS -sharing makes the peer's qualified zone available in the local context. Keep -the local `datum.internal` zone and search domain; do not import a peer's default -zone or automatically add peer search domains. Cross-project peers require a -unique project qualifier or an explicitly chosen unique domain. Peer naming and -sharing authorization require review alongside the peering design. - -Network services supply the managed domain and approved additional search -domains. Compute providers apply and report the effective ordered list. -Associating a private zone does not automatically add it to the search list. -Bound list size and qualify resolver search order across runtimes. The default -domain and naming contract require joint DNS and network services API review. +Use `.datum.internal` for instance identity in every VPC and `datum.internal` as the default search domain. The authorized DNS context scopes resolution to the attached network. Namespace, VPC, and project labels are not part of the default hostname. + +Compute derives the label from the Instance name: it lowercases the name, replaces every character outside `[a-z0-9-]` with `-`, and trims leading and trailing hyphens. A label longer than 63 characters is cut to 46 characters and suffixed with a hash of the full Instance name. DNS supplies the canonical name in registration status; consumers read it from `Instance.status.dns` and do not construct it. + +Two VPCs can resolve `web-01.datum.internal` to different addresses. The authorized DNS context selects the zone before lookup and cache access; matching domain names do not share records or grant access across VPCs. + +#### Name uniqueness + +Each VPC's managed zone is one flat namespace shared by every publisher: Compute instances, Connect exports, other products, and the customer's own static records. Instance names are unique only within a Kubernetes namespace, so Compute cannot guarantee that a label is free. Labels collide when two Instance names normalize to the same label (`web.01` and `web-01`), or when another publisher already holds the name. + +DNS resolves every collision with one rule, defined in the [DNS architecture](https://github.com/datum-cloud/dns-operator/pull/229) under "Publication and service discovery": static records claim first, then the oldest registration by creation time, with the UID breaking ties. Compute does not add a suffix and does not retry under another name. For the losing Instance: + +- Its `DNSRegistration` reports `Accepted=False` with reason `NameConflict` and publishes nothing. +- `Instance.status.dns` for that network lists no hostname and reports `Ready=False` with reason `NameConflict`. +- The Instance itself runs normally; only its name is missing. + +When the holder's registration is deleted, the next oldest waiting registration takes the name on the next compilation. A recreated Instance gets a new registration with a new creation time, so it does not reclaim its previous name ahead of a registration that was already waiting. There is no hold on a name across instance replacement. + +Today Compute publishes only for a Network in the Instance's own namespace; an interface on a Network in another namespace reports `Ready=Unknown` with reason `NetworkObservationUnavailable` and gets no name. Supporting cross-namespace attachments would make Instance-to-Instance collisions common and needs its own naming decision first. + +> [!NOTE] +> Open question: whether a replacement Instance should keep its predecessor's name, for example through a short hold on the claim, and whether Compute should fall back to a suffixed label on `NameConflict`. Neither is designed; the current behavior is first-come by registration age. + +#### Instances on several VPCs + +Compute publishes one registration per Instance and attached VPC, each with that VPC's addresses only, so other guests on either VPC can resolve the Instance under the same label. The guest itself supports one managed DNS context in the initial release. A guest whose interfaces span two managed VPCs gets no managed resolver: the provider reports `DNSConfigured=False` with reason `UnsupportedMultipleContexts` instead of merging resolver settings, as the [VPC DNS integration](https://github.com/datum-cloud/network-services-operator/pull/589) defines. A guest with several interfaces on the same VPC receives that VPC's settings once. + +#### Deferred: peered networks + +Names for peered networks, such as qualified aliases or sharing a peer's zone, belong in the DNS and network services designs and are out of scope here. Compute publishes only into the managed zone of each VPC it attaches to. + +Network services supply the managed domain and approved additional search domains. Compute providers apply and report the effective ordered list. Associating a private zone does not automatically add it to the search list. ### Resolver configuration -`NetworkInterface` is a public API. Its proposed read-only DNS status exposes -resolver addresses, search domains, and configuration readiness. Customers -select the network; they do not supply DNS authorization or renewal values. -This selected-field example includes proposed controller-written status: +Network services owns resolver settings and delivers them as read-only `NetworkInterface.status.dns`; the [VPC DNS integration](https://github.com/datum-cloud/network-services-operator/pull/589) defines that contract and the `DNSConfigured` condition. Customers select the network; they do not supply resolver settings or authorization. -```yaml -apiVersion: networking.datumapis.com/v1alpha -kind: NetworkInterface -metadata: - name: web-01-eth0 - namespace: production -spec: - network: - name: application - interfaceName: eth0 -status: - # Report effective settings without exposing internal authorization state. - dns: - # Galactic exposes this well-known address inside the authorized VPC. - nameservers: ["fd53::53"] - # Resolve short names within the attached VPC's DNS context. - searches: - - datum.internal - conditions: - # The provider reports success after applying authorized settings to the guest. - - type: DNSConfigured - status: "True" - reason: ResolverConfigured - lastTransitionTime: "2026-10-09T20:00:00Z" -``` +Compute providers apply the settings from the current interface status through the runtime's native resolver configuration and report `DNSConfigured` on the interface: -Network services deliver resolver intent through an access-restricted provider -contract bound to the project, interface, and VPC lifetimes. Providers verify -the current Galactic network configuration before applying resolver settings. -Public DNS status reports observed configuration; it cannot authorize provider -actions. DNS manages publication ordering within its own control plane. +- `True` with reason `Applied` after the settings are in effect in the guest. +- `False` with reason `UnsupportedMultipleContexts` when the guest's interfaces span more than one managed DNS context. -Providers apply current, authorized settings and report `DNSConfigured` only -after successful application. They reconcile explicit updates and reject stale revisions. Applied settings -persist during control plane outages. Resolver failure must not send private names to a public -resolver. The internal delivery schema and runtime-specific guest configuration -mechanism require joint review with network services before implementation. +Providers reconcile updates and ignore stale revisions. Applied settings persist during control-plane outages. A resolver failure must not send private names to a public resolver. The runtime-specific guest configuration mechanism requires joint review with network services before implementation; no Compute provider reports `DNSConfigured` yet. ### Record publication -Compute creates one registration per instance and VPC, using the managed zone -reported by the matching DNS context. The registration pins the immutable Instance lifetime; the hostname uses the -instance name. A recreated instance can use the same hostname after the previous -registration is withdrawn. +Compute publishes through the DNS-owned `DNSRegistration` and `DNSRecordContribution` resources. Their schema and validation are defined by the [dns-operator API types](https://github.com/datum-cloud/dns-operator/blob/fix/internal-dns-name-references/api/v1alpha1/internaldns_project_types.go) ([dns-operator#244](https://github.com/datum-cloud/dns-operator/pull/244)); this document describes only what Compute writes. + +For each Instance and attached VPC, Compute finds the ready `DNSResolverContext` whose `consumerID` is `/` and creates, in the project namespace: + +- one `DNSRegistration` in the context's managed zone, with the label from [Names and search domains](#names-and-search-domains), record types `A` and `AAAA`, and a 30 second TTL; +- one `DNSRecordContribution` that pins the registration's name, UID, and generation and carries the addresses and eligibility together. + +Both are named from a hash of the Instance UID and VPC UID and are owned by the project-side Instance, so deletion and replacement clean them up through garbage collection even without a final reconcile. Compute refuses to publish from an edge copy whose UID does not match the project Instance. -Project RBAC permits the Compute publisher to manage registrations and -contributions. Network services manages resolver access separately. DNS -validates current registration references and orders publication internally. +Project RBAC permits the Compute publisher to manage registrations and contributions. Network services manages resolver access separately. DNS validates current registration references, resolves name conflicts, and orders publication internally. Consumers read assigned names and publication readiness from `Instance.status.dns`: @@ -232,9 +208,9 @@ status: - network: name: application namespace: production - # Display names allocated by DNS within this network. + # Names allocated by DNS within this network. hostnames: [web-01.datum.internal] - # Report publication independently of runtime readiness. + # Publication is reported independently of runtime readiness. conditions: - type: Ready status: "True" @@ -244,81 +220,20 @@ status: lastTransitionTime: "2026-10-09T20:00:00Z" ``` -These examples use the proposed DNS API. UIDs, generations, and times are -illustrative. Controllers write the resources; `status` shows controller output. +The `Ready` condition reports `Published`, `PendingPublication`, `NameConflict`, `AddressUnavailable`, `PublicationFailed`, or, with status `Unknown`, `NetworkObservationUnavailable`. -```yaml -apiVersion: dns.networking.miloapis.com/v1alpha1 -kind: DNSRegistration -metadata: - name: instance-web-01 -spec: - # Select the context's DNS-allocated managed zone by name. - dnsZoneRef: - name: managed-application - # Reserve the allocated instance name within this VPC's managed zone. - name: web-01 - recordTypes: [A, AAAA] - ttlSeconds: 30 -status: - # DNS-owned output used to display the assigned name to consumers. - canonicalFQDN: web-01.datum.internal ---- -apiVersion: dns.networking.miloapis.com/v1alpha1 -kind: DNSRecordContribution -metadata: - name: web-01-eth0 -spec: - # Pin the reserved name and its current generation. - registrationRef: - name: instance-web-01 - uid: 66666666-6666-4666-8666-666666666666 - generation: 1 - # Publish addresses and eligibility together in one update. - eligible: true - reason: NetworkInterfaceProgrammed - # Include only private host addresses allocated and programmed for this VPC. - recordSets: - - recordType: A - records: - - name: web-01 - ttl: 30 - a: - content: 10.20.0.10 - - recordType: AAAA - records: - - name: web-01 - ttl: 30 - aaaa: - content: fd20::10 -status: - # DNS-owned acknowledgement of the contribution generation. - observedGeneration: 1 -``` +### Lifecycle and health -Compute observes the authoritative project Instance and verified interface -state. It updates records and eligibility together using conditional API writes. -Federation and edge copies retain source lifetime identity; their locally -assigned UIDs cannot establish publication authority. DNS owns contribution -status. Unavailable observations retain the last reported contribution. +Addresses come from the Instance's `NetworkInterface` allocations, not from the runtime Pod, because not every runtime reports Pod IPs ([compute#518](https://github.com/datum-cloud/compute/pull/518)). Each allocated host address is published as-is; for a delegated IPv6 block, Compute publishes the block's first address, which Galactic configures as the guest address. Each interface contributes only addresses from its own VPC. -### Lifecycle and health +An interface is eligible only when it is allocated and programmed for the Instance's current lifetime and location, its runtime Pod exists, and the Pod's node is healthy. The Pod and node gate eligibility; they do not supply addresses. -Instance identity eligibility requires a live Instance and allocated, programmed -private host addresses. Public addresses and delegated prefixes are excluded. -Each interface contributes only addresses from its own VPC. +Address release, network deprogramming, or Instance deletion triggers an ineligible update and cleanup. Failed API observations and unknown runtime health retain published state and report unknown readiness. Confirmed unhealthy state or deletion withdraws records explicitly. Client caches follow the advertised TTL. -Address release, network deprogramming, or instance deletion triggers -an atomic ineligible update and cleanup. Project-side owner references support cleanup. Failed API observations and -unknown runtime health retain published state and report unknown readiness. -Confirmed unhealthy state or deletion withdraws records explicitly. DNS retains -last reported eligibility during publisher outages; client caches follow the -advertised TTL. +> [!IMPORTANT] +> Accepted risk: published records have no maximum age. If Compute cannot observe an Instance, for example during a Compute or project API outage, DNS keeps serving its last reported addresses until Compute reports again or the records are withdrawn. This matches the dns-operator decision to keep applied configuration through control-plane outages rather than expire it ([dns-operator#229](https://github.com/datum-cloud/dns-operator/pull/229), "Publication and service discovery"). A stale answer is preferred over losing every name during an outage. -Workload service discovery needs a separate registration and explicit application -eligibility policy. It must withdraw unhealthy endpoints and accept recovery -only from a fresh observation. That capability requires a separate Compute API -proposal before implementation. +Workload service discovery needs a separate registration and explicit application eligibility policy. It must withdraw unhealthy endpoints and accept recovery only from a fresh observation. That capability requires a separate Compute API proposal before implementation. ## Production Readiness Review Questionnaire @@ -343,8 +258,6 @@ name collisions between product publishers, short-name resolution, fully qualified queries, and explicit additional-zone search configuration. Qualify guest configuration per runtime and mixed controller versions before enabling production. These are release criteria, not validation claims. -Before enabling peer DNS sharing, test qualified aliases, zone collisions, -sharing revocation, and unchanged local short-name resolution. ### Monitoring Requirements From a448301d16e4f15cb530fa9070d822caa7e1ee2e Mon Sep 17 00:00:00 2001 From: Scot Wells Date: Thu, 22 Oct 2026 14:43:13 -0500 Subject: [PATCH 22/22] docs: Label peered-network names as future work Peered names are not part of the delivered design. Label the section as future work and state, as the DNS architecture does, that network peering does not share zones or extend DNS access. Co-Authored-By: Claude Opus 5.5 --- docs/enhancements/internal-dns/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/enhancements/internal-dns/README.md b/docs/enhancements/internal-dns/README.md index bf3721bf..4003c492 100644 --- a/docs/enhancements/internal-dns/README.md +++ b/docs/enhancements/internal-dns/README.md @@ -170,9 +170,9 @@ Today Compute publishes only for a Network in the Instance's own namespace; an i Compute publishes one registration per Instance and attached VPC, each with that VPC's addresses only, so other guests on either VPC can resolve the Instance under the same label. The guest itself supports one managed DNS context in the initial release. A guest whose interfaces span two managed VPCs gets no managed resolver: the provider reports `DNSConfigured=False` with reason `UnsupportedMultipleContexts` instead of merging resolver settings, as the [VPC DNS integration](https://github.com/datum-cloud/network-services-operator/pull/589) defines. A guest with several interfaces on the same VPC receives that VPC's settings once. -#### Deferred: peered networks +#### Future work: peered networks -Names for peered networks, such as qualified aliases or sharing a peer's zone, belong in the DNS and network services designs and are out of scope here. Compute publishes only into the managed zone of each VPC it attaches to. +Peered (cross-VPC) names are not part of this design. Network peering does not automatically share zones or extend DNS access, matching the [DNS architecture](https://github.com/datum-cloud/dns-operator/pull/229). Qualified aliases or sharing a peer's zone would belong in the DNS and network services designs. Compute publishes only into the managed zone of each VPC it attaches to. Network services supply the managed domain and approved additional search domains. Compute providers apply and report the effective ordered list. Associating a private zone does not automatically add it to the search list.