Deepen Water Data endpoint acquisition

[thodson-usgs]

Problem Statement

A caller can configure the Water Data adapter with a different base URL and reasonably expect every Water Data query in that scope to avoid the default host. PR #353 supports that behavior across the OGC, Samples, Statistics, and Ratings collection families, but the guarantee currently depends on every collection-family call site remembering to wrap a raw endpoint constant before building a request.

That is a shallow module interface: it exposes values that look ready to use while placing the critical redirection invariant on each caller. A new collection family or an additional request path can accidentally bypass the configured base URL and send traffic to the default Water Data host. The current safeguard is a source-level AST test that searches for bare endpoint constants after the fact. This makes correctness procedural rather than structural, reduces locality, and forces tests to understand implementation shape instead of using the same interface as callers.

Solution

Deepen the Water Data endpoint module so it is the single seam through which collection-family modules obtain request destinations. Endpoint acquisition must apply the active WaterdataConfiguration base URL at request time, preserving each collection family's path beneath that root. Collection-family modules must not receive a valid, unredirected endpoint that they can accidentally use directly.

From a caller's perspective, behavior remains simple and unchanged except that the existing promise becomes reliable: inside a scoped Water Data configuration, every Water Data getter targets the configured root; outside that scope, every getter targets the established default root. The implementation continues to honor ContextVar isolation, lazy configuration resolution, code-only base URL configuration, host-scoped credentials, and the existing transport behavior established by PR #353 and its ADRs.

User Stories

1. As a Water Data caller, I want a configured base URL to redirect every Water Data query, so that I can use a mirror or test deployment without accidental traffic to the default host.
2. As a Water Data caller, I want OGC collection getters to honor the configured root, so that monitoring locations, time series, daily values, and other OGC collections stay within the selected deployment.
3. As a Water Data caller, I want Samples getters to honor the configured root, so that discrete sample queries do not bypass the same scoped configuration.
4. As a Water Data caller, I want Statistics getters to honor the configured root, so that period-of-record and date-range statistics use the selected deployment.
5. As a Water Data caller, I want Ratings getters to obtain their catalog endpoint from the configured root, so that rating discovery does not contact the default deployment unexpectedly.
6. As a Water Data caller, I want one Water Data configuration to move all four endpoint families together, so that the adapter behaves as one coherent unit.
7. As an existing caller, I want unconfigured getters to continue using the established Water Data root, so that the refactor does not change normal retrieval behavior.
8. As a caller using a scoped configuration block, I want redirection to begin and end with that block, so that later queries return to their previous effective configuration.
9. As a caller using nested configuration blocks, I want endpoint acquisition to use the innermost effective configuration, so that the established precedence rules remain true.
10. As a caller running concurrent tasks, I want each task's endpoint acquisition to honor its own ContextVar state, so that one task's mirror cannot redirect another task's query.
11. As a caller using threads, I want endpoint acquisition to remain isolated by the existing configuration mechanism, so that scoped redirection does not become process-global state.
12. As a caller, I want each collection family's established path to be preserved beneath the configured root, so that changing the root does not alter which collection or resource is requested.
13. As a caller inspecting retrieval metadata, I want the reported request URL to reflect the destination actually contacted, so that troubleshooting remains trustworthy.
14. As a security-conscious caller, I want credentials to remain host-scoped after endpoint acquisition, so that redirecting Water Data to another host does not send the Water Data key there.
15. As a security-conscious caller, I want response-provided asset and pagination destinations to continue using existing validation and credential rules, so that endpoint deepening does not broaden trust to unrelated hosts.
16. As a caller with an invalid configured base URL, I want the existing configuration validation and error taxonomy to remain unchanged, so that failures still occur at the established seam with familiar errors.
17. As a contributor adding a Water Data collection family, I want one endpoint-acquisition seam to handle scoped redirection, so that I do not need to learn and reproduce a wrapper convention.
18. As a contributor adding another request within an existing collection family, I want the normal way of obtaining the destination to be safe by construction, so that omitting a special wrapper cannot silently bypass configuration.
19. As a maintainer, I want endpoint roots and collection-family paths to remain local to the endpoint module, so that changes do not require searching unrelated callers.
20. As a maintainer, I want redirection behavior to be implemented once, so that a fix applies to every Water Data collection family.
21. As a maintainer, I want collection-family modules to depend on a small endpoint interface rather than raw endpoint data plus usage rules, so that the endpoint module has greater depth and leverage.
22. As a maintainer, I want the wrong endpoint usage to be difficult or impossible to express through the endpoint module, so that correctness does not depend on a source-text audit.
23. As a test author, I want to verify redirection through public Water Data getters, so that tests exercise the same interface and behavior that callers use.
24. As a test author, I want one parameterized behavioral contract to cover representative getters from every endpoint family, so that the test seam stays small while the guarantee remains broad.
25. As a test author, I want all HTTP behavior mocked offline, so that endpoint tests are deterministic and do not spend quota or require network access.
26. As a test author, I want the AST use-site test removed once bypass is structurally unavailable, so that tests no longer encode internal syntax.
27. As a maintainer, I want the endpoint refactor to preserve the package's dependency direction, so that Water Data collection families do not create cycles or reach into configuration internals.
28. As a maintainer, I want no new ContextVar or parallel configuration grammar, so that show_configuration() and request behavior cannot disagree.
29. As an existing caller, I want all public getter signatures, return shapes, metadata behavior, and import paths to remain unchanged, so that this architectural improvement requires no migration.
30. As a documentation reader, I want the existing meaning of Water Data base_url to remain accurate, so that no new user-facing configuration concept must be learned.

Implementation Decisions

• The Water Data endpoint module is the single seam for obtaining initial destinations for Water Data collection families.
• The module must be deepened rather than deleted. It continues to own the default Water Data root and the paths for OGC, Samples, Statistics, and Ratings, while also owning the rule that combines those paths with the effective configured root.
• Endpoint acquisition resolves the effective Water Data base URL at request time. It must not capture configuration at import time because scoped configuration is delivered through a ContextVar.
• Collection-family modules must obtain destinations through the endpoint module's interpreted interface. They must not import or receive a valid raw endpoint value that can bypass scoped redirection through ordinary use.
• The exact private representation is an implementation choice. It may use functions, immutable endpoint descriptors, or another internal shape, provided callers have one small interface and cannot accidentally skip configuration resolution.
• One configured Water Data root moves the OGC, Samples, Statistics, and Ratings families together, as required by ADR 0011.
• Collection-family path suffixes remain stable beneath either the default or configured root.
• This change modifies no public getter interface, return contract, metadata contract, or documented configuration grammar.
• WaterdataConfiguration(base_url=...) remains code-only. The file and environment continue to reject base URL redirection.
• The existing seven-rung, per-setting precedence chain remains authoritative. This work does not add a source, alter source ordering, or introduce a second configuration resolver.
• The existing configuration ContextVar remains the sole scoped-state mechanism. Endpoint acquisition must not add process-global mutable state or another ContextVar.
• Host-scoped credential policy remains owned by the existing credential and transport modules. Endpoint acquisition chooses a destination; it does not decide whether a key may be attached.
• Response-provided cursors, STAC asset destinations, and other follow-up links remain governed by their existing validation and host-scoping rules. A configured initial root does not authorize arbitrary response-provided hosts.
• Shared transport remains adapter-neutral. The deepening belongs in the Water Data adapter because endpoint paths and the meaning of one Water Data root are adapter conventions.
• Existing lazy validation and ConfigurationError behavior remain unchanged.
• The endpoint module must preserve dependency direction and remain free of collection-family imports, avoiding cycles.
• The source-level rule that every raw endpoint use be wrapped should be deleted when the new module interface makes that misuse structurally unavailable. Do not preserve an obsolete AST test merely to test the old implementation shape.
• No new public exports are required. Internal names may change as needed because the outcome is behavior preservation and a deeper internal module.
• If current documentation already describes one Water Data base URL moving all families, no conceptual documentation change is needed. Update only references that name an internal mechanism removed by the refactor.

Testing Decisions

• Good tests assert externally observable behavior through public Water Data getters. They should not assert that a helper was called, inspect an internal endpoint representation, or search source syntax.
• Use one existing highest seam: public Water Data getters invoked inside dataretrieval.configure(WaterdataConfiguration(base_url=...)).
• Build a parameterized behavioral contract with representative getters for the four endpoint families:
  • an OGC collection getter;
  • a Samples getter;
  • a Statistics getter;
  • a Ratings getter.
• Mock HTTP with the existing pytest-httpx fixture and existing response fixtures. The tests must remain fully offline.
• For each representative getter, assert that the initial request uses the configured root and preserves the expected family path and query semantics.
• Assert that no request from the exercised query contacts the default Water Data host when a different root is configured, except where a response intentionally names an independently validated asset destination.
• Assert that the same representative getters use their established default destinations when no base URL is configured.
• Assert that leaving the scoped block restores default endpoint behavior, using public getters rather than inspecting ContextVar state.
• Preserve or extend the existing credential-host-scoping behavioral test: a configured root on another host must not receive the Water Data key.
• For Ratings, distinguish the configured catalog destination from response-provided asset destinations. The catalog must use the configured root; assets must continue to follow the existing per-destination credential and validation policy.
• Existing mocked getter tests are the prior art for exact request paths and parameters. Existing configuration redirect tests are the prior art for scoped base URL behavior and credential stripping.
• Remove the AST test that enumerates bare endpoint-constant uses after the deepened interface makes raw use unavailable through normal imports. The public getter contract becomes the authoritative test surface.
• Keep architecture checks limited to durable dependency-direction claims that behavioral tests cannot express. Do not add a new syntax-shape test for the replacement implementation.
• Run the complete Water Data, configuration, credential-host-scoping, architecture, typing, and import-contract suites. The full offline suite must remain green.

Out of Scope

• Changing the configuration precedence chain, source ordering, profile grammar, lazy validation, or ContextVar delivery.
• Adding base URL configuration to the environment or configuration file.
• Adding a base URL parameter to public getters.
• Changing public getter signatures, return shapes, metadata types, or import paths.
• Redesigning adapter configuration classes or the adapter registry.
• Changing NGWMN, NWDC, NLDI, WQP, StreamStats, or deprecated NWIS endpoint behavior.
• Modelling shared quota pools or host-level adapter settings.
• Changing credential eligibility, key attachment, redirect stripping, or response-link security policy.
• Rewriting response-provided pagination cursors or rating asset destinations under the configured Water Data root.
• Redesigning service-neutral transport, retry, pagination, chunking, planning, or fan-out.
• Introducing a general endpoint abstraction for unrelated adapters. This work has one adapter and one concrete invariant; a cross-adapter seam would be hypothetical.
• Renaming collections, collection families, settings, or existing public functions.
• Retaining implementation-shape tests solely to preserve the old wrapper design.

Further Notes

• This specification is a follow-up to the current state of PR feat(config)!: resolve settings through a layered chain #353 and assumes its accepted layered configuration, adapter-scoped settings, configuration profiles, and code-only base URL behavior.
• ADR 0011 explicitly defines Water Data as one adapter whose configured root moves OGC, Samples, Statistics, and Ratings together. This work makes that accepted decision structural rather than procedural.
• The architecture review found nine endpoint use sites across five Water Data collection-family modules and a dedicated AST test enforcing wrapper use. That evidence is why this candidate is stronger than broader configuration refactors.
• The deletion test supports deepening rather than deletion: removing the endpoint module would spread endpoint paths and redirection logic across collection-family callers; making its interface smaller and safer increases leverage and locality.
• The issue is intentionally limited to endpoint acquisition. Other architecture-review candidates—adapter identity threading and the private configuration seam—require separate decisions and are not bundled here.