Get rid of pydantic and move data validation to standard dataclasses

[francbartoli]

Overview

This PR aims at removing pydantic from the core dependencies in favor of the standard dataclasses while offering the same level of data validation in the codebase. At the same time it improves the extensibility of the plugin system in order to write custom plugins for data validation with pydantic or other external validation libraries.
A new PluginContext dataclass has been added to pass dependencies in the plugin system. Downstream projects/forks can extend PluginContext via subclassing to integrate a custom validation logic.
The change has a minimal footprint on the plugin design because it guarantees backwards compatibility so existing pygeoapi instances can update with zero-risk impact.

Related Issue / discussion

Mainly discussed and agreed during the summit with the ultimate goal to get rid of pydantic and provide downstream data validation with custom pydantic-based (or similar tools) plugins

#1547
#1664

Additional information

A new RFC has been raised to voting about this design change

The plan to achieve the above goal is the following:

• Extend the load_plugin in plugin.py method with an optional context based on a PluginContext dataclass
• Refactor BaseProvider, BaseProcessor and BaseManager for backwards compatibility
• Migrate APIRules to dataclass
• Migrate TileSetMetadata to dataclass
• Migrate MVTTilesJson to dataclass
• Remove Pydantic from the dependencies

Dependency policy (RFC2)

• I have ensured that this PR meets RFC2 requirements

Updates to public demo

• I have ensured that breaking changes to the pygeoapi master demo server have been addressed
  • https://github.com/geopython/demo.pygeoapi.io/blob/master/services/pygeoapi_master/local.config.yml

Contributions and licensing

(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

• I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution
• I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

[francbartoli]

CI failures are not related to these changes

[webb-ben]

Its not entirely clear to me how the introduction of PluginContext is coupled this closesly with the removal of pydantic. Furthermore I am not quite sure I understand the intended utility of it - perhaps this is a documentation question. Where / How would this be used?

[francbartoli]

Its not entirely clear to me how the introduction of PluginContext is coupled this closesly with the removal of pydantic. Furthermore I am not quite sure I understand the intended utility of it - perhaps this is a documentation question. Where / How would this be used?

That's a very good question @webb-ben, I'm happy to document what's the intended usage downstream indeed

[francbartoli]

@webb-ben I've added an example about a potential use case for data validation with two variants, one with a default validator (standard library only) and the second with a downstream override (pydantic via ValidatingContext).
Let me also clarify why PluginContext and pydantic removal are in the same PR.

They are technically independent but share the same design goal: making the plugin system more flexible for downstream projects.

In the new example there is a ValidatedGeoJSONProvider that explains how the PluginContext would be used. In short, I would consider these key points:

1. The provider is technology-agnostic: it calls self._feature_validator(**properties) on create()/update() without knowing whether the validator is a dataclass, a pydantic model, or any other callable. The only contract is: accept *kwargs, raise on invalid input.

2. Without PluginContext (pygeoapi core today), the current provider could be improved with a default validator from its own JSON Schema (provider.fields) using only dataclasses and validate_type from pygeoapi core, no external dependencies. This catches type errors on transactions that pygeoapi does not catch today. For example, a POST with "scalerank": "not_a_number" to a collection where scalerank is declared as integer is currently accepted and written to disk silently — this is just an example not in scope for this PR.

3. With PluginContext (downstream), a project that uses pygeoapi as a library can subclass PluginContext, inject a stricter validator (e.g. a pydantic BaseModel with value ranges, allowed values, required fields), and the same provider code applies those rules without any modification. The validation logic is decoupled from the provider implementation.

Again, this is just one of the possible use cases. The pattern would be the same as self.logger in BaseProvider: sensible default when no context is provided, override via injection when it is.