config: load and validate generated config models by MikeGoldsmith · Pull Request #4898 · open-telemetry/opentelemetry-python

MikeGoldsmith · 2026-02-06T13:39:33Z

Description

The next step of implementing configuration file support. This PR loads and validates the generated config models. Using the model to configure the SDK will come in later PRs.

Based on the following PR which added model generation from the schema:

config: generate model code from json schema #4879

Contributes to:

Add declarative configuration prototype #3631

Type of change

New feature (non-breaking change which adds functionality)

Changes

Stores a local version of the schema for use with data model generation and schema validation
Updates code generation tooling to use the local bundled schema instead of fetching from URL
Adds file configuration support: YAML/JSON loading, env var substitution, and schema validation against the vendored OTel config JSON schema
Adds jsonschema >= 4.0 to the file-configuration optional extra
Bumps typing_extensions pin to 4.12.0 in test requirements for Python 3.13+ compatibility (referencing, a transitive dep of jsonschema, uses TypeVar defaults which require this version)

How Has This Been Tested?

36 unit tests covering file loading, parsing (YAML/JSON), env substitution, and schema validation.

Configuration Loading Tests (test_loader.py - 18 tests)
Environment Variable Substitution Tests (test_env_substitution.py - 15 tests)
Model Tests (test_models.py - 3 tests)

Does This PR Require a Contrib Repo Change?

No.

Checklist:

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

Proposing that the first step towards implementing OpenTelemetry Configuration is to produce the model code from the json schema. I did a quick search for tools available to do this and came across datamodel-codegen which seems to do what i expected. Will open following pull requests (in draft) to use this model code, i just want to keep these as clearly separated as possible to make reviewing them easier. Signed-off-by: alex boten <223565+codeboten@users.noreply.github.com>

… into codeboten/generate-config-model-from-schema

…ate-config-model-from-schema fix code-generation command and regenerate models

…ate-config-model-from-schema.2 Fix lint errors and update uv.lock file

Signed-off-by: alex boten <223565+codeboten@users.noreply.github.com>

… into codeboten/generate-config-model-from-schema

…com:codeboten/opentelemetry-python into mike/otel-config/1-loading-validation

datamodel-codegen section was inserted between [tool.pyright] and its include/exclude config, causing pyright to check entire repo (599 files) instead of just included paths. Moved datamodel-codegen section after pyright config.

…com:codeboten/opentelemetry-python into mike/otel-config/1-loading-validation

- Fix re.sub callback return type (must return str, not str | None) - Rename DOLLAR_PLACEHOLDER to dollar_placeholder (pylint naming) - Rename f to temp_file/config_file (pylint naming) - Update uv.lock

Fixes yaml import warning by installing PyYAML type stubs and the file-configuration optional dependencies.

github-actions · 2026-03-03T04:00:03Z

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

Assisted-by: Claude Sonnet 4.6

- Bundle schema.json (v1.0.0-rc.3) alongside models.py - Add jsonschema >= 4.0 to file-configuration optional extra - Validate parsed config against schema before constructing model, with field path included in error messages for nested violations - Switch datamodel-codegen to use local schema (drop [http] extra) - Add schema validation tests (wrong type, missing required, nested path, enum violation) Assisted-by: Claude Sonnet 4.6

- Replace global statement with list cache in _get_schema - Extract _validate_schema helper to reduce branches/statements in load_config_file Assisted-by: Claude Sonnet 4.6

jsonschema's `referencing` dep uses TypeVar defaults which requires typing_extensions>=4.12.0 on Python 3.13+ (4.10.0 raises AttributeError on TypeVar.__default__). Assisted-by: Claude Sonnet 4.6

tammy-baylis-swi · 2026-03-04T23:45:21Z

Thank you for this @MikeGoldsmith ! I'm new to opentelemetry-configuration so figuring this out.

The local schema copy is nice for performance. Is this something to semi-automatically manage later, kinda like the semconvgen?

Once this gets merged, is this correct?: opentelemetry-sdk[file-configuration] is optional. It can be used to load config from yaml/json but not both. The local schema defines which env vars can be substituted -- these may or may not overlap with current SDK env vars. Config is loaded but values are not applied to SDK components, i.e. that'll be later.

There's mention of required_env_var in a few places. Where do I look to know which are required?

MikeGoldsmith · 2026-03-05T11:46:07Z

The local schema copy is nice for performance. Is this something to semi-automatically manage later, kinda like the semconvgen?

We added the model generation code in a previous PR, this PR builds on top of it to support loading a customer config and validating it. Previously we were looking up the schema from a URL, in this PR I decided to vendor it so we can do runtime validation too. There is a script here that can be used to regenerate the schema models:

PR to add model generation config: generate model code from json schema #4879
model generation script in tox.ini generate-config-from-jsonschema

Once this gets merged, is this correct?: opentelemetry-sdk[file-configuration] is optional. It can be used to load config from yaml/json but not both.

It supports both YAML and JSON — the loader detects the format based on file extension (.yaml/.yml for YAML, .json for JSON).

The local schema defines which env vars can be substituted

The schema defines the expected structure of the config file. Which env vars get substituted depends on what ${VAR} references appear in the user's config file — the schema doesn't control those.

Config is loaded but values are not applied to SDK components, i.e. that'll be later.

Correct - I've put an implementation plan in this issue to help break up the work into smaller chunks:

Add declarative configuration prototype #3631 (comment)

There's mention of required_env_var in a few places. Where do I look to know which are required?

"Required" here means any ${VAR} reference without a default value (i.e. ${VAR} vs ${VAR:-default}). There's no fixed list — it's determined by what's in the user's config file. If a referenced env var is unset and has no default, a ConfigurationError is raised.

Hope that helps, thanks @tammy-baylis-swi 😄

Assisted-by: Claude Sonnet 4.6

tammy-baylis-swi · 2026-03-06T19:52:52Z

Thank you @MikeGoldsmith , especially for those links. If only I had scrolled down 😁

It supports both YAML and JSON — the loader detects the format based on file extension (.yaml/.yml for YAML, .json for JSON).

Yes of course! Noticed that I had poorly-worded my original question, so (basic) follow-up: these changes only support loading from a single config file, not multiple?

"Required" here means any ${VAR} reference without a default value (i.e. ${VAR} vs ${VAR:-default}). There's no fixed list — it's determined by what's in the user's config file. If a referenced env var is unset and has no default, a ConfigurationError is raised.

Got it, thanks! This is implementation of validating config file like this example against schema.

tammy-baylis-swi

Lgtm! I think the error raising behaviour makes sense with https://opentelemetry.io/docs/specs/otel/error-handling/#basic-error-handling-principles

MikeGoldsmith · 2026-03-09T10:00:15Z

Yes of course! Noticed that I had poorly-worded my original question, so (basic) follow-up: these changes only support loading from a single config file, not multiple?

Correct, only one configuration file can be used to configure the SDK - either using the env var or by importing and using the parse functions directly.

… into mike/otel-config/1-loading-validation

xrmx · 2026-03-10T14:45:51Z

opentelemetry-sdk/src/opentelemetry/sdk/_configuration/file/__init__.py

+    >>> from opentelemetry.sdk._configuration.file import load_config_file
+    >>> config = load_config_file("otel-config.yaml")
+    >>> print(config.file_format)
+    '1.0-rc.3'


Suggested change

'1.0-rc.3'

'1.0.0'

xrmx · 2026-03-10T14:51:22Z

opentelemetry-sdk/src/opentelemetry/sdk/_configuration/file/_env_substitution.py

+
+    # Pattern matches: ${VAR_NAME} or ${VAR_NAME:-default_value}
+    # Variable names must start with letter or underscore, followed by alphanumerics or underscores
+    pattern = r"\$\{([A-Za-z_][A-Za-z0-9_]*)(:-([^}]*))?\}"


Suggested change

pattern = r"\$\{([A-Za-z_][A-Za-z0-9_]*)(:-([^}]*))?\}"

pattern = r"[^\$]\$\{([A-Za-z_][A-Za-z0-9_]*)(:-([^}]*))?\}"

Will something like this avoid the need for replacing the $$?

xrmx · 2026-03-10T14:58:59Z

CHANGELOG.md


 ## Unreleased

+- `opentelemetry-sdk`: Add file configuration support with YAML/JSON loading, environment variable substitution, and schema validation against the vendored OTel config JSON schema


Please add the link to the PR on a newline

xrmx · 2026-03-10T14:59:51Z

opentelemetry-sdk/test-requirements.txt

 py-cpuinfo==9.0.0
 pytest==7.4.4
+jsonschema==4.23.0
+pyyaml==6.0.2


Suggested change

pyyaml==6.0.2

pyyaml==6.0.3

xrmx · 2026-03-10T15:00:45Z

opentelemetry-sdk/test-requirements.txt

 psutil==7.2.2
 py-cpuinfo==9.0.0
 pytest==7.4.4
+jsonschema==4.23.0


Suggested change

jsonschema==4.23.0

jsonschema==4.26.0

xrmx · 2026-03-10T15:03:14Z

opentelemetry-sdk/tests/_configuration/file/data/minimal_config.yaml

@@ -0,0 +1 @@
+file_format: "1.0-rc.3"


Can we bump to 1.0.0?

codeboten and others added 13 commits January 15, 2026 09:57

update tox command’s deps and allowlist

72c3729

add use-union-operator to datamodel-codegen and regenerate models file

a2dfa41

Merge branch 'main' of github.com:open-telemetry/opentelemetry-python…

659ab65

… into codeboten/generate-config-model-from-schema

add changelog

dd6a2cd

disable union-operator and set target python to 3.10

99e9570

Merge pull request open-telemetry#13 from honeycombio/codeboten/gener…

43f2c09

…ate-config-model-from-schema fix code-generation command and regenerate models

ignore generated file for linting

1a3bde8

fix TypeAlias import for python 3.9 in generated models file

6936e54

update uv.lock with new dev dependencies

7949792

Merge pull request open-telemetry#14 from honeycombio/codeboten/gener…

9f5f21e

…ate-config-model-from-schema.2 Fix lint errors and update uv.lock file

run precommit

81ac395

Signed-off-by: alex boten <223565+codeboten@users.noreply.github.com>

config: add yam/json file loading and env var substitution

2b25010

MikeGoldsmith mentioned this pull request Feb 11, 2026

Proposal: SDK Autoconfiguration #4856

Open

MikeGoldsmith added 7 commits February 11, 2026 09:57

Merge branch 'main' of github.com:open-telemetry/opentelemetry-python…

45bc753

… into codeboten/generate-config-model-from-schema

Merge branch 'codeboten/generate-config-model-from-schema' of github.…

65b2d16

…com:codeboten/opentelemetry-python into mike/otel-config/1-loading-validation

Fix pyright config structure in pyproject.toml

801a425

datamodel-codegen section was inserted between [tool.pyright] and its include/exclude config, causing pyright to check entire repo (599 files) instead of just included paths. Moved datamodel-codegen section after pyright config.

Merge branch 'codeboten/generate-config-model-from-schema' of github.…

e8180f2

…com:codeboten/opentelemetry-python into mike/otel-config/1-loading-validation

Fix typecheck and pylint errors

4c24e59

- Fix re.sub callback return type (must return str, not str | None) - Rename DOLLAR_PLACEHOLDER to dollar_placeholder (pylint naming) - Rename f to temp_file/config_file (pylint naming) - Update uv.lock

Add types-PyYAML and file-configuration extra to typecheck

dea74e9

Fixes yaml import warning by installing PyYAML type stubs and the file-configuration optional dependencies.

run precommit

df082ac

github-actions bot added the Stale label Mar 3, 2026

MikeGoldsmith added 2 commits March 3, 2026 13:03

Merge upstream/main into mike/otel-config/1-loading-validation

3b704c6

Assisted-by: Claude Sonnet 4.6

MikeGoldsmith force-pushed the mike/otel-config/1-loading-validation branch from b22fc34 to 9778ae5 Compare March 3, 2026 13:51

MikeGoldsmith added 2 commits March 3, 2026 13:55

config: fix pylint warnings in _loader.py

2a122da

- Replace global statement with list cache in _get_schema - Extract _validate_schema helper to reduce branches/statements in load_config_file Assisted-by: Claude Sonnet 4.6

fix: bump typing_extensions to 4.12.0 for Python 3.13+ compat

3998be7

jsonschema's `referencing` dep uses TypeVar defaults which requires typing_extensions>=4.12.0 on Python 3.13+ (4.10.0 raises AttributeError on TypeVar.__default__). Assisted-by: Claude Sonnet 4.6

MikeGoldsmith marked this pull request as ready for review March 3, 2026 14:18

MikeGoldsmith requested a review from a team as a code owner March 3, 2026 14:18

tammy-baylis-swi added this to Python PR digest Mar 3, 2026

tammy-baylis-swi moved this to Ready for review in Python PR digest Mar 3, 2026

tammy-baylis-swi removed the Stale label Mar 3, 2026

MikeGoldsmith mentioned this pull request Mar 5, 2026

fix(ci): intermittent Core Contrib Test failures due to git SHA propagation lag open-telemetry/opentelemetry-python-contrib#4304

Closed

merge upstream/main into mike/otel-config/1-loading-validation

09802fe

Assisted-by: Claude Sonnet 4.6

tammy-baylis-swi approved these changes Mar 6, 2026

View reviewed changes

Merge branch 'main' of github.com:open-telemetry/opentelemetry-python…

131632e

… into mike/otel-config/1-loading-validation

MikeGoldsmith moved this from Ready for review to Approved PRs in Python PR digest Mar 9, 2026

xrmx mentioned this pull request Mar 10, 2026

Add declarative configuration prototype #3631

Open

7 tasks

xrmx reviewed Mar 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

config: load and validate generated config models#4898

config: load and validate generated config models#4898
MikeGoldsmith wants to merge 26 commits intoopen-telemetry:mainfrom
honeycombio:mike/otel-config/1-loading-validation

MikeGoldsmith commented Feb 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 3, 2026

Uh oh!

tammy-baylis-swi commented Mar 4, 2026

Uh oh!

MikeGoldsmith commented Mar 5, 2026

Uh oh!

tammy-baylis-swi commented Mar 6, 2026

Uh oh!

tammy-baylis-swi left a comment

Uh oh!

MikeGoldsmith commented Mar 9, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

xrmx Mar 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	pattern = r"\$\{([A-Za-z_][A-Za-z0-9_])(:-([^}]))?\}"
	pattern = r"[^\$]\$\{([A-Za-z_][A-Za-z0-9_])(:-([^}]))?\}"


		## Unreleased

		- `opentelemetry-sdk`: Add file configuration support with YAML/JSON loading, environment variable substitution, and schema validation against the vendored OTel config JSON schema

Conversation

MikeGoldsmith commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Changes

How Has This Been Tested?

Does This PR Require a Contrib Repo Change?

Checklist:

Uh oh!

github-actions bot commented Mar 3, 2026

Uh oh!

tammy-baylis-swi commented Mar 4, 2026

Uh oh!

MikeGoldsmith commented Mar 5, 2026

Uh oh!

tammy-baylis-swi commented Mar 6, 2026

Uh oh!

tammy-baylis-swi left a comment

Choose a reason for hiding this comment

Uh oh!

MikeGoldsmith commented Mar 9, 2026

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

xrmx Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MikeGoldsmith commented Feb 6, 2026 •

edited

Loading