Add semantic-convention conformance test for the Flask instrumentation - #4966
Draft
ocelotl wants to merge 2 commits into
Draft
Add semantic-convention conformance test for the Flask instrumentation#4966ocelotl wants to merge 2 commits into
ocelotl wants to merge 2 commits into
Conversation
Check the Flask instrumentation against the HTTP semantic conventions using the semantic-conventions-conformance tooling, run through its pytest plugin so a convention violation is an ordinary test failure. The test is a normal pytest test collected by the Flask instrumentation's existing test suite: no dedicated tox environment and no separate workflow. It runs in the normal test matrix for the flask-3 environment. The tooling is not on PyPI, so it is installed from a pinned git ref alongside the other flask-3 test dependencies (that env's pytest is bumped to >= 8, which the tooling requires; the older flask envs are unchanged and do not collect it). The scenario exercises this repository's own working-tree Flask instrumentation, not a released pin. A conftest provisions the Weaver binary (pinned) and the pinned semantic-conventions registry on demand under plain pytest, and skips the test with a clear reason when they cannot be obtained (for example offline), so no special command or manual setup is needed. Reusable for the next instrumentation: a scenario directory with a conformance.yaml, the shared conftest, and the tooling added to that instrumentation's existing test-requirements. No per-instrumentation tox env.
Remove the Flask-specific reference in the conftest comment and note that this file is intended to be byte-for-byte identical across every instrumentation's tests/semconv_conformance/conftest.py. This lets the same provisioning harness be copied unchanged to the next instrumentation (for example requests) instead of each one carrying a divergent copy.
Pull request dashboard statusWaiting on the author · refreshed 2026-08-20 17:54 UTC Move out of draft to request review. Status above doesn't look right?
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a semantic-convention conformance test for the Flask instrumentation. It runs as an ordinary
pytesttest in the Flask instrumentation's existing test suite (no separate tox environment or workflow): the test drives the Flask instrumentation over a defined set of HTTP requests and checks that the emitted telemetry conforms to the HTTP semantic conventions. A convention violation is a normal test failure.The check reuses the tooling from
open-telemetry/semantic-conventions-conformance(the generic runner, the HTTP domain runner, the mock server, and the HTTP test client) via its pytest plugin, which collects theconformance.yamland turns each scenario into a pytest item. The telemetry is validated with Weaver against a pinned semantic-conventions registry.This is the first of a planned series: the intent is to add semantic-convention conformance tests to the Python instrumentations where this kind of testing applies, one at a time, with Flask first.
What it adds
instrumentation/opentelemetry-instrumentation-flask/tests/semconv_conformance/: a shared Flask app, a scenario that enablesFlaskInstrumentor, aconformance.yaml, and aconftest.pythat provisions the Weaver binary and the pinned registry on demand (and skips with a clear reason if they cannot be obtained, for example offline).test-requirements-3.txt(andpytest>=8, which the tooling requires). Older Flask test environments do not install the plugin and simply do not collect the file.Draft status and open points
This is a draft for discussion. Known points:
Feedback on placement and on the overall approach is welcome.