Migrate exasol provider to pyexasol 2.x and remove the <2 cap - #72119
Open
joseph-bergin wants to merge 1 commit into
Open
Migrate exasol provider to pyexasol 2.x and remove the <2 cap#72119joseph-bergin wants to merge 1 commit into
joseph-bergin wants to merge 1 commit into
Conversation
pyexasol 2.x ships type information, which surfaced six mypy errors in the exasol hook and forced the `pyexasol>=0.26.0,<2` cap added in apache#68933. Adapt the hook to the driver's real contract: - `get_conn` reused one local for both the Airflow `Connection` and the `ExaConnection` returned by `pyexasol.connect`. Use separate names. - `ExaConnection.execute`/`export_to_pandas` take `query_params: dict | None` and render named placeholders via `format(query, **query_params)`, so a positional sequence has always raised `TypeError: argument after ** must be a mapping`. Normalize mappings to `dict` and reject sequences up front with a message that names the fix. - `execute` runs exactly one statement, so the `str | list[str]` that `DbApiHook` allows was never executable here. Reject lists and point at `run()`, which does handle them. Both rejected paths were already broken against a real Exasol; only the error message changes. `test_run_with_parameters` passed a tuple and passed only because the connection is mocked, so it now uses a mapping, with new coverage for mapping normalization and for both rejections. Closes: apache#69123 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrates the exasol provider to pyexasol 2.x and removes the
pyexasol>=0.26.0,<2cap added in #68933.What the cap was hiding
pyexasol 2.x ships type information, so mypy now checks these calls. Against pyexasol 2.3.2 the hook reports 6 errors:
ExaConnection.execute/export_to_pandasare typedquery_params: dict | None, but the hook forwardedIterable | Mapping[str, Any] | None(lines 148, 191, 208, 337).executeis typedquery: str, butget_records/get_firstforwarded thestr | list[str]thatDbApiHookaccepts (lines 191, 208).get_connalso reused one local for both the AirflowConnectionand theExaConnectionreturned bypyexasol.connect; that one only surfaces with Airflow's own sources onmypy_path, and is fixed here too.Changes
get_connuses separate locals (airflow_connand the returned connection)._to_query_paramspassesNonethrough, copies any mapping into adict, and rejects sequences._to_single_statementunwraps thestrand rejects a list.providers/exasol/pyproject.toml.On the two rejected paths
Both were already broken against a real Exasol, so only the error message changes:
ExaStatement._format_querydoesself.connection.format.format(query, **query_params). A tuple or list there raisesTypeError: argument after ** must be a mapping, not tuple, so positional parameters have never reached the driver intact.ExaConnection.executeruns exactly one statement, so alist[str]was never executable inget_records/get_first.run()is the method that splits and loops, and the new message points there.test_run_with_parameterspassed("param1", "param2")and passed only because the connection is aMagicMock. It now uses a mapping, with new coverage for mapping normalization and for both rejections.If you would rather keep those paths permissive and satisfy mypy with
castinstead, I am happy to switch — I went with explicit errors because the status quo is a confusing failure deep inside the driver.Verification
Against pyexasol 2.3.2, Python 3.11:
mypy providers/exasol/src/— 6 errors before, 0 afterpytest providers/exasol/tests/unit/exasol/— 58 passed (previously 53 passed with 1 failing after the migration)ruff checkandruff format --checkat the pinned 0.16.3 — cleancloses: #69123
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines