Skip to content

[tools] Update install.sh for Flink Agents 0.3.0 and Python 3.12#888

Open
Olawoyin007 wants to merge 3 commits into
apache:mainfrom
Olawoyin007:install-sh-0.3.0
Open

[tools] Update install.sh for Flink Agents 0.3.0 and Python 3.12#888
Olawoyin007 wants to merge 3 commits into
apache:mainfrom
Olawoyin007:install-sh-0.3.0

Conversation

@Olawoyin007

@Olawoyin007 Olawoyin007 commented Jul 9, 2026

Copy link
Copy Markdown

Linked issue: #863

Purpose of change

install.sh did not offer the 0.3.0 release (shipped June 19th), and it rejected Python 3.12 even though 0.3.0 supports it.

  • Add 0.3.0 to FLINK_AGENTS_SUPPORTED_VERSIONS and make it the recommended default, matching https://flink.apache.org/downloads/#apache-flink-agents (0.3.0 ships JARs for Flink 1.20 / 2.0 / 2.1 / 2.2, same matrix as 0.2.x).
  • Make the Python upper bound follow the selected release instead of a hardcoded <3.12: 0.1.x/0.2.x keep >=3.10,<3.12, while 0.3.0+ accepts Python 3.12 (its wheels publish requires-python >=3.10,<3.13). The version-mismatch messages now print the bound that actually applies.

The version choice happens in stage 1 (plan_flink_agents) before Python resolution in stage 2 (plan_pyflink), so FLINK_AGENTS_VERSION is always settled when validate_python_bin runs.

One known edge left as-is: on Python 3.12 with Flink 1.20/2.0, apache-flink has no cp312 wheel on PyPI, so pip falls back to the sdist. That combination fails at pip time with a clear resolver message rather than being pre-blocked by the installer.

Tests

  • Reworked the 3.12/3.13 cases in tools/test/unit/validate_python_bin.bats to cover both bounds (3.12 rejected for 0.2.x, accepted for 0.3.0+, 3.13 still rejected) and updated the default-version assertion in dry_run_extra.bats.
  • bash tools/test/run.sh: all 149 tests pass.
  • Manually verified on Linux with Python 3.12.3: accepted with the 0.3.0 default, correctly rejected with --flink-agents-version 0.2.1.

API

No public API changes; installer script only.

Documentation

  • doc-needed
  • doc-not-needed (quickstart docs already state "Python 3.10, 3.11 or 3.12")
  • doc-included

🤖 Generated with Claude Code

Add 0.3.0 to the supported versions (new recommended default) and make
the Python upper bound follow the selected release: 0.1.x/0.2.x keep
>=3.10,<3.12, while 0.3.0+ accepts Python 3.12 per its requires-python
metadata (>=3.10,<3.13).
@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs fixVersion/0.4.0 priority/major Default priority of the PR or issue. labels Jul 9, 2026
@Olawoyin007

Copy link
Copy Markdown
Author

The it-java [flink-2.1] failure looks like CI infrastructure rather than this change: the Ollama runner crashed mid-suite (OllamaException: llama-server process has terminated: signal: segmentation fault (core dumped)), and all three surefire retries of the 5 erroring tests hit the same dead server. This PR only touches tools/install.sh and its bats tests (covered by the separate install.sh tests jobs, green on both ubuntu and macos); the Java IT job doesn't execute install.sh. The same leg passes on recent main runs. A re-run should clear it - happy to rebase instead if you prefer.

@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs and removed doc-not-needed Your PR changes do not impact docs labels Jul 9, 2026
Comment thread tools/install.sh
py_minor="${py_version_output##*.}"

if [[ "$py_major" -ne 3 ]] || [[ "$py_minor" -lt 10 ]] || [[ "$py_minor" -ge 12 ]]; then
if [[ "$py_major" -ne 3 ]] || [[ "$py_minor" -lt 10 ]] || [[ "$py_minor" -ge "$(python_minor_ceiling)" ]]; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is where the interpreter check starts depending on the selected release — and once resolve_python has run, the confirm screen can still change that release out from under it. edit_plan_interactive's flink_agents_version) arm reassigns FLINK_AGENTS_VERSION (install.sh:1112-1114693) without re-checking the already-resolved PYTHON_BIN, unlike the enable_pyflink) arm right below it, which re-runs resolve_python when its dependent changes (install.sh:1153-1160). Concretely: a user on Python 3.12 accepts the new 0.3.0 default (validates here, ceiling 13), then at the confirm screen edits the version down to 0.2.1; setup_python_env then runs pip install flink-agents==0.2.1 (install.sh:1434), whose requires-python is <3.12, and the install fails after the plan looked good. Before this PR the interpreter was independent of the release, so this window didn't exist. Would re-validating PYTHON_BIN after a version edit — mirroring the enable_pyflink) arm — catch the mismatch at plan time rather than mid-install?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - fixed in c4f81b4. The flink_agents_version) arm now calls a new revalidate_python_for_agents_version after the version prompt: if PyFlink is enabled and the already-resolved PYTHON_BIN no longer satisfies the new release's ceiling, it warns and re-runs resolve_python (same semantics as the enable_pyflink) arm - and since PYTHON_BIN is part of the edit-state dump, the re-resolved interpreter propagates back to the plan). Your exact scenario (3.12 accepted under 0.3.0, then edited down to 0.2.1) is covered by a new bats test, plus four more for the keep/no-op paths.

🤖 Generated with Claude Code

@wenjin272

Copy link
Copy Markdown
Contributor

Minor test fixture drift: reset_install_sh_state in load.bash still resets the Flink Agents default/recommended/supported versions to the old 0.2.1 matrix, while install.sh now defaults to 0.3.0. Could we update the helper so sourced unit tests exercise the same defaults as the real installer?

The Python ceiling now depends on the selected release, so editing the
version at the confirm screen could invalidate an interpreter that
resolve_python had already accepted, failing later in setup_python_env.
Mirror the enable_pyflink edit arm: re-resolve at plan time.
reset_install_sh_state still carried the 0.2.1 version matrix, so
sourced unit tests exercised different defaults than the installer
ships. Align it with the 0.3.0 defaults.
@Olawoyin007

Copy link
Copy Markdown
Author

Thanks - synced in ece24ba: reset_install_sh_state now carries the 0.3.0 default/recommended/supported matrix, matching install.sh. The full suite still passes (154 tests, now including 5 for the confirm-screen re-validation weiqingy flagged above).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs fixVersion/0.4.0 priority/major Default priority of the PR or issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants