Skip to content

Python: Fix repetition_penalty and length_penalty ranges in OnnxGenAIPromptExecutionSettings - #14738

Open
Qiming Bao (14H034160212) wants to merge 1 commit into
microsoft:mainfrom
14H034160212:fix-onnx-penalty-ranges
Open

Qiming Bao (14H034160212) wants to merge 1 commit into
microsoft:mainfrom
14H034160212:fix-onnx-penalty-ranges

Conversation

@14H034160212

Copy link
Copy Markdown

Motivation and Context

Part of #14735

OnnxGenAIPromptExecutionSettings limited repetition_penalty and length_penalty to [0, 1]. In onnxruntime-genai:

  • repetition_penalty defaults to 1.0, which means no penalty (config.h). It is applied as score < 0 ? score * penalty : score / penalty to tokens already generated (sampling_distribution.h). Values above 1.0 discourage repetition and values below 1.0 encourage it. The old validator rejected every value that reduces repetition, such as 1.1, and accepted the values that make it worse.
  • length_penalty is the exponent in the beam-search length normalization, sum_logprobs / pow(length, length_penalty) (beam_search_scorer.cpp). config.h documents "> 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences", so values above 1 and below 0 are both valid.

onnxruntime-genai does not range-check either value (it checks only top_p/top_k, in generators.cpp). The same definitions are in v0.18.0, the version the onnx extra pins.

Description

  • repetition_penalty: Field(ge=0.0, le=1.0) changed to Field(gt=0.0). 0 would divide by zero and negative values flip the sign of the logits, so both stay rejected.
  • length_penalty: the range constraint is removed (float | None).
  • Defaults stay None, so nothing is sent to onnxruntime-genai unless the user sets a value. diversity_penalty (unused in onnxruntime-genai) is unchanged.
  • Tests in tests/unit/connectors/ai/onnx/test_onnx_prompt_execution_settings.py:
    • repetition_penalty 0.5, 1.0, 1.1, 1.5 and 2.0 are accepted and reach prepare_settings_dict().
    • repetition_penalty 0 and -1 are rejected.
    • length_penalty -1.0, -0.5, 0.0, 1.0, 1.5 and 2.0 are accepted.

Verification (from python/):

uv run --no-sync pytest tests/unit/connectors/ai/onnx            # 47 passed (8 of the new cases fail on main)
uv run --no-sync ruff check <changed files> && uv run --no-sync ruff format --check <changed files>
uv run --no-sync mypy -p semantic_kernel.connectors.ai.onnx --config-file mypy.ini   # Success: no issues found
uv run --no-sync pre-commit run -c python/.pre-commit-config.yaml --files <changed files>  # all hooks passed

Contribution Checklist

…OnnxGenAIPromptExecutionSettings

Both fields were limited to [0, 1]. In onnxruntime-genai repetition_penalty
1.0 means no penalty and values above 1.0 penalize repeated tokens (positive
logits are divided by it, negative ones multiplied), so the useful values
were rejected while values below 1.0, which encourage repetition, were
accepted. length_penalty is the exponent of the beam-search length
normalization and may be any float.

Require repetition_penalty > 0 and leave length_penalty unconstrained,
matching onnxruntime-genai, which does not validate either value.
Copilot AI balanced review requested due to automatic review settings November 7, 2026 04:32
@14H034160212
Qiming Bao (14H034160212) requested a review from a team as a code owner November 7, 2026 04:32
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Nov 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the ONNX penalty-validation changes.

No open issues 🎉

What changed in this PR

This PR addresses the penalty-validation portion of #14735, aligning the Python ONNX connector’s execution settings with onnxruntime-genai while preserving existing defaults.

Changes:

  • Allows any positive repetition_penalty, including values above 1 that discourage repetition.
  • Removes range constraints from length_penalty.
  • Adds parameterized tests for accepted values, rejection of non-positive repetition penalties, and serialization.
File Description
python/​tests/​unit/​connectors/​ai/​onnx/​test_onnx_prompt_execution_settings.py Adds regression tests for penalty validation and serialization.
python/​semantic_kernel/​connectors/​ai/​onnx/​onnx_gen_ai_prompt_execution_settings.py Corrects penalty constraints while retaining None defaults.

Review effort: Balanced

This branch was previously deployed

1 inactive deployment
github-app-auth — 15454a5b Deployed Nov 7, 2026 by 14H034160212 via add_label #29618
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants