Repository navigation
Python: Fix repetition_penalty and length_penalty ranges in OnnxGenAIPromptExecutionSettings - #14738
Open
Qiming Bao (14H034160212) wants to merge 1 commit into
Open
Qiming Bao (14H034160212) wants to merge 1 commit into
Qiming Bao (14H034160212) wants to merge 1 commit into
Conversation
…OnnxGenAIPromptExecutionSettings Both fields were limited to [0, 1]. In onnxruntime-genai repetition_penalty 1.0 means no penalty and values above 1.0 penalize repeated tokens (positive logits are divided by it, negative ones multiplied), so the useful values were rejected while values below 1.0, which encourage repetition, were accepted. length_penalty is the exponent of the beam-search length normalization and may be any float. Require repetition_penalty > 0 and leave length_penalty unconstrained, matching onnxruntime-genai, which does not validate either value.
Qiming Bao (14H034160212)
temporarily deployed
to
github-app-auth
November 7, 2026 04:32 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the ONNX penalty-validation changes.
No open issues 🎉
What changed in this PR
This PR addresses the penalty-validation portion of #14735, aligning the Python ONNX connector’s execution settings with onnxruntime-genai while preserving existing defaults.
Changes:
- Allows any positive
repetition_penalty, including values above 1 that discourage repetition. - Removes range constraints from
length_penalty. - Adds parameterized tests for accepted values, rejection of non-positive repetition penalties, and serialization.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/onnx/test_onnx_prompt_execution_settings.py |
Adds regression tests for penalty validation and serialization. |
python/semantic_kernel/connectors/ai/onnx/onnx_gen_ai_prompt_execution_settings.py |
Corrects penalty constraints while retaining None defaults. |
Review effort: Balanced
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Part of #14735
OnnxGenAIPromptExecutionSettingslimitedrepetition_penaltyandlength_penaltyto[0, 1]. In onnxruntime-genai:repetition_penaltydefaults to 1.0, which means no penalty (config.h). It is applied asscore < 0 ? score * penalty : score / penaltyto tokens already generated (sampling_distribution.h). Values above 1.0 discourage repetition and values below 1.0 encourage it. The old validator rejected every value that reduces repetition, such as 1.1, and accepted the values that make it worse.length_penaltyis the exponent in the beam-search length normalization,sum_logprobs / pow(length, length_penalty)(beam_search_scorer.cpp).config.hdocuments "> 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences", so values above 1 and below 0 are both valid.onnxruntime-genai does not range-check either value (it checks only
top_p/top_k, ingenerators.cpp). The same definitions are in v0.18.0, the version theonnxextra pins.Description
repetition_penalty:Field(ge=0.0, le=1.0)changed toField(gt=0.0). 0 would divide by zero and negative values flip the sign of the logits, so both stay rejected.length_penalty: the range constraint is removed (float | None).None, so nothing is sent to onnxruntime-genai unless the user sets a value.diversity_penalty(unused in onnxruntime-genai) is unchanged.tests/unit/connectors/ai/onnx/test_onnx_prompt_execution_settings.py:repetition_penalty0.5, 1.0, 1.1, 1.5 and 2.0 are accepted and reachprepare_settings_dict().repetition_penalty0 and -1 are rejected.length_penalty-1.0, -0.5, 0.0, 1.0, 1.5 and 2.0 are accepted.Verification (from
python/):Contribution Checklist