Repository navigation
fix(optimizer): keep slow-update and meta-skill content as text - #420
Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
Lanre Shittu (Shizoqua) wants to merge 1 commit into
Conversation
run_slow_update and run_meta_skill accepted any truthy content field and passed it through str(). When the optimizer returned the guidance as a JSON list of strings, the list's Python repr was written into the skill's slow-update region or the optimizer memory, for example "['Check units first.', 'Prefer exact spans.']". A dict or number was written the same way, and whitespace-only content was accepted as an empty update. A null reasoning field became the text "None". Add content_text(): a string is stripped, a list of strings is joined one item per line, and anything else is unusable. Both parsers use it for the content and the reasoning, and return None when the content is empty, as they already do for a parse failure.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #416.
When the optimizer returned slow-update guidance as a JSON list of strings, the list's Python repr, for example
['Check units first.', 'Prefer exact spans.'], was written into the skill's protected slow-update region. The optimizer meta-skill memory had the same problem. A list of strings is now joined one item per line; any other non-text value, or blank content, counts as no update.User impact: the slow-update region is part of the skill the target model reads, so it got Python syntax instead of guidance. Whitespace-only content was also accepted as an empty update, which wiped the existing guidance, and a
nullreasoning field was logged asNone.Proof: the regression tests fail on upstream
af8f9efand pass with this change.Root cause
run_slow_update()inskillopt/optimizer/slow_update.pyandrun_meta_skill()inskillopt/optimizer/meta_skill.pyaccepted any truthy content field and passed it throughstr(). The prompts ask for a string, but a list of bullet strings is a common model deviation, and nothing normalized it.Changes
content_text(value)inslow_update.py: a string is stripped; a list of strings is joined one item per line; anything else (None, a dict, a number, a mixed list) gives"".run_slow_update()andrun_meta_skill()use it for the content and the reasoning, and returnNonewhen the content is empty, as they already do for a parse failure.tests/test_slow_update_content.py, calling the real functions with a fake optimizer response and checking the guidance written into the skill.Reproduction
Before:
After:
Testing
python -m pytest tests/test_slow_update_content.py -q: on upstreamaf8f9ef, 8 failed and 1 passed; with the fix, 9 passed.python -m pytest tests/test_slow_update_content.py tests/test_slow_update_robustness.py tests/test_hierarchical_merge.py tests/test_skill_aware_reflection.py tests/test_trainer_resume.py -q: 26 passed.python -m pytest -q: 2296 passed, 12 skipped, 4 xfailed.python -m ruff checkandpython -m ruff format --checkon the changed files: clean.python -m mkdocs build --strict: passed.Risk and compatibility
String content is handled exactly as before. Only list, non-text and blank content change: lists of strings become readable text, and the rest now count as no update instead of writing a repr or an empty region. No backend, config key, file format or CLI change.