Skip to content

fix(evoprompt): isolate registry state during concurrent evaluation - #253

Open
bitkira wants to merge 1 commit into
ANative-Lab:mainfrom
bitkira:fix/evoprompt-state-isolation
Open

fix(evoprompt): isolate registry state during concurrent evaluation#253
bitkira wants to merge 1 commit into
ANative-Lab:mainfrom
bitkira:fix/evoprompt-state-isolation

Conversation

@bitkira

@bitkira bitkira commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Motivation

Fixes #252.

EvoPrompt temporarily applies a candidate prompt configuration to the shared ParamRegistry before evaluating the program, then restores the previous configuration afterwards.

Because evaluations are run concurrently and ParamRegistry mutates the live program state, concurrent evaluation tasks can interleave. This can make one prompt combination get scored under another registry state, and can also leave the program in a temporary configuration when restoration interleaves with another task or when evaluation raises.

Summary

  • Isolate temporary EvoPrompt registry mutation with a per-optimizer async lock.
  • Apply a combination once while evaluating its examples, then restore the original config in a finally block.
  • Keep example-level execution concurrent for the active combination.
  • Separate current-config example evaluation from registry mutation.
  • Defer type-only/heavy imports used by optimizer modules so targeted EvoPrompt tests do not need unrelated optional dependencies at import time.
  • Add deterministic async tests for registry state isolation and exception recovery.

Tests

.venv/bin/python -m pytest tests/src/optimizers/test_evoprompt_state_isolation.py tests/src/optimizers/test_optimizer_package_imports.py -q
4 passed
.venv/bin/python -m ruff check evoagentx/optimizers/evoprompt_optimizer.py evoagentx/optimizers/engine/base.py evoagentx/optimizers/engine/decorators.py tests/src/optimizers/test_evoprompt_state_isolation.py
All checks passed!

Notes

This serializes temporary config mutation across different prompt combinations. Example-level program execution for the active combination can still run concurrently. This prioritizes correctness of EvoPrompt fitness scores over unsafe cross-combination mutation of shared program state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] EvoPrompt concurrent evaluation can contaminate registry state

1 participant