feat: disk cache for profile/config LLM generation (closes #20)#32
Merged
Conversation
Re-running a campaign on the same entities re-issues identical, expensive profile- and config-generation LLM calls. Add a content-addressed disk cache so identical requests skip the model. - app/utils/llm_cache.py: LLMResponseCache — namespaced, keyed by a SHA-256 of the request material (model + system + prompt), one JSON file per entry under <cache_dir>/<namespace>/. Best-effort I/O (any error degrades to a miss), optional TTL, and an enable flag. - Config: LLM_CACHE_ENABLED (default on), LLM_CACHE_DIR, LLM_CACHE_TTL. - Wired into the two generation choke points: oasis _generate_profile_with_llm and simulation_config _call_llm_with_retry check the cache before calling and store on success (genuine LLM results only — never rule-based fallbacks). Keying on model+prompt means a prompt or model change invalidates naturally. - Documented env vars in .env.example. - tests/test_llm_cache.py: hit/miss, order-independent keys, disabled no-op, TTL expiry, namespace isolation, corrupt-entry-as-miss. Stacked on the #15 branch (shared config.py + generator changes). Co-Authored-By: PRATHAMESH75 <prathamesh290504@gmail.com>
This was referenced Jun 20, 2026
PRATHAMESH75
changed the base branch from
feat/15-per-task-model-fallback
to
main
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #20. Adds a content-addressed disk cache so re-running a campaign on the same entities skips redundant, expensive profile/config generation calls.
Changes
app/utils/llm_cache.py—LLMResponseCache: namespaced, keyed by SHA-256 of the request material (model + system + prompt), one JSON file per entry under<cache_dir>/<namespace>/. Best-effort I/O (any error → cache miss), optional TTL, enable flag, atomic writes viaos.replace.config.py—LLM_CACHE_ENABLED(default on),LLM_CACHE_DIR(defaultbackend/uploads/llm_cache),LLM_CACHE_TTL(0 = no expiry).oasis_profile_generator._generate_profile_with_llmandsimulation_config_generator._call_llm_with_retry. Only genuine LLM results are cached (never rule-based fallbacks); keying on model+prompt means prompt/model changes invalidate naturally..env.example— documented the new vars.Testing
tests/test_llm_cache.py(9 tests): set/get round-trip, distinct-material isolation, order-independent keys, disabled no-op, TTL=0 never-expires, TTL expiry, namespace isolation, corrupt-entry-as-miss, hash stability.Notes
LLM_CACHE_ENABLED=false.🤖 Generated with Claude Code