llama-server prompt cache support

[0x0539]

Does llama-server support prompt caching between requests, similar to llama-cli prompt file? I have a use case where the prefix stays the same between requests, with just a few characters changing at the end.

[0x0539]

Oh wait maybe that's on by default:

        params.cache_prompt     = json_value(data, "cache_prompt",       true);

[mashdragon]

Yes, it's on by default. If you submit multiple requests sequentially with the same prefix, the prompt decoding for that common prefix will be reused for subsequent requests.

[YannFollet]

is it easy to implement multi cache support, if we have calls from different agents, so instead of cache_prompt true, we could have a number that is the maximum cache prompt
could have a permanant caches prompt (can load them when starting the server)
and some caches for multiple requests sequentially