Obtain JSR223 usage report after test execution

[KingRabbid]

JSR223 compiled script can be cached; size of cache can be configured. What lacks now is the view of actual cache usage after test run.

Description

Allow configuration of the Caffeine cache API in detail, as per supported parameters of the API; one of the parameters allow you to specify a cache usage report should be generated. This will be done at the end of the test run.
Used cache size will always be printed.

Motivation and Context

One could configure the JSR223 cache usage but could not see how much of the cache is actually used, if sizing is appropriate as per JSR223 elements used in a JMX, if code can actually be cached, etc. This is especially useful in load and performance testing to reduce computing time on client side.

Sample output when report is disabled (only used cache size is printed):

2025-10-06 14:32:13,887 INFO o.a.j.u.JSR223TestElement: Compiled cache size: 3, stats: CacheStats{hitCount=0, missCount=0, loadSuccessCount=0, loadFailureCount=0, totalLoadTime=0, evictionCount=0, evictionWeight=0}

Sample output when report is enabled (used cache size and details are printed):

2025-10-06 14:45:52,434 INFO o.a.j.u.JSR223TestElement: Compiled cache size: 3, stats: CacheStats{hitCount=2, missCount=3, loadSuccessCount=3, loadFailureCount=0, totalLoadTime=200523600, evictionCount=0, evictionWeight=0}

How Has This Been Tested?

Simple runs of JSR223 script elements with checked or unchecked 'Cache compile script if available' and monitor the report; parameter left as the current pushed default in jmeter.properties or with recordStats property removed.

Screenshots (if appropriate):

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• I have updated the documentation accordingly.

[vlsi]

This looks interesting, however, I'm not sure it solves true issues though.
If you want improve the caching, then we should probably forbid ${...} within script code.

[vlsi]

Can you refer the documentation or copy the relevant bits here?

[KingRabbid]

Hi,

some details in the Javadoc: https://www.javadoc.io/static/com.github.ben-manes.caffeine/caffeine/3.2.2/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/Caffeine.html

[KingRabbid]

Spec/configurable attributes description: https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/latest/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/CaffeineSpec.html and logic behind counters: https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/latest/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/stats/CacheStats.html

[vlsi]

I mean it would be worth adding the links to JMeter documentation

[KingRabbid]

I've pushed new code, not sure it addresses your suggestion with regards to documentation, please let me know where to edit.

[vlsi]

Please add braces. We do use them, however, the check was missing: #6516

[vlsi]

I'm not sure the stats are actionable. The stats do not seem to help users to identify the offending elements.

What are the units for "Compiled cache size"?

[KingRabbid]

Hi,

I've patched the code; the report only is the starting point of investigation, something which isn't available now? I'll come back in the following days with some sample JMX to help demo the outcome.
Indeed, doesn't point to exact "missing" JSR223 element, but still it's a starting point.
Disabling ${} is not necessary, maybe a warning is enough? Anyway, this matters only if the variable is stored in vars, if defined locally in the JSR223 then it works fine, i.e. is cached afaik.

Thanks

[KingRabbid]

I've pushed more changes; now I've implemented the logic to print, at debug level, JSR223 elements that don't get into cache: unchecked flag, compile problem.
Also, I've extended the output of the stats to include all relevant KPIs currently available in the API:
2025-10-23 00:40:00,693 INFO o.a.j.u.JSR223TestElement: JSR223 cached scripts: 2, requestsCount: 13 (hitCount: 4 + missedCount: 9), (hitRate: 0.31, missRate: 0.69), loadCount: 9 (loadSuccessCount: 2 + loadFailureCount: 7), evictionCount: 0, evictionWeight: 0, totalLoadTime: 1471.00 ms, averageLoadPenalty: 163.44 ms

[KingRabbid]

Attaching sample JMX used for trials:
CCH-trials.zip
I've also updated the Caffeine library to v.3.1.8, latest 3.2.0 seems to bring incompatibilities or more changes because of dependencies.

[KingRabbid]

As per latest push, below a sample output report. Top and whether to generate the report is fully configurable via parameters:

#jsr223.compiled_scripts_cache_size=100
# Configurable options on the compiled scripts cache, overrides the jsr223.compiled_scripts_cache_size property
# See com.github.benmanes.caffeine.cache.Caffeine API for details
jsr223.compiled_scripts_cache_spec=maximumSize=100,recordStats
# How many entries to show in the JSR223 cache report (if enabled, i.e. > 0)
#jsr223.statsReportsTop=5

Sample report displayed at the end of the run when log level is INFO:

2026-09-03 16:08:09,985 INFO o.a.j.u.JSR223TestElement: JSR223 cache stats => scripts: 2, requestsCount: 121 (hitCount: 58 + missedCount: 63), (hitRate: 0.48, missRate: 0.52), loadCount: 63 (loadSuccessCount: 2 + loadFailureCount: 61), evictionCount: 0, evictionWeight: 0, totalLoadTime: 19.38 ms, averageLoadPenalty: 0.31 ms

2026-09-03 16:08:09,988 INFO o.a.j.u.JSR223TestElement: Heaviest top 5 contributors by their total 'computeScriptCacheKey' execution times (ms)

===> By total duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler - Editor                                                30           4.026        0.134
2     JSR223 Sampler                                                          1           0.055        0.055
3     JSR223 Timer                                                           30           0.051        0.002
4     JSR223 Assertion                                                       30           0.017        0.001
5     JSR223 Sampler - Color listing                                         30           0.008        0.000
      Total:                                                                121           4.157        0.034

===> By average duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler - Editor                                                30           4.026        0.134
2     JSR223 Sampler                                                          1           0.055        0.055
3     JSR223 Timer                                                           30           0.051        0.002
4     JSR223 Assertion                                                       30           0.017        0.001
5     JSR223 Sampler - Color listing                                         30           0.008        0.000
      Total:                                                                121           4.157        0.034

2026-09-03 16:08:09,988 INFO o.a.j.u.JSR223TestElement: Heaviest top 3 contributors by their total 'getCompiledScript' execution times (ms)

===> By total duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler - Editor                                                30          19.882        0.663
2     JSR223 Timer                                                           30           5.735        0.191
3     JSR223 Sampler                                                          1           2.519        2.519
      Total:                                                                 61          28.136        0.461

===> By average duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler                                                          1           2.519        2.519
2     JSR223 Sampler - Editor                                                30          19.882        0.663
3     JSR223 Timer                                                           30           5.735        0.191
      Total:                                                                 61          28.136        0.461

2026-09-03 16:08:09,988 INFO o.a.j.u.JSR223TestElement: Heaviest top 5 contributors by their total 'processFileOrScript' execution times (ms)

===> By total duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler - Color listing                                         30         413.583       13.786
2     JSR223 Assertion                                                       30         165.279        5.509
3     JSR223 Sampler - Editor                                                30          71.982        2.399
4     JSR223 Timer                                                           30          17.303        0.577
5     JSR223 Sampler                                                          1           2.981        2.981
      Total:                                                                121         671.127        5.547

===> By average duration (ms) <===
Rank  Element name                                                        Calls    Duration(ms)      Avg(ms)
1     JSR223 Sampler - Color listing                                         30         413.583       13.786
2     JSR223 Assertion                                                       30         165.279        5.509
3     JSR223 Sampler                                                          1           2.981        2.981
4     JSR223 Sampler - Editor                                                30          71.982        2.399
5     JSR223 Timer                                                           30          17.303        0.577
      Total:                                                                121         671.127        5.547