Skip to content
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -427,6 +427,30 @@ Refer to our [comprehensive guide](docs/configuration-options.md) to learn about
Often you want to have benchmarks in the same project, but separated from main code, much like tests.
Refer to our [detailed documentation](docs/separate-benchmark-source-set.md) on configuring your project to set up a separate source set for benchmarks.

## Analyzing results

Upon finishing benchmarks execution, the plugin prints out a summary table with results.
In addition to that table, it also saves results into a file. By default, that would be a JSON file with a format
compatible with the format used by JMH. The format is configured using `reportFormat` option, you can read more about
it in the [configuration documentation](docs/configuration-options.md).

While runs consisting of a few benchmarks could be analyzed without any additional tools,
analyzing run results consisting of dozens of benchmarks could quickly become tedious.

To facilitate this process, you can use [Kotlin Notebooks](https://kotlinlang.org/docs/kotlin-notebook-overview.html),
which provides means for manipulating, analyzing and visualizing data.

We prepared several notebooks demonstrating different analysis techniques using our [example](#Examples) projects.
Namely,
- [Performing single run analysis](examples/simple-benchmark-analysis.ipynb)
![SingleRunAnalysis](docs/notebooks-single-run-demo.png)
- [Comparing different runs with each other](examples/compare-benchmark-runs.ipynb)
![MultiRunAnalysis](docs/notebooks-multi-run-demo.png)
- [Comparing benchmark functions from the same run with each other](examples/compare-hypothesis.ipynb)
![HypothesisAnalysis](docs/notebooks-compare-hypothesis-demo.png)

You can always copy these notebooks into your projects and adapt for your particular needs.

## Examples

To help you better understand how to use the `kotlinx-benchmark` library, we've provided an [examples](examples) subproject.
Expand Down
1 change: 1 addition & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ apiValidation {
"java",
"kotlin-jvm-separate-benchmark-source-set",
"kotlin-jvm",
"kotlin-jvm-compare-hypothesis",
"kotlin-multiplatform",
"integration",
)
Expand Down
Binary file added docs/notebooks-compare-hypothesis-demo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/notebooks-multi-run-demo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/notebooks-single-run-demo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,518 changes: 795 additions & 723 deletions examples/compare-benchmark-runs.ipynb

Large diffs are not rendered by default.

Loading