From 1775df7a74bb9651259623ce4cbcf410db6590d7 Mon Sep 17 00:00:00 2001 From: Louise Bowler <25640708+LouiseABowler@users.noreply.github.com> Date: Fri, 9 May 2025 11:28:52 +0100 Subject: [PATCH 1/2] Provide alternative line_profiler installation command for Zsh users --- episodes/profiling-lines.md | 10 ++++++++-- learners/setup.md | 12 ++++++++++-- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/episodes/profiling-lines.md b/episodes/profiling-lines.md index 2efaaf63..33dfb96e 100644 --- a/episodes/profiling-lines.md +++ b/episodes/profiling-lines.md @@ -49,9 +49,15 @@ pip install line_profiler[all] :::::::::::::::: spoiler -### Mac OS +### Installation issues -If you are unable to install `line_profiler` via `pip` on MacOS. Instead it can be installed via `conda`. +If you come across the error message `zsh: no matches found: line_profiler[all]`, try wrapping the package name in quotation marks: + +```sh +pip install 'line_profiler[all]' +``` + +Alternatively, the package can be installed via `conda`. ```sh conda install line_profiler diff --git a/learners/setup.md b/learners/setup.md index b320652b..dbfdc2f6 100644 --- a/learners/setup.md +++ b/learners/setup.md @@ -50,9 +50,17 @@ pip install shapely :::::::::::::::: spoiler -### Mac OS (line_profiler) +### Issues installing line_profiler -If you are unable to install `line_profiler` via `pip` on MacOS. Instead it can be installed via `conda`. +If you use Zsh as your shell (which is the default on Mac OS), you may come across the error `zsh: no matches found: line_profiler[all]` when installing `line_profiler[all]`. +In Zsh, we need to ensure that the square brackets are treated as standard characters; wrapping them in quotation marks resolves the issue. + +```sh +pip install 'line_profiler[all]' +pip install pytest snakeviz numpy pandas matplotlib +``` + +Alternatively, you can install `line_profiler` via `conda`. ```sh conda install -c conda-forge line_profiler From def8d0c95d3dcba121714d9aac8e3ac7d213a27a Mon Sep 17 00:00:00 2001 From: Louise Bowler <25640708+LouiseABowler@users.noreply.github.com> Date: Fri, 9 May 2025 12:31:40 +0100 Subject: [PATCH 2/2] Introduce line_profiler with new method of running the tool --- episodes/profiling-lines.md | 65 +++++++++++++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 7 deletions(-) diff --git a/episodes/profiling-lines.md b/episodes/profiling-lines.md index 33dfb96e..03c981b8 100644 --- a/episodes/profiling-lines.md +++ b/episodes/profiling-lines.md @@ -106,16 +106,47 @@ print(is_prime(1087)) This tells `line_profiler` to collect metrics for the lines within the method `is_prime()`. You can still execute your code as normal, and these changes will have no effect. -Similar to the earlier tools, `line_profiler` can then be triggered via `kernprof`. +The use of `line_profiler` is triggered by an environment variable. +The command below sets the `LINE_PROFILE` variable to `1` immediately before the script is run as usual, and unsets the variable afterwards. ```sh -python -m kernprof -lvr my_script.py +LINE_PROFILE=1 python my_script.py ``` +This will print the following output to console: + +```output +True +Timer unit: 1e-09 s + + 0.00 seconds - /Users/k2472505/Projects/2025-02-19-p-o-python-jcmb/myscript.py:3 - is_prime +Wrote profile results to profile_output.txt +Wrote profile results to profile_output_2025-05-09T113346.txt +Wrote profile results to profile_output.lprof +To view details run: +python -m line_profiler -rtmz profile_output.lprof +``` + +The profiler has written three output files, all with the same content: + +- `profile_output.txt` +- `profile_output_2025-05-09T113346.txt` +- `profile_output.lprof` +The first two are text files, which can be opened in your preferred text editor or terminal. +The second includes a timestamp in the filename to prevent it being overwritten by subsequent uses of the profiler. +The third file contains the same information as the other two, but uses `lprof` format. +`lprof` files are not human-readable, but provide additional helpful formatting when the file is opened with `line_profiler`: + +```sh +python -m line_profiler -rtmz profile_output.lprof +``` + +where `-rtmz` are flags for rich formatting, sorting by ascending total time, printing a summary of total function time, and hiding functions which were not called. +These options, and more, can be listed with `python -m line_profiler --help`. + This will output a table per profiled method to console: ```output -Wrote profile results to my_script.py.lprof Timer unit: 1e-06 s Total time: 1.65e-05 s @@ -148,13 +179,33 @@ The columns have the following definitions: As `line_profiler` must be attached to specific methods and cannot attach to a full Python file or project, if your Python file has significant code in the global scope it will be necessary to move it into a new method which can then instead be called from global scope. -The profile is also output to file, in this case `my_script.py.lprof`. -This file is not human-readable, but can be printed to console by passing it to `line_profiler`, which will then display the same table as above. +To run the code again without the profiler enabled, simply leave out the `LINE_PROFILE` variable from the command: + +```sh +python my_script.py +``` + +:::::::::::::::: spoiler + +### Earlier versions of line_profiler + +If you have an older version of `line_profiler`, the option of running the profiler with `LINE_PROFILE` may not be available to you. +This was introduced in version 4.1.0 (August 2023). + +You can check your current version of `line_profiler` using ```sh -python -m line_profiler -rm my_script.py.lprof +pip list | grep profiler ``` - + + +and upgrade to the latest available with + +```sh +pip install --upgrade line_profiler +``` + +:::::::::::::::::::::::: ## Worked Example