Provide clear instructions for running pre-check locally

[dave2wave]

I made changes to pre-commit and ran into a few issues.

There must be instructions on how to install and run pre-checks / linters locally.

(a) the linters only run with PRs so prior to a commit we need to run.
(b) need to test changes.

[jbampton]

I wrote these docs for Apache CloudStack:

https://github.com/apache/cloudstack/blob/main/PRE_COMMIT.md

I can expand on this

[jbampton]

We could explain the Makefile approach and use some of what I wrote previously

[dave2wave]

Yes. These steps work.

1. Setup and activate venv
2. Add pre-commit to requirements.txt
3. make install
4. make checkinstall
5. make check

I like this better than explaining git commit. I actually use GH Desktop to do most git commands.

[potiuk]

One comment here: while I understand muscle memory for make - it's a bit alien in Python's word. Also requirements.txt is already largely abandoned and everyone but some really old-style projects that are not really updated are following pyproject.toml project definition, where prek can be added to dev dependency groups - and installed with uv sync automatically without thinking of dev-specific dependencies.

I would strongly recommend (and @sbp I think is using it) to use prek instead of pre-commit. It's a fully compatible with pre-commit rust-written replacement, reimagined for speed, dev UX and functionality.

It has built-in uv integration, has a great auto-completion support and is generally very useful as the driver for your checks - way more usable than any make approach can get. With auto-complete, nice colorful output that drags your attention to important things it's as good as it can get.

• uv tool install prek -> to have prek installed and available on command line
• prek install -> installs prek to run automatically with git commit (or push or whatever options of install you choose)
• prek -> runs check on currently staged changes
• prek --last-commit -> will run pre-commit checks on last commit only
• prek auto-update -> will update all hooks to latest versions
• prek run --from-ref main -> will run prek checks for all changes since you branched off
• prek run --all-files -> runs checks on all files

All this is heavily optimized for performance - prek's real power is the "git" integration - i.e. it will figure the minimum set of hooks to be run for the change in the currently staged changes, or the commit (or range of commits) that you specify. While on CI it runs with --all-files to add a safety net in case modified files did not trigger the right hooks. And it does it heavily in-parallel if possible, usually using all available cores and splitting the file lists into chunks where checks are run in parallel. This way git commit in even medium size projects takes as little time as possible time when prek install has been run before and it has the benefit that you don't even have to think of running anything. Which I find extremaly good for incrementally fixing issues when you incrementally commit your changes.

Also uv frees you from installing and activating venv at all. It uses it under the hood, but whenever you run uv run it will automatically sync your repo with pyproject.toml, insalls upgraded code and regenerates .lock file so that development environment can be reproducible between different people. Nobody even has to think (or even need to know) about venv, activation, installing dependencies, different requirements.txt (dev/prod) etc. etc. This is a thing of a past with modern tooling. Generally you should be able to do:

• git clone project
• cd project
• uv run pytest

And the venv should be created for you, python should be installed, all necessary packages (including development packages) will be installed in exactly the same version like last version the last person who committed the code used it, activate the environment and run pytest for all tests of yours in that environment. And it will continue working like that - i.e. if tomorrow you pull changes that other developer added with new requirements and dependencies, you have to do NOTHING.

You just do this:

• uv run pytest

And new dependencies will be installed, outdated dependencies will be updated, venv will be reinstalled (this usually takes milliseconds) and your tests will be run in that new environment.

I think - if you want to document setting up an environment, my recommendation is to teach people modern ways of setting up the environment and running it that takes seconds, rather than quite "dated" things how python (and non-python) projects were managed literally years ago.

My 3c.

[potiuk]

BTW. UV is just a tool - all the rest (except pre-commit.config) things that it uses is something that has been standardized during the last 3-4 years (most of it completed some 2 years ago as standards) in PEP packaging world - there are also other tools that use those standards (for example hatch - it will use pyproject.toml, use dependency groups, etc. and those tools are largely catching up with what uv heavily "simplifying" dev-env approach does, so what you learn and use here will be the absolute default of pretty much all the various tools that people will use soon.

Which will also mean that you won't have to force people to use uv - they can use hatch or pip all along or any other tools that will appear there that will follow the standards. So it's really quite different world than it was even 3 years ago where by choosing a tool (essentialy build frontend), you also had to commit to it and make everyone uses it - because all those things are already well established standards.

[potiuk]

(BTW. Nice thing about uv that it will even install python for you automatically when you run uv run so you do not have to list python as prerequisite). This is really a game-changer when it comes to how "easy" it is to start contributing to a Python project.

[dave2wave]

Since this repository is for documentation and builds the tooling.apache.org website with ASF Pelican I'm ok with a simple make approach for now.

[dave2wave]

#74 switches to uv