Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: CI

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
# Lint only the files changed in the PR rather than the whole tree: the repo
# predates this CI and still carries flake8 debt in older modules, so
# `--all-files` would fail on unrelated code. This runs on pull requests only,
# where the diff against the base branch is well defined. pre-commit runs each
# hook in its own isolated env, so this does not install the heavy project
# dependencies (torch, etc.).
lint:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.12"
enable-cache: true

- name: Cache pre-commit hooks
uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}

- name: Run pre-commit on changed files
run: |
git fetch --no-tags origin "$GITHUB_BASE_REF"
uvx pre-commit run \
--from-ref "origin/$GITHUB_BASE_REF" --to-ref HEAD \
--show-diff-on-failure

test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: "3.12"
enable-cache: true

- name: Install dependencies
run: uv sync --frozen --extra dev --extra research

- name: Test CLI commands
run: |
uv run ami-dataset --help
uv run ami-classification --help

- name: Run test suite
run: uv run pytest tests/ -v

- name: Verify imports
run: |
uv run python -c "import torch; import numpy; import pandas; print('Core dependencies imported successfully')"

- name: Verify optional dependencies (research)
run: |
uv run python -c "import awscli; import absl; print('Research dependencies imported successfully')"
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
42 changes: 30 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,30 +21,46 @@ Software, algorithms and research related to the Automated Monitoring of Insects

## Setup

Poetry is used to manage the dependencies common to all scripts and sub-projects. Some sub-projects may manage their own dependencies if necessary.
[uv](https://github.com/astral-sh/uv) is used to manage the dependencies common to all scripts and sub-projects. uv is a fast, modern Python package manager.

1. Install [Poetry](https://python-poetry.org/docs/#installation)
1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/)
Comment thread
mihow marked this conversation as resolved.
```bash
# On macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# On Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. Clone this repository
2. Create a `.env` or copy `.env.example` and update the values
3. Run `poetry install` in the root of the repository
4. Install pre-commit hooks `poetry run pre-commit install`
3. Create a `.env` or copy `.env.example` and update the values
4. Install dependencies in the root of the repository (`--extra dev` adds the development tools, including pre-commit):
```bash
uv sync --extra dev
```
5. Install pre-commit hooks:
```bash
uv run pre-commit install
```

### [Optional] Conda + uv
An optional way to setup the environment is to use [Conda](https://conda.io/projects/conda/en/latest/index.html) for creating and managing the Python environment, while using [uv](https://docs.astral.sh/uv/) for managing the packages and dependencies.

### [Optional] Conda + Poetry
An optional way to setup the environment is to use [Conda](https://conda.io/projects/conda/en/latest/index.html) for creating and managing the environment, while using [Poetry](https://python-poetry.org/) for managing the packages and dependencies. Run the following steps to setup:
1. [Install Conda](https://docs.anaconda.com/free/miniconda/)
2. Create conda environment using the `environment.yml`: `conda env create -f environment.yml`
3. Activate the conda environment: `conda activate ami-ml`
4. Install packages in the root of the repository using Poetry: `poetry install`
4. Install packages in the root of the repository using uv: `uv sync`
Comment thread
mihow marked this conversation as resolved.

## Usage

Activate the virtual environment before running scripts

```bash
poetry shell
source .venv/bin/activate # On Linux/macOS
# or, on Windows PowerShell
.\.venv\Scripts\Activate.ps1
```

Example for running a script (in the poetry shell):
Example for running a script (in the activated environment):

```bash
python src/localization/inference_localization.py \
Expand All @@ -53,8 +69,10 @@ python src/localization/inference_localization.py \
--model_type fasterrcnn_mobilenet_v3_large_fpn
```

Alternatively, one can run the scripts without activating poetry's shell:
Alternatively, one can run scripts without activating the environment:

```bash
poetry run python <script>
uv run python <script>
```
Comment thread
mihow marked this conversation as resolved.

`uv run` automatically uses the project's virtual environment, so there is no need to activate it first.
4 changes: 2 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ channels:
- conda-forge
- defaults
dependencies:
- python=3.11
- poetry
- python=3.12
- uv
Loading
Loading