Updates to pipeline & tools for 2026#74

Draft

mihow wants to merge 22 commits into

mainfrom

melabbas/pipeline-and-tools

Commits on Mar 8, 2026

chore: exclude data, plans, and SLURM log files from version control

mohamedelabbas1996
and
claude
committed
config: add GBIF credentials env vars and fiftyone optional dependency

mohamedelabbas1996
and
claude
committed
chore: update lockfiles after adding fiftyone optional dependency

mohamedelabbas1996
and
claude
committed
chore: standardize SLURM headers and venv activation in existing job scripts

mohamedelabbas1996
and
claude
committed
feat: include verbatimScientificName in DwCA data loading

mohamedelabbas1996
and
claude
committed
feat: allow clean-dataset to run without prior verification step

mohamedelabbas1996
and
claude
committed
feat: add download-gbif module and CLI command for GBIF DwCA downloads

mohamedelabbas1996
and
claude
committed
feat: add fetch-and-pack module for quota-safe chunked webdataset creation

mohamedelabbas1996
and
claude
committed
feat: add FiftyOne webdataset visualization module

mohamedelabbas1996
and
claude
committed
feat: add Vermont species pipeline SLURM scripts

mohamedelabbas1996
and
claude
committed
feat: add GBIF download and DwCA inspection job scripts

mohamedelabbas1996
and
claude
committed
feat: add FiftyOne visualization job scripts

mohamedelabbas1996
and
claude
committed
feat: add fetch-and-pack chunked pipeline and test scripts

mohamedelabbas1996
and
claude
committed
chore: add archived and legacy pipeline scripts

mohamedelabbas1996
and
claude
committed
feat: add webdataset inspection script with class distribution stats
mohamedelabbas1996
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates to pipeline & tools for 2026#74

Updates to pipeline & tools for 2026#74
mihow wants to merge 22 commits into
mainfrom
melabbas/pipeline-and-tools

Commits on Jan 20, 2026

Initial plan

Migrate from Poetry to uv: Convert pyproject.toml to PEP 621 format and update README

Co-authored-by: mihow <158175+mihow@users.noreply.github.com>

Add GitHub Actions CI workflow for uv-based testing

Update Python requirement to >=3.10, simplify CI to test only Python 3.12, and add research extras testing

Commits on Mar 8, 2026

chore: exclude data, plans, and SLURM log files from version control

config: add GBIF credentials env vars and fiftyone optional dependency

chore: update lockfiles after adding fiftyone optional dependency

chore: standardize SLURM headers and venv activation in existing job scripts

feat: include verbatimScientificName in DwCA data loading

feat: allow clean-dataset to run without prior verification step

feat: add download-gbif module and CLI command for GBIF DwCA downloads

feat: add fetch-and-pack module for quota-safe chunked webdataset creation

feat: add FiftyOne webdataset visualization module

feat: add Vermont species pipeline SLURM scripts

feat: add GBIF download and DwCA inspection job scripts

feat: add FiftyOne visualization job scripts

feat: add fetch-and-pack chunked pipeline and test scripts

chore: add archived and legacy pipeline scripts

feat: add webdataset inspection script with class distribution stats

Commits on Jun 4, 2026

Merge remote-tracking branch 'origin/main' into melabbas/pipeline-and-tools

test: compare command help against whitespace-normalized output