Fix/pixi env missing deps - #56
Merged
Merged
Conversation
…rom env manifests pixi.toml was missing openpyxl, wbgapi, and highspy, all of which the pipeline actually needs but has never declared consistently: - openpyxl/wbgapi: imported directly by rules/preparation.smk steps (retrieve_iron_ore, download_labour_data); a fresh pixi install could not run the pipeline end to end. - highspy: needed by calculate_lcox.py's SHIFT_SOLVER=highs path, which CI (.github/workflows/ci.yml) relies on since CI runners have no Gurobi license. Was present in environment.yaml but absent from pixi.toml. environment.yaml had drifted the other way: it carried ~15 packages with zero references anywhere in the repo (country_converter, powerplantmatching, jpype1, entsoe-py, descartes, fiona, rasterio, rioxarray, memory_profiler, pytz, xlrd, pyxlsb, dask, tsam, pyscipopt, glpk), confirmed by repo-wide grep across scripts, rules, and notebooks, and by checking config.yaml's solver_options (only gurobi-default/highs-default are configured - no scip or glpk path is actually exercised despite both being listed). Removed those; kept pytables/lxml/pypsatopo/graphviz since they're used by standalone analysis notebooks even though not wired into the Snakemake DAG. Also temporarily disables the pre-commit/pre-commit-hooks repo in .pre-commit-config.yaml: its compiled console-script hooks (check-yaml, check-added-large-files, etc.) hit an OS-level "Access is denied" on Windows, reproduced via both PowerShell and Git Bash - looks like local endpoint security blocking a freshly pip-installed unsigned exe.
tests/ was deleted in a prior commit, so the "Run tests" step (pixi run -e test unit-tests) has been silently failing/meaningless - there's nothing for pytest to collect. Replace the job with what CI can actually verify right now: that the pixi environment resolves and installs cleanly on both ubuntu and windows. Also fixes two things that undermined even that goal: - the "**.ya?ml" paths-ignore meant CI never ran on changes to environment.yaml/pixi.toml/this workflow itself - exactly the files most likely to break a clean install. - the job only ran after a PR was already merged (types: [closed] + if: merged == true), which made sense when it ran expensive solves, but is too late to be useful for a cheap install check. Switched to a normal pre-merge PR trigger.
`pip install pixi` does not install prefix.dev's pixi package manager - PyPI's "pixi" is an unrelated Pixiv API client (pulls in pixiv-api, cloudscraper, etc. as dependencies). This was already broken in the original workflow; it just went unnoticed because CI previously only ran post-merge (see the earlier trigger-timing fix), so nobody watched it fail. Confirmed live: `pixi install` failed with "The database needs to be migrated. Run `pixi migrate`." - the telltale error from that wrong package's own CLI, not from real pixi. Use the official setup-pixi action instead, which installs the real pixi binary directly (no PyPI ambiguity) and works across both matrix platforms.
energyLS
approved these changes
Aug 13, 2026
energyLS
left a comment
energyLS
left a comment
Owner
There was a problem hiding this comment.
Looks good, thank you @JanTautorus
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Critical: updated pixi.toml with all required dependencies
Also pruned environmental.yaml of unused dependencies
Fixed the ci pipeline since we don't have tests.
Also pruned pre-commit.