Skip to content

Require policyengine-core 3.32.9 and stop exporting HF_TOKEN in CI - #1898

Merged
MaxGhenis merged 1 commit into
mainfrom
fix/drop-hf-token-export
Sep 30, 2026
Merged

MaxGhenis merged 1 commit into
mainfrom
fix/drop-hf-token-export

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1816

Summary

policyengine-core 3.32.9 (PolicyEngine/policyengine-core#538, released 2026-09-29) treats a Hugging Face repo as requiring authentication when it is private or gated, so download_huggingface_dataset now passes HUGGING_FACE_TOKEN explicitly to hf_hub_download for policyengine/policyengine-uk-data-private, which has been public but gated since 31 July 2026. Before that release core sent token=None for this repo, and #1817 worked around it by also exporting HF_TOKEN, huggingface_hub's implicit-token variable.

This PR:

  • raises the dependency floor to policyengine-core>=3.32.9 (and relocks: uv.lock moves core from 3.30.4 to 3.32.9; the lock's own entry for this package also catches up from 2.102.0 to the 2.102.3 already in pyproject.toml, and nothing else changes);
  • removes the HF_TOKEN export and its comment from code_changes.yaml and pr_code_changes.yaml, keeping the HUGGING_FACE_TOKEN export.

policyengine_uk/simulation.py is the only place in this repo that downloads from Hugging Face, and it goes through core's download_huggingface_dataset. test_behavioral_responses.py checks HUGGING_FACE_TOKEN, not HF_TOKEN, so it is unaffected. CI's separate uv pip install policyengine --system step does not pin core (the policyengine-core== pins in policyengine 6.1.2 are only in its extras).

Verification

  • Live check against the Hub, with only HUGGING_FACE_TOKEN set (no HF_TOKEN, HF_HUB_DISABLE_IMPLICIT_TOKEN=1, a fresh HF_HOME), downloading enhanced_frs_2023_24_tiny.h5 from policyengine/policyengine-uk-data-private:
  • The Test job in this PR's CI runs every dataset-backed test with only HUGGING_FACE_TOKEN exported, so it is the end-to-end check that the workaround is no longer needed.
  • Not run locally: make test (dataset-backed and heavy; CI runs it) and make format (no Python files change).

axiom: n/a: CI and dependency floor only, no policy change.

🤖 Generated with Claude Code

policyengine-core 3.32.9 (PolicyEngine/policyengine-core#538) treats a
Hugging Face repo as requiring authentication when it is private or
gated, so it now passes HUGGING_FACE_TOKEN explicitly to hf_hub_download
for policyengine/policyengine-uk-data-private, which is public but gated.
The HF_TOKEN export added in #1817 only worked around core sending
token=None for that repo, so remove it from both test workflows and
raise the core floor so the fix is always installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review September 30, 2026 00:51
@MaxGhenis
MaxGhenis merged commit 8849887 into main Sep 30, 2026
7 checks passed
@MaxGhenis
MaxGhenis deleted the fix/drop-hf-token-export branch September 30, 2026 00:52
@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Merged at the reviewed head 7b578f5 with a merge commit, per this repo's history. Gates at merge time:

  • gh pr checks 1898 exited 0 on 7b578f5 (Install + smoke-import py3.11–3.14, Lint, Test, Test documentation builds). The Test job ran every dataset-backed test with only HUGGING_FACE_TOKEN exported.
  • MERGEABLE / CLEAN against main 34e10d5, no CHANGES_REQUESTED review.
  • An independent Opus 5.5 review of 7b578f5 approved it with no blocking findings. I fixed its one body finding before merging: the uv.lock bullet now also mentions the lock's own version catching up from 2.102.0 to 2.102.3.

Follow-up outside this repo: policyengine 6.2.0 pins policyengine-core==3.32.5 together with policyengine-uk==2.102.3 in its extras. So when policyengine.py next bumps its uk pin past this release, it has to move core to >=3.32.9 in the same change.

Carried out under decision d051 (Max 2026-09-29).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI HUGGING_FACE_TOKEN failing with 401 on private dataset downloads

1 participant