-
Notifications
You must be signed in to change notification settings - Fork 544
Pull requests: huggingface/lighteval
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
Fix continuation tokens after distributed gathering
#1357
opened Aug 25, 2026 by
Excelius-Wang
Loading…
Fix DROP exact match: the length check compares gold to gold
#1356
opened Aug 25, 2026 by
Ag3497120
Loading…
fix(sglang): size batch truncation from the longest input in the batch, not the first
#1355
opened Aug 24, 2026 by
shoemoney
Loading…
docs: fix broken code examples in evaluating-a-custom-model.mdx
#1352
opened Aug 22, 2026 by
mohansree14
Loading…
fix: keep every task's responses when reloading them from details
#1350
opened Aug 19, 2026 by
tonycoder-hub
Loading…
2 tasks done
Fix relational comparisons with different condition counts
#1349
opened Aug 19, 2026 by
Excelius-Wang
Loading…
Parse hellaswag_arabic endings with ast.literal_eval
#1348
opened Aug 19, 2026 by
Nakul-Sinha
Loading…
Add add_generation_prompt=True when tokenizing judge prompts
#1347
opened Aug 19, 2026 by
Nakul-Sinha
Loading…
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing
#1335
opened Aug 16, 2026 by
arthi-arumugam-git
Loading…
Fix inspect-ai GenerateConfig field names in
lighteval eval
#1331
opened Aug 13, 2026 by
harshasiddartha
Loading…
fix(inspect): align GenerateConfig field names with current inspect-ai (fixes #1327)
#1328
opened Aug 12, 2026 by
sk8ordie84
Loading…
Fix stale openai reference in endpoint command help text
#1324
opened Aug 9, 2026 by
RudrenduPaul
Loading…
Expose inspect-ai sample_shuffle in the eval command
#1323
opened Aug 7, 2026 by
mshimkus-positron
Loading…
fix: restore hydra-core compatibility
#1321
opened Aug 6, 2026 by
Kloudedge-apex
•
Draft
6 of 8 tasks
Pass custom_tasks through to Registry in the Inspect backend
#1320
opened Aug 2, 2026 by
MohammadSadeghSalehi
Loading…
fix(utils): preserve keys for numpy array values in flatten_dict
#1319
opened Jul 31, 2026 by
cupkk
Loading…
Restore log-likelihood HellaSwag and PIQA variants
#1315
opened Jul 27, 2026 by
kunwar-vikrant
Loading…
Previous Next
ProTip!
no:milestone will show everything without a milestone.