Skip to content

Fix duplicate custom task module imports - #1351

Open
yangweigbh wants to merge 1 commit into
huggingface:mainfrom
yangweigbh:fix/reuse-custom-task-module
Open

Fix duplicate custom task module imports#1351
yangweigbh wants to merge 1 commit into
huggingface:mainfrom
yangweigbh:fix/reuse-custom-task-module

Conversation

@yangweigbh

Copy link
Copy Markdown

Summary

  • reuse an already imported custom task module when its resolved file path matches
  • avoid rerunning module-level metric and task registration code
  • add regression coverage for preloaded and newly loaded custom task files

Fixes #805

Testing

  • pytest tests/unit/tasks/test_registry.py (10 passed)
  • ruff check src/lighteval/tasks/registry.py tests/unit/tasks/test_registry.py
  • ruff format --check src/lighteval/tasks/registry.py tests/unit/tasks/test_registry.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Registry.task_registry not reimporting the custom task module

1 participant