Checklist
Bug Description
Describe the bug
After installing lmms-eval via pip install lmms-eval==0.6.1, running the command lmms-eval --tasks list returns an empty list for available tasks.
Root Cause Analysis
I investigated the installation directory in site-packages/lmms_eval/tasks and found that no .yaml configuration files were included in the pip package. It seems the resource files (YAML) are missing from the PyPI distribution, causing the TaskManager to find zero tasks.
Questions
Is this a known packaging issue where .yaml files are excluded from the wheel file?
Is it mandatory to install from source (pip install -e .) to get the task configurations?
If I want to keep the pip installation, is there a recommended way to sync the tasks/ directory separately?
Steps to Reproduce
1. pip install lmms-eval==0.6.1
2. lmms-eval --tasks list
Error Message / Traceback
Environment
- Python 3.11.14
- lmms-eval 0.6.1
Additional Context
No response
Checklist
Bug Description
Describe the bug
After installing lmms-eval via pip install lmms-eval==0.6.1, running the command lmms-eval --tasks list returns an empty list for available tasks.
Root Cause Analysis
I investigated the installation directory in site-packages/lmms_eval/tasks and found that no .yaml configuration files were included in the pip package. It seems the resource files (YAML) are missing from the PyPI distribution, causing the TaskManager to find zero tasks.
Questions
Is this a known packaging issue where .yaml files are excluded from the wheel file?
Is it mandatory to install from source (pip install -e .) to get the task configurations?
If I want to keep the pip installation, is there a recommended way to sync the tasks/ directory separately?
Steps to Reproduce
Error Message / Traceback
Environment
Additional Context
No response