Skip to content

head_size = getattr(model.config, "head_dim",...) #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

head_size = getattr(model.config, "head_dim",...) #149

wants to merge 22 commits into from

Conversation

@rzbhatti
Copy link

@rzbhatti rzbhatti commented Oct 6, 2025

To support the smaller models where
model.config.emb_dim // model.config.nheads != head_dim -

head_size = getattr(model.config, "head_dim", model.config.emb_dim // model.config.nheads )

head_size = getattr(model.config, "head_dim", model.config.emb_dim /…
4e611a9 
…/ model.config.nheads )

Signed-off-by: Rashed Z. Bhatti, PhD <rzbhatti@us.ibm.com>
@rzbhatti rzbhatti requested a review from ani300 October 6, 2025 19:57
@rzbhatti rzbhatti mentioned this pull request Oct 6, 2025
Merged
kcirred and others added 21 commits October 15, 2025 01:30
@kcirred
[dpp] store enforce_sizes in log name and added generic kwargs to get…
043926c 
…_default_validation_prefix, enable sample_key

Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[utils] added doc string, refactor sample_key, added return_key to sq…
6d6541f 
…uad_v2 sampler

Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[dpp/validation] restore sample_key in logic after rebase of main
97aeea3 
Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[validation] Modified final file string to hash due to OSError name t…
9efb6e7 
…oo long

Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[test_validation] remove unused line
ea1a8ac 
Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[validation] removed enforce_sizes from find_validation_info_path
b637ea5 
Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@kcirred
[dpp] added handling of return_key for __custom_line_sampler
aef361e 
Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@JRosenkranz
updated llama model expectation tests using v1.0.0 aiu software stack…
178b75b 
… as modeling code changed

Signed-off-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
@kcirred
[testing] changed get_default_validation_prefix to generic kwargs, fi…
73f6551 
…le names now sorted, testing of file names modified for new order

Signed-off-by: kcirred <16872435+kcirred@users.noreply.github.com>
@JRosenkranz
fixed test_scripts program assertion
86d60d6 
Signed-off-by: Joshua Rosenkranz <jmrosenk@us.ibm.com>
@avery-blanchard
Add test case for caching
f3dcee1 
Signed-off-by: Avery Blanchard <avery.blanchard@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@avery-blanchard
Update cache test, add validation for cached run
f936eb5 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
Squashed commit of the following:
81870d5 
commit ed571f728a351f8dd92737be5554c3dc46f71a30
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Tue Jul 29 09:20:06 2025 -0600

    Remove cache tests

commit 2848f7b2785b91c60c536b8993c3193c40c381ea
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Mon Jul 28 08:07:01 2025 -0600

    Add leading underscores, revert model name

commit c30b7b70a0f6e464d3212fd9bed4f9ea33f9de93
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Mon Jul 28 07:15:08 2025 -0600

    Explictly clear cache paths

commit 42aaf666d7f8ffb2fb611df7ad2d06b48e480dd7
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Mon Jul 28 07:14:23 2025 -0600

    Set the cache dir in conftest

commit b978e7225f02bf1d9a5f7b919ca6cbe2ee8d641a
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Mon Jul 28 06:15:10 2025 -0600

    run formatting

commit 8d64df08333991927c45f9a982ddaf95f39c94cf
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Fri Jul 25 11:18:13 2025 -0600

    refactor cache miss into fixture

commit 0b524b8c818495cb646add2adfc27a2884ac8de5
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Fri Jul 25 07:11:09 2025 -0600

    Consolidate cache test with common

commit d8a36d405a101e101ab9ede3b8d12fa3026cd01f
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Fri Jul 25 06:41:13 2025 -0600

    Run cache test first

commit 2efb797fb21587e9136b314c44ec56c658636826
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Fri Jul 25 05:48:25 2025 -0600

    Finish splitting out common shape test helpers

commit 4ae73dea18848005f86d1c9bcdf29f153711330f
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Fri Jul 25 05:28:31 2025 -0600

    refactor most of common shape test

commit 083afdc3a468649ec4b0bbadc921d40b47e37498
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Thu Jul 24 14:08:20 2025 -0600

    Move torch sendnn cache dir to common

commit e9b576381a738c59f91d5fc904ceaa2a0e410864
Author: Alex-Brooks <Alex.Brooks@ibm.com>
Date:   Thu Jul 24 14:02:06 2025 -0600

    Use caps for constants, common post proc

Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
don't skip save on aiu iter0
ad3a584 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
fix fp8 dtype, always use persistent model fixture
54cc09b 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
remove model path from get_cpu_model args
75eb5fb 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
fix casing error
f1a810e 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
Rebase fixes, linting
15cf522 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
fix input prep
a743968 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
@alex-jw-brooks
use setdefault for torch sendnn cache dir
b25e44f 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Added head_dim override option to inference.py
4609ad5 
Signed-off-by: Rashed Z. Bhatti, PhD <rzbhatti@us.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ani300 ani300 Awaiting requested review from ani300

At least 1 approving review is required to merge this pull request.

Assignees

@ani300 ani300

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@rzbhatti @ani300 @kcirred @JRosenkranz @avery-blanchard @alex-jw-brooks