@@ -19,7 +19,7 @@ The idea is to run, the prompts through the model with the pre- and post-hooks a
1919The script [ generate_layers_metrics.py] ( ../scripts/generate_layers_metrics.py ) requires the following arguments to be run:
2020
2121``` bash
22- usage: generate_layers_metrics.py [-h] [--architecture ARCHITECTURE] [--variant VARIANT] [--model_path MODEL_PATH] --mode {generate,model-forward} --batch_sizes BATCH_SIZES --seq_lengths SEQ_LENGTHS --max_new_tokens MAX_NEW_TOKENS [--output_path OUTPUT_PATH] [--sharegpt_path SHAREGPT_PATH]
22+ usage: generate_layers_metrics.py [-h] [--architecture ARCHITECTURE] [--variant VARIANT] [--model_path MODEL_PATH] --mode {generate,model-forward} --model_loader {fms,hf} -- batch_sizes BATCH_SIZES --seq_lengths SEQ_LENGTHS --max_new_tokens MAX_NEW_TOKENS [--output_path OUTPUT_PATH] [--sharegpt_path SHAREGPT_PATH]
2323
2424Script to generate the model' s metrics by layer
2525
@@ -32,6 +32,8 @@ options:
3232 Paths to the directory containing model' s weights (.pth files sharded by tensor parallel rank, not HF weights)
3333 --mode {generate,model-forward}
3434 Sets the output generation mode.
35+ --model_loader {fms,hf}
36+ Which model loader/runner to be used; fms - IBM' s Foundation Model Stack or hf - HuggingFace Transformers.
3537 --batch_sizes BATCH_SIZES
3638 Batch sizes separated by comma. Eg.: 1,2
3739 --seq_lengths SEQ_LENGTHS
@@ -79,7 +81,7 @@ cd aiu-fms-testing-utils/tests/resources
7981
8082mkdir /tmp/output
8183
82- python3 generate_layers_metrics.py --mode model-forward --variant ibm-granite/granite-3.2-8b-instruct --architecture hf_pretrained --batch_sizes 1 --seq_lengths 64 --max_new_tokens 128
84+ python3 generate_layers_metrics.py --mode model-forward --variant ibm-granite/granite-3.2-8b-instruct --architecture hf_pretrained --batch_sizes 1 --seq_lengths 64 --max_new_tokens 128 --model_loader fms
8385```
8486The files should get created at `/tmp/output` dir:
8587```bash
@@ -95,7 +97,7 @@ To get the second step of the flow and get the thresholds by layer, run:
9597```bash
9698cd /aiu-fms-testing-utils/tests/resources
9799
98- python3 get_thresholds.py --models ibm-granite/granite-3.2-8b-instruct --metrics abs_diff cos_sim_avg cos_sim_men --file_base /tmp/output --layer_io
100+ python3 get_thresholds.py --models ibm-granite/granite-3.2-8b-instruct --metrics abs_diff cos_sim_avg cos_sim_mean --file_base /tmp/output --layer_io
99101```
100102It should print the metric of each layer:
101103```bash
0 commit comments