Skip to content

Commit da5696c

Browse files
committed
Adds arg usage to docs
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
1 parent 719f29d commit da5696c

File tree

1 file changed

+5
-3
lines changed

1 file changed

+5
-3
lines changed

tests/LAYERS.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ The idea is to run, the prompts through the model with the pre- and post-hooks a
1919
The script [generate_layers_metrics.py](../scripts/generate_layers_metrics.py) requires the following arguments to be run:
2020

2121
```bash
22-
usage: generate_layers_metrics.py [-h] [--architecture ARCHITECTURE] [--variant VARIANT] [--model_path MODEL_PATH] --mode {generate,model-forward} --batch_sizes BATCH_SIZES --seq_lengths SEQ_LENGTHS --max_new_tokens MAX_NEW_TOKENS [--output_path OUTPUT_PATH] [--sharegpt_path SHAREGPT_PATH]
22+
usage: generate_layers_metrics.py [-h] [--architecture ARCHITECTURE] [--variant VARIANT] [--model_path MODEL_PATH] --mode {generate,model-forward} --model_loader {fms,hf} --batch_sizes BATCH_SIZES --seq_lengths SEQ_LENGTHS --max_new_tokens MAX_NEW_TOKENS [--output_path OUTPUT_PATH] [--sharegpt_path SHAREGPT_PATH]
2323

2424
Script to generate the model's metrics by layer
2525
@@ -32,6 +32,8 @@ options:
3232
Paths to the directory containing model's weights (.pth files sharded by tensor parallel rank, not HF weights)
3333
--mode {generate,model-forward}
3434
Sets the output generation mode.
35+
--model_loader {fms,hf}
36+
Which model loader/runner to be used; fms - IBM's Foundation Model Stack or hf - HuggingFace Transformers.
3537
--batch_sizes BATCH_SIZES
3638
Batch sizes separated by comma. Eg.: 1,2
3739
--seq_lengths SEQ_LENGTHS
@@ -79,7 +81,7 @@ cd aiu-fms-testing-utils/tests/resources
7981
8082
mkdir /tmp/output
8183
82-
python3 generate_layers_metrics.py --mode model-forward --variant ibm-granite/granite-3.2-8b-instruct --architecture hf_pretrained --batch_sizes 1 --seq_lengths 64 --max_new_tokens 128
84+
python3 generate_layers_metrics.py --mode model-forward --variant ibm-granite/granite-3.2-8b-instruct --architecture hf_pretrained --batch_sizes 1 --seq_lengths 64 --max_new_tokens 128 --model_loader fms
8385
```
8486
The files should get created at `/tmp/output` dir:
8587
```bash
@@ -95,7 +97,7 @@ To get the second step of the flow and get the thresholds by layer, run:
9597
```bash
9698
cd /aiu-fms-testing-utils/tests/resources
9799
98-
python3 get_thresholds.py --models ibm-granite/granite-3.2-8b-instruct --metrics abs_diff cos_sim_avg cos_sim_men --file_base /tmp/output --layer_io
100+
python3 get_thresholds.py --models ibm-granite/granite-3.2-8b-instruct --metrics abs_diff cos_sim_avg cos_sim_mean --file_base /tmp/output --layer_io
99101
```
100102
It should print the metric of each layer:
101103
```bash

0 commit comments

Comments
 (0)