Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Commit 02b903b

Browse files
martinpopelrsepassi
authored andcommitted
Show t2t-bleu usage (#522)
* Show t2t-bleu usage Many researchers are still reporting approx_bleu and comparing it with proper BLEU reported in other papers. We should guide them to not do this and minimize the risk of false-news papers, such as https://arxiv.org/abs/1712.09662. * make sure ref-translation.de exists So the Walkthrough works out of the box
1 parent f0e638e commit 02b903b

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,7 @@ t2t-trainer \
133133
DECODE_FILE=$DATA_DIR/decode_this.txt
134134
echo "Hello world" >> $DECODE_FILE
135135
echo "Goodbye world" >> $DECODE_FILE
136+
echo -e 'Hallo Welt\nAuf Wiedersehen Welt' > ref-translation.de
136137
137138
BEAM_SIZE=4
138139
ALPHA=0.6
@@ -144,9 +145,12 @@ t2t-decoder \
144145
--hparams_set=$HPARAMS \
145146
--output_dir=$TRAIN_DIR \
146147
--decode_hparams="beam_size=$BEAM_SIZE,alpha=$ALPHA" \
147-
--decode_from_file=$DECODE_FILE
148+
--decode_from_file=$DECODE_FILE \
149+
--decode_to_file=translation.en
148150
149-
cat $DECODE_FILE.$MODEL.$HPARAMS.beam$BEAM_SIZE.alpha$ALPHA.decodes
151+
# Eval BLEU
152+
# (Always report proper BLEU in papers, not the internal approx_bleu.)
153+
t2t-bleu --translation=translation.en --reference=ref-translation.de
150154
```
151155

152156
---

0 commit comments

Comments
 (0)