Bot repeating itself

[zhangce]

Problem

The bot often repeats itself.

Examples

User: hi
Bot: Hello, how may I help you?
User: how many legs does a cat has?
Bot: Let me check the information for you. A cat has four legs.
User: could you write a story
Bot: Sure, a story about how a cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs.
User: that was not a story
Bot: Let me try this. A cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs. A cat has four legs.
User: please do not repeat words, tell a story about bear and honey
Bot: Sure, a story about bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey. Bear and honey.

Potential Solutions

https://arxiv.org/abs/2210.00660

[LorrinWWW]

Another solution without touching the model:
https://github.com/huggingface/transformers/blob/ae54e3c3b18bac0832ad62ea9b896dfd52a09850/src/transformers/generation/logits_process.py#L469
It should be trivial to add to HF-based bots. FT does not support it natively but should be possible to add it.