[Bug]: Disallowed special token found

[quan-ngo-ts]

What happened?

When I asked the agent to view my source code, which contains a special token, the agent stopped with an error log: Disallowed special token found: <|im_end|>

What did you expect to happen?

The agent should be able to view the source code. The solution may involve filtering out special tokens from the tool's response or user input using the CLI.

Steps to reproduce

1. Let's create a code file in any language (in my case, it's JS), that contains a special token <|im_end|>
2. Ask Auggie to view the file
3. observe: Auggie stop (checking the process log will get "Disallowed special token found: <|im_end|>"

Auggie version

0.17.0

Request ID

c359d6e6-73b0-44e9-9599-ac1fd18f04dc

Environment details

Environment

• OS: macOS
• Shell: zsh
• Tool/CLI version: 0.17.0

Anything else we need to know?

No response