Currently when using jwt-cli with a payload that is not JSON, we'll get an error.
According to RFC7515 Section 3.3:
Note that the payload can be any content and need not be a representation of a JSON object.
I can try to write the code for it when I have the time.