Inconsistent character encoding from windows output

[jtpio]

Looking at this issue in the JupyterLab repo, we wanted to migrate it here since it's not related to the JupyterLab frontend: jupyterlab/jupyterlab#1239

Original report:

---

trying "!dir" in Jupyterlab Notebook, hoping for the best as there may be some "utf-8" standardization thanks to PEP 528 / 529, I see no effect: still badly decoded characters.

I have difficulties to guess from bugs.python.org if something can be done in Jupyterlab (or qtconsole) case: https://bugs.python.org/issue28675

Maybe I don't take the right example to see the PEP 528 / 529 effects.

[minrk]

This shouldn't be JupyterLab's problem. The protocol does specify that all text is UTF-8. So it is between IPython (or ipykernel, but more likely IPython), where the bytes->str decoding happens exactly once, and there's an incorrect guess (maybe).

I think the encoding is defined here.

If someone can write a simple bit of Python code that does the right thing just with Popen('dir').communicate(), I think we can (maybe) fix it.