Skip to content

Strange new line behavior and missing color in the iOS testbed logs #150932

@timrid

Description

@timrid

Bug report

Bug description:

When using cibuildwheel to build and test a Python package for iOS, pytest output captured through the Apple testbed runner is difficult to read. Two distinct problems occur simultaneously:

1. Progress output is split across individual lines.

pytest emits progress characters (., F, E, s, …) without a trailing newline and flushes after each one. In the captured log each character appears on its own line instead of forming a compact progress line:

============================= test session starts ==============================
 platform ios -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0
 rootdir: /Users/timrid/Library/Developer/CoreSimulator/Devices/84EFB4EB-1C63-49FF-972C-D7276CB71115/data/Containers/Bundle/Application/3CA6642E-C9F4-4388-B444-2FA31C2AA074/iOSTestbed.app/app
 configfile: setup.cfg
 testpaths: tests/
 plugins: mock-3.15.1, cov-7.1.0, timeout-2.4.0, asyncio-1.4.0
 timeout: 120.0s
 timeout method: signal
 timeout func_only: False
 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
 collected 4612 items / 24 deselected / 14 skipped / 4588 selected
 
 tests/test_base_protocol.py
 .
 .
 .
 .
 .
 .
 .
 .
                           [  0%]
                      [  0%]
 
 tests/test_circular_imports.py
 s
 s
 s
 s
 s [  1%]
 s
 s
 s
 s
 s
                                                             [  1%]
 
 tests/test_classbasedview.py
 .
 .
 .
 .
                                         [  1%]
 
...

The same problem affects any output that is deliberately written without a newline — e.g. print(".", end="", flush=True).

2. ANSI color codes are shown as raw escape sequences.

When color is explicitly enabled in pytest via --color=yes, pytest's colored output is not rendered. Instead, raw escape sequences appear as unreadable caret-encoded text:

\^[[1m============================= test session starts ==============================\^[[0m
 platform ios -- Python 3.13.11, pytest-9.0.3, pluggy-1.6.0
 rootdir: /Users/timrid/Library/Developer/CoreSimulator/Devices/84EFB4EB-1C63-49FF-972C-D7276CB71115/data/Containers/Bundle/Application/F9002FB8-67E9-4424-AC42-8F5807D4DB47/iOSTestbed.app/app
 configfile: setup.cfg
 testpaths: tests/
 plugins: mock-3.15.1, cov-7.1.0, timeout-2.4.0, asyncio-1.4.0
 timeout: 120.0s
 timeout method: signal
 timeout func_only: False
 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
 collected 4612 items / 24 deselected / 14 skipped / 4588 selected
 
 tests/test_base_protocol.py
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m.\^[[0m
 \^[[32m                          [  0%]\^[[0m
 
 tests/test_circular_imports.py
 \^[[33ms\^[[0m
 \^[[33ms\^[[0m
...

Together these two issues make it very hard to interpret test results, especially in CI in combination with bad Github Actions performance on huge test outputs.

I think that this is an upstream CPython issue because the root cause is in Python's Apple system-logger redirection (Lib/_apple_support.py), which is used by the testbed runner to capture output from iOS builds.

Maybe a possible solution for problem 1 is to modify the Apple system-logger redirection in Lib/_apple_support.py to append a marker byte (e.g. 0x1f, ASCII Unit Separator) to any log message that does not end with \n. The testbed runner can then detect the marker, suppress the line break, and join the message with the next one. This would fix it without buffering output or breaking the flush() contract. To solve problem 2, the testbed runner can reverse the system log's caret encoding (\^[ etc.) back into the original control characters, restoring colored output.

@freakboy3742 What do you think?

CPython versions tested on:

3.13

Operating systems tested on:

Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-mactype-bugAn unexpected behavior, bug, or error
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions