yes: use as_encoded_bytes

[oech3]

Closes #12236

[codspeed-hq]

Merging this PR will degrade performance by 12.08%

❌ 2 regressed benchmarks
✅ 315 untouched benchmarks
⏩ 46 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Memory	cp_recursive_deep_tree[(120, 4)]	562.5 KB	699.2 KB	-19.55%
❌	Simulation	ls_recursive_balanced_tree[(6, 4, 15)]	48.9 ms	50.9 ms	-3.92%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing oech3:yes-win-utf (aeec6b1) with main (40c36f6)}

Footnotes

1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[sylvestre]

please add a test to make sure we don't regress

[oech3]

I don't know what letter corresponds with invalid utf8.

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)

[oech3]

@ChrisDenton I cannot write test working on Windows

[ChrisDenton]

Windows OsStr does not support non UTF-8. Well, it does support something called "WTF-8" but it's an extension of UTF-8 (so \xFF isn't valid). Also it's an internal encoding and not guaranteed.

[oech3]

@sylvestre Would you merge this PR as mere refactoring without adding tests?

[oech3]

Does it cause crush at production? Or is invalid args rejected by OS?

[ChrisDenton]

Does it cause crush at production? Or is invalid args rejected by OS?

It is UB, so it's undefined what will happen. The standard library needs to be able to convert an OsStr (which is [u8]) to a platform string (which for Windows is [u16]). So the OsStr needs to have a special encoding to make that possible.

[oech3]

Does as_encoded_bytes have defined behavior at production at least for?

[ChrisDenton]

It has defined cross-platform behaviour only for the UTF-8 subset. You can split on UTF-8. You can join UTF-8. Anything else is platform specific. On Unix it's fine to use bytes. On Windows the encoding isn't guaranteed except that it's a superset of UTF-8.

Production is where most issues will arise since it's optimized for performance.

[oech3]

We only need yes > binary. But it is impossible to test since yes does not use file as input.

[xtqqczze]

I don't know what letter corresponds with invalid utf8.

I think the byte 0xff is never used in the UTF-8 encoding.

https://manpages.ubuntu.com/manpages/stonking/man7/utf-8.7.html

[oech3]

But I can't put it as args.