fix: skip source files that are not valid UTF-8 instead of failing refresh

[TheCryptoDonkey]

One file that was not valid UTF-8 failed the whole navigation refresh, so the checkout got no navigation at all. The case that surfaced it was a checkout with HLS video segments (MPEG transport streams saved as .ts), the only failure in the 0.4.2 survey (267 of 268 checkouts).

• Behaviour: refresh and the freshness manifest skip a source file whose bytes are not valid UTF-8 and count it in a new invalidUtf8 exclusion. The rest of the checkout is indexed. Both paths skip the same files, so freshness stays exact: a skipped file that becomes valid text, or an indexed file that stops being valid, makes the index stale. The status completeness string names the exclusion.
• Unchanged: policy files (.gitignore, .z1p-navigation.json) still fail closed on invalid UTF-8. Quota failures (such as maxFileBytes) still fail the refresh and keep the prior generation.
• Tests: a new unit test covers a fake transport-stream .ts file, both freshness transitions and search results. The failed-refresh retention test, the unknown-freshness test and the stdio smoke previously used invalid UTF-8 as their failure trigger; they now use an oversized file. The smoke adds an invalidUtf8SkippedAndCounted check, and its second session confirms the same revision with the skipped file present.
• Docs: LOCAL-NAVIGATION.md and NAVIGATION-SMOKE.md.

Checks: the pre-push npm run check passed (34 core and 191 tools tests, the remaining suites and packed-package smoke), plus the navigation stdio smoke and both benchmark gates. Two earlier hook runs under a load average of about 80 on 8 cores timed out tests at vitest's 5-second limit (the existing 10,050-line pagination test, then five untouched core tests); they pass at normal load.

Local probe with this build on the affected checkout: 1,920 files indexed, 6 skipped as invalidUtf8, freshness current after refresh. Single run.

Not released; no version bump.