Send binary files via application/octet-stream in boxel-cli sync (CS-11075)#4852
Open
FadhlanR wants to merge 3 commits into
Open
Send binary files via application/octet-stream in boxel-cli sync (CS-11075)#4852FadhlanR wants to merge 3 commits into
FadhlanR wants to merge 3 commits into
Conversation
Make boxel-cli's push / pull / sync / watch and the single-file `boxel file write` / `boxel file read` commands handle binary assets (images, PDFs, fonts, etc.) byte-identically by mirroring the host package's wire format: a per-file POST with `Content-Type: application/octet-stream` and raw bytes as the body, which the realm-server already routes to `upsertBinaryFile`. Detection uses the existing `isBinaryFilename` helper from runtime-common, so SVG (XML-based) stays on the text path while `image/*`, `font/*`, `application/pdf`, and `.eot` switch to bytes. The /_atomic batch endpoint stays text-only — binary entries are carved out and POSTed per-file alongside the atomic call, matching the host package which never sends binary through /_atomic. Adds binary roundtrip tests across realm-push, realm-pull, realm-watch, file-write, and file-read using an inline 1x1 transparent PNG fixture so non-UTF-8 bytes (PNG signature, IDAT chunk) round-trip verbatim. Linear: CS-11075 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7a60cacb53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
When uploadFilesAtomic split binaries out of /_atomic, a partial binary failure left the atomic text successes unrecorded on disk — the next push hit a 409 against the files the server already had. Record result.succeeded unconditionally in push and sync, drop the brittle status-from-message regex in favor of a status field on the thrown Error (with the response body in the message), and reject source/destination binary-classification mismatches in `file write`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bare `from '@cardstack/runtime-common'` pulled the whole barrel into the bundle and the type-check program: transpile.ts → content-tag, plus ~50 modules that import `https://cardstack.com/base/*`. That broke the smoke tests (`ENOENT: dist/content_tag_bg.wasm` at startup) and `lint:types` (TS2307 cascade). Switching to `/infer-content-type` keeps the bundle to the leaf file's tiny graph, matching the subpath-only convention every other boxel-cli source file follows. Bundle shrinks from 7.2 MB to 370 KB; lint clean; integration suite green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
U+FFFD).isBinaryFilename(already exported byruntime-common) and route binary files through a per-file POST withContent-Type: application/octet-stream— exactly the wire formatpackages/host'sFileUploadServiceandWriteBinaryFileCommandalready use, which the realm-server routes toupsertBinaryFile./_atomicbatch endpoint stays text-only (it embeds content inside a JSON string). When a push batch contains binary files, they are carved out and POSTed per-file in parallel alongside the atomic call for the text files. Failures are folded back into the existingperFileerror shape so callers don't have to learn a second contract.boxel file write/boxel file readcommands grow a symmetric branch:write()now acceptsUint8Arrayand switches to octet-stream;read()populates a newbytes?: Uint8Arrayfield for binary paths and renders bytes to stdout (or base64 under--json).Test plan
pnpm lintclean inpackages/boxel-cli(the runtime-commonhttps://cardstack.com/base/*resolution errors that surface duringlint:typesare pre-existing and unrelated to this change).pnpm test:unit-exclude-smoke— 187/187 unit tests pass.packages/boxel-cli/tests/integration/realm-push.test.ts— 23/23 pass (4 new binary tests: PNG, PDF, mixed batch, SVG-stays-text).packages/boxel-cli/tests/integration/realm-pull.test.ts— 13/13 pass (1 new binary PNG test).packages/boxel-cli/tests/integration/realm-sync.test.ts— 16/16 pass (no new tests; regression coverage for the partitioned upload path).packages/boxel-cli/tests/integration/realm-watch.test.ts— 15/15 pass (1 new binary PNG test).packages/boxel-cli/tests/integration/file-write.test.ts— 5/5 pass (2 new binary tests: PNG, PDF).packages/boxel-cli/tests/integration/file-read.test.ts— 5/5 pass (1 new binary PNG test)..png+.pdf+.woff2+.gts, pull into an empty dir, hash both sides withshasum -a 256to confirm byte-equality. (Steps documented indocs/cs-11075-binary-upload-plan.md.)Test fixture is a 67-byte 1x1 transparent PNG defined inline in
packages/boxel-cli/tests/helpers/binary-fixtures.ts— the IDAT chunk contains non-UTF-8 byte sequences (0x89, embedded nulls) that get mangled if anything in the pipeline forces UTF-8 round-tripping, so a byte-levelBuffer.equalsassertion is enough to catch any regression.Linear: https://linear.app/cardstack/issue/CS-11075
🤖 Generated with Claude Code