Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/bgcode-meatpack-phase2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@chestnutlabs/gcode-bgcode': minor
---

Binary G-code decode **phase 2** (DD-011, #188): the **MeatPack** G-code encoding (both variants —
`MeatPack` and `MeatPack (comments preserved)`). A `.bgcode` GCode block encoded with MeatPack now
decodes end-to-end (optionally after DEFLATE) to plain G-code.

The decoder is a faithful TypeScript port of the **MIT** `jamesgopsill/meatpack` unpacker (© 2025
James Gopsill), which is itself derived from the published Prusa spec — attribution preserved, and no
AGPL `libbgcode`/OctoPrint-MeatPack code (RR-003 §8). It is validated against **hand-computed vectors**
(the nibble table applied by hand as an independent oracle: packing, left/right/double full-width
escapes, the newline special case, and the no-spaces + disable-packing commands), plus block-level
integration through `openBgcode`. Output is bounded (decompression-bomb defense) and invalid command
bytes are structured errors. heatshrink compression remains phase 3.
9 changes: 5 additions & 4 deletions packages/gcode-bgcode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ network), zero runtime dependencies beyond `@chestnutlabs/toolpath-core` and
## Licensing (why this is a clean-room decoder)

The reference `libbgcode` and OctoPrint-MeatPack are **AGPL-3.0**; none of their code is used here — the
decoder is written clean-room from the published format spec ([RR-003](https://github.com/ChestnutLabs/gcode-preview/blob/dev/docs/research/RR-003-bgcode-licensing-and-format-audit.md)).
The heatshrink codec (phase 3) is a license-clean port of the **ISC** heatshrink library, with its notice
preserved. This package is MIT.
block walker is written clean-room from the published format spec ([RR-003](https://github.com/ChestnutLabs/gcode-preview/blob/dev/docs/research/RR-003-bgcode-licensing-and-format-audit.md)).
The **MeatPack** decoder is a faithful port of the **MIT** [`jamesgopsill/meatpack`](https://github.com/jamesgopsill/meatpack)
(© 2025 James Gopsill), attribution preserved; the heatshrink codec (phase 3) is a license-clean port of
the **ISC** heatshrink library. This package is MIT.

## Status — phased (DD-011 §14)

Expand All @@ -23,7 +24,7 @@ preserved. This package is MIT.
| Block walker + CRC32 + `sniff`/`open` | ✅ phase 1 |
| Compression: None, DEFLATE | ✅ phase 1 |
| Encoding: None | ✅ phase 1 |
| Encoding: MeatPack | ⏳ phase 2 (honest `E_BGCODE_UNSUPPORTED_ENCODING` until then) |
| Encoding: MeatPack (both variants) | ✅ phase 2 |
| Compression: heatshrink 11/12 | ⏳ phase 3 (honest `E_BGCODE_UNSUPPORTED_COMPRESSION` until then) |
| Container-adapter + worker registration + metadata/thumbnails + golden-equivalence | ⏳ phase 4 |

Expand Down
7 changes: 1 addition & 6 deletions packages/gcode-bgcode/src/__tests__/bgcode.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,11 @@ describe('openBgcode — honest rejection & bounds', () => {
await expect(openBgcode(buf.subarray(0, 14))).rejects.toMatchObject({ code: 'E_BGCODE_TRUNCATED' });
});

it('honestly reports heatshrink (phase 3) and MeatPack (phase 2) as not-yet-supported', async () => {
it('honestly reports heatshrink compression (phase 3) as not-yet-supported', async () => {
const hs = await assembleBgcode([
{ type: 1, compression: BgcodeCompression.Heatshrink12, encoding: 0, data: enc(GCODE) }
]);
await expect(openBgcode(hs)).rejects.toMatchObject({ code: 'E_BGCODE_UNSUPPORTED_COMPRESSION' });

const mp = await assembleBgcode([
{ type: 1, compression: BgcodeCompression.None, encoding: BgcodeEncoding.MeatPack, data: enc(GCODE) }
]);
await expect(openBgcode(mp)).rejects.toMatchObject({ code: 'E_BGCODE_UNSUPPORTED_ENCODING' });
});

it('enforces the output-bytes cap (decompression-bomb defense)', async () => {
Expand Down
87 changes: 87 additions & 0 deletions packages/gcode-bgcode/src/__tests__/meatpack.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
/**
* MeatPack decoder — phase 2 (DD-011, #188). Validated against HAND-COMPUTED vectors: each expected
* byte stream is assembled by applying the published nibble table by hand (an oracle independent of
* the decoder's own table), so a wrong table/order/escape is caught — not merely a self-consistent
* round-trip. Real PrusaSlicer `.bgcode` MeatPack blocks are added for golden-equivalence in phase 4.
*
* Table (low nibble = 1st char): 0-9→'0'-'9', A→'.', B→' '/'E', C→'\n', D→'G', E→'X', F→escape.

Check warning on line 7 in packages/gcode-bgcode/src/__tests__/meatpack.test.ts

View workflow job for this annotation

GitHub Actions / build

tsdoc-unnecessary-backslash: A backslash can only be used to escape a punctuation character
* A packed byte is `(highNibble << 4) | lowNibble`; `FF FF FB` enables packing (stream starts disabled).
*/
import { describe, expect, it } from 'vitest';
import { meatpackDecode, openBgcode, BgcodeBlockType, BgcodeCompression, BgcodeEncoding } from '../index.js';
import { assembleBgcode } from './assemble.js';

const HEADER = [0xff, 0xff, 0xfb]; // MEATPACK_HEADER: enable packing
const dec = (b: Uint8Array): string => new TextDecoder().decode(b);
const mp = (...bytes: number[]): string => dec(meatpackDecode(Uint8Array.from(bytes), 1 << 20));

describe('meatpackDecode — hand-computed vectors', () => {
it('decodes a realistic packable line "G1 X2.5\\n"', () => {
// (G,1)=0x1D (space,X)=0xEB (2,.)=0xA2 (5,\n)=0xC5
expect(mp(...HEADER, 0x1d, 0xeb, 0xa2, 0xc5)).toBe('G1 X2.5\n');
});

it('right full-width escape: 2nd char not in the table ("GT")', () => {
// (G, escape)=0xFD, then the literal 'T' (0x54) follows.
expect(mp(...HEADER, 0xfd, 0x54)).toBe('GT');
});

it('left full-width escape: 1st char not in the table ("TG")', () => {
// (escape, G)=0xDF, then the literal 'T' (0x54) follows.
expect(mp(...HEADER, 0xdf, 0x54)).toBe('TG');
});

it('double full-width via a lone 0xFF data byte ("TT")', () => {
// both chars unpackable → 0xFF then the two literals 'T','T'.
expect(mp(...HEADER, 0xff, 0x54, 0x54)).toBe('TT');
});

it('a packed (\\n,\\n) byte collapses to a single newline', () => {
// 0xCC = (\n high, \n low) → one '\n'.
expect(mp(...HEADER, 0xcc)).toBe('\n');
});

it("the no-spaces command remaps 0b1011 from space to 'E'", () => {
// FF FF F7 = enable no-spaces; then 0xBD = (space-slot high=E, G low) → "GE".
expect(mp(...HEADER, 0xff, 0xff, 0xf7, 0xbd)).toBe('GE');
});

it('a mid-stream disable-packing command passes bytes through literally', () => {
// FF FF FA = disable packing; then raw ASCII "G1\n".
expect(mp(...HEADER, 0xff, 0xff, 0xfa, 0x47, 0x31, 0x0a)).toBe('G1\n');
});

it('rejects an invalid command byte', () => {
expect(() => meatpackDecode(Uint8Array.from([0xff, 0xff, 0x00]), 1024)).toThrow(/MeatPack command/);
});
});

describe('openBgcode — MeatPack-encoded GCode block (phase 2 integration)', () => {
it('decodes a MeatPack block end-to-end (None compression + MeatPack encoding)', async () => {
const packed = Uint8Array.from([...HEADER, 0x1d, 0xeb, 0xa2, 0xc5]); // "G1 X2.5\n"
const buf = await assembleBgcode([
{
type: BgcodeBlockType.GCode,
compression: BgcodeCompression.None,
encoding: BgcodeEncoding.MeatPack,
data: packed
}
]);
const { gcode } = await openBgcode(buf);
expect(dec(gcode)).toBe('G1 X2.5\n');
});

it('decodes a DEFLATE-compressed MeatPack block (both layers)', async () => {
const packed = Uint8Array.from([...HEADER, 0x1d, 0xeb, 0xa2, 0xc5]);
const buf = await assembleBgcode([
{
type: BgcodeBlockType.GCode,
compression: BgcodeCompression.Deflate,
encoding: BgcodeEncoding.MeatPackComments,
data: packed
}
]);
const { gcode } = await openBgcode(buf);
expect(dec(gcode)).toBe('G1 X2.5\n');
});
});
16 changes: 8 additions & 8 deletions packages/gcode-bgcode/src/bgcode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
* supported" errors until phases 2 and 3.
*/
import { ContainerError, crc32, crc32Final } from '@chestnutlabs/gcode-containers';
import { meatpackDecode } from './meatpack.js';

/** ASCII magic at the start of every `.bgcode` file: "GCDE". */
export const BGCODE_MAGIC = 'GCDE';
Expand Down Expand Up @@ -164,14 +165,12 @@ async function decompress(
throw new ContainerError('E_BGCODE_UNSUPPORTED_COMPRESSION', `unknown compression id ${compression}`);
}

/** Decode a GCode block's post-decompression bytes by its encoding ID. */
function decode(encoding: number, bytes: Uint8Array): Uint8Array {
/** Decode a GCode block's post-decompression bytes by its encoding ID, bounded to `limit` output bytes. */
function decode(encoding: number, bytes: Uint8Array, limit: number): Uint8Array {
if (encoding === BgcodeEncoding.None) return bytes;
// Both MeatPack variants decode identically — comment stripping is an encoder choice (#188 phase 2).
if (encoding === BgcodeEncoding.MeatPack || encoding === BgcodeEncoding.MeatPackComments) {
throw new ContainerError(
'E_BGCODE_UNSUPPORTED_ENCODING',
'MeatPack encoding is not yet supported (DD-011 phase 2)'
);
return meatpackDecode(bytes, limit);
}
throw new ContainerError('E_BGCODE_UNSUPPORTED_ENCODING', `unknown gcode encoding id ${encoding}`);
}
Expand All @@ -180,7 +179,8 @@ function decode(encoding: number, bytes: Uint8Array): Uint8Array {
* Decode a `.bgcode` v1 buffer to plain G-code. Walks every block, verifies per-block CRC32 (when the
* file declares it), decompresses + decodes the GCode blocks, and concatenates them in file order.
* Non-GCode blocks are walked past (their metadata/thumbnails are surfaced in phase 4). Every failure
* is a structured, bounded {@link ContainerError} — never a crash or an unbounded read/allocation.
* is a structured, bounded `ContainerError` (from `@chestnutlabs/gcode-containers`) — never a crash or
* an unbounded read/allocation.
*/
export async function openBgcode(bytes: Uint8Array, opts: BgcodeDecodeOptions = {}): Promise<BgcodeDecodeResult> {
const limit = opts.maxOutputBytes ?? DEFAULT_MAX_OUTPUT;
Expand Down Expand Up @@ -239,7 +239,7 @@ export async function openBgcode(bytes: Uint8Array, opts: BgcodeDecodeOptions =

if (type === BgcodeBlockType.GCode) {
const decompressed = await decompress(compression, data, uncompressedSize, limit - totalOut);
const decoded = decode(encoding, decompressed);
const decoded = decode(encoding, decompressed, limit - totalOut);
totalOut += decoded.length;
if (totalOut > limit)
throw new ContainerError('E_BGCODE_BOMB', `decoded G-code exceeds the output limit (${totalOut})`);
Expand Down
6 changes: 4 additions & 2 deletions packages/gcode-bgcode/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/**
* @chestnutlabs/gcode-bgcode — Binary G-code (`.bgcode`) decode adapter (DD-011, epic #188).

Check warning on line 2 in packages/gcode-bgcode/src/index.ts

View workflow job for this annotation

GitHub Actions / build

tsdoc-characters-after-block-tag: The token "@ChestnutLabs" looks like a TSDoc tag but contains an invalid character "/"; if it is not a tag, use a backslash to escape the "@"
*
* A license-clean, in-memory decoder for Prusa's binary G-code container. `.bgcode` is a container
* of ordinary G-code, so decoding it (block walk → CRC verify → decompress → decode → concatenate)
Expand All @@ -9,8 +9,9 @@
* `@chestnutlabs/toolpath-core` and `@chestnutlabs/gcode-containers` (for `crc32`/`ContainerError`);
* no `three`, framework, filesystem, or network.
*
* Phase 1: block walker + CRC32 + None/DEFLATE compression + None encoding. MeatPack (phase 2) and
* heatshrink (phase 3) decode to honest, structured "not yet supported" errors.
* Phase 1: block walker + CRC32 + None/DEFLATE compression + None encoding. Phase 2: **MeatPack**
* encoding (both variants). heatshrink compression (phase 3) still decodes to an honest, structured
* "not yet supported" error.
*/
export {
BGCODE_MAGIC,
Expand All @@ -22,3 +23,4 @@
openBgcode
} from './bgcode.js';
export type { BgcodeBlockInfo, BgcodeDecodeResult, BgcodeDecodeOptions } from './bgcode.js';
export { meatpackDecode } from './meatpack.js';
152 changes: 152 additions & 0 deletions packages/gcode-bgcode/src/meatpack.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
/**
* MeatPack G-code decoder (DD-011 phase 2, #188).
*
* A faithful TypeScript reimplementation of the MeatPack unpacker state machine from
* **jamesgopsill/meatpack** (MIT, © 2025 James Gopsill — https://github.com/jamesgopsill/meatpack),
* which is itself derived from the published Prusa spec (`libbgcode/binarize/meatpack.cpp`). Porting
* from an MIT source with attribution is the sanctioned path (RR-003 §8) — the AGPL `libbgcode` and
* OctoPrint-MeatPack are never copied.
*
* MeatPack packs the 15 most common G-code characters into 4-bit nibbles (two per byte); a nibble of
* `0b1111` escapes a full literal byte, and a `0xFF 0xFF <cmd>` sequence toggles packing / no-spaces
* state. The stream begins in the *disabled* state; the encoder emits `FF FF FB` (enable packing) up
* front (the `MEATPACK_HEADER`). The first character of a packed byte is the LOW nibble.
*/
import { ContainerError } from '@chestnutlabs/gcode-containers';

const SIGNAL = 0xff;
// Command bytes (spec §meatpack; the two 0xFF signal bytes precede one of these).
const CMD_NO_SPACES_DISABLED = 246;
const CMD_NO_SPACES_ENABLED = 247;
const CMD_QUERY_CONFIG = 248;
const CMD_RESET_ALL = 249;
const CMD_PACKING_DISABLED = 250;
const CMD_PACKING_ENABLED = 251;

/**
* 4-bit code → byte. Codes 0..14 are the packable characters; `0b1011` is space (or `'E'` when
* no-spaces is active). Code `0b1111` (15) is the full-width escape marker — returned here as `0`,
* which callers treat as "a full literal byte follows".
*/
function reverseLookup(nibble: number, noSpaces: boolean): number {
if (nibble <= 9) return 0x30 + nibble; // '0'..'9'
switch (nibble) {
case 10:
return 0x2e; // '.'
case 11:
return noSpaces ? 0x45 : 0x20; // 'E' or ' '
case 12:
return 0x0a; // '\n'
case 13:
return 0x47; // 'G'
case 14:
return 0x58; // 'X'
default:
return 0; // 0b1111 → NUL: full-width escape marker
}
}

type State = 'disabled' | 'enabled' | 'first' | 'second' | 'rightFull' | 'leftFull';

/**
* Decode a MeatPack byte stream to plain G-code, bounded to `limit` output bytes (decompression-bomb
* defense). Both bgcode GCode encodings (MeatPack, MeatPack-comments) decode through this identically —
* comment preservation is an *encoder* choice; the decoder simply reproduces the packed bytes.
*/
export function meatpackDecode(data: Uint8Array, limit: number): Uint8Array {
let out = new Uint8Array(Math.max(64, Math.min(limit, data.length * 2)));
let pos = 0;
const push = (b: number): void => {
if (pos >= limit) throw new ContainerError('E_BGCODE_BOMB', `MeatPack output exceeds the limit (${pos})`);
if (pos >= out.length) {
const bigger = new Uint8Array(Math.min(limit, Math.max(out.length * 2, pos + 1)));
bigger.set(out.subarray(0, pos));
out = bigger;
}
out[pos++] = b;
};

let state: State = 'disabled';
let noSpaces = false;

for (let i = 0; i < data.length; i++) {
const byte = data[i];
if (byte === SIGNAL) {
if (state === 'first') state = 'second';
else if (state === 'disabled' || state === 'enabled') state = 'first';
else throw new ContainerError('E_BGCODE_MEATPACK', `invalid MeatPack state at byte ${i}`);
continue;
}
switch (state) {
case 'disabled':
push(byte);
break;
case 'enabled': {
const most = reverseLookup(byte >> 4, noSpaces); // high nibble → 2nd char
const least = reverseLookup(byte & 0x0f, noSpaces); // low nibble → 1st char
if (most === 10 && least === 10) {
push(10); // a packed (\n, \n) collapses to a single newline
} else if (most === 0 && least !== 0) {
// 2nd char is full-width (follows in the stream); 1st char is real.
push(least);
state = 'rightFull';
} else if (most !== 0 && least === 0) {
// 1st char is full-width (follows); place a placeholder, then the real 2nd char.
push(least);
push(most);
state = 'leftFull';
} else if (most === 0 && least === 0) {
// byte 0xFF — a signal, intercepted above; never reachable here.
throw new ContainerError('E_BGCODE_MEATPACK', `unexpected 0xFF in packed data at byte ${i}`);
} else {
push(least); // 1st char (low nibble)
push(most); // 2nd char (high nibble)
}
break;
}
case 'second': {
switch (byte) {
case CMD_PACKING_ENABLED:
state = 'enabled';
break;
case CMD_PACKING_DISABLED:
state = 'disabled';
break;
case CMD_RESET_ALL:
state = 'disabled';
noSpaces = false;
break;
case CMD_NO_SPACES_ENABLED:
noSpaces = true;
state = 'enabled';
break;
case CMD_NO_SPACES_DISABLED:
noSpaces = false;
state = 'enabled';
break;
case CMD_QUERY_CONFIG:
// A serial-link query, not expected inside a file — no-op, resume packing.
state = 'enabled';
break;
default:
throw new ContainerError('E_BGCODE_MEATPACK', `invalid MeatPack command ${byte} at byte ${i}`);
}
break;
}
case 'first':
// A lone 0xFF (not a command) → this byte and the next are two full-width literals.
state = 'rightFull';
push(byte);
break;
case 'rightFull':
state = 'enabled';
push(byte);
break;
case 'leftFull':
state = 'enabled';
out[pos - 2] = byte; // overwrite the placeholder with the full-width byte
break;
}
}
return out.subarray(0, pos);
}
3 changes: 3 additions & 0 deletions tools/pack-check-snapshots.json
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@
"dist/index.d.ts",
"dist/index.d.ts.map",
"dist/index.js",
"dist/meatpack.d.ts",
"dist/meatpack.d.ts.map",
"dist/meatpack.js",
"package.json"
],
"gcode-parser": [
Expand Down
Loading