BQ to SquashFS download pipeline with test suite#79

Open

mohamedelabbas1996 wants to merge 26 commits into

mainfrom

feature/bq-squashfs-pipeline

Commits on May 1, 2026

Add BQ/SquashFS pipeline scripts and documentation

mohamedelabbas1996
and
claude
committed

Commits on Jun 4, 2026

test: add shared conftest.py with fixtures for all pipeline stage tests

mohamedelabbas1996
and
claude
committed
docs(download): improve --help descriptions for all arguments

mohamedelabbas1996
and
claude
committed
test(download): add multi-task distribution and merge tests

mohamedelabbas1996
and
claude
committed
feat(pack): add stream_chunks_to_tar.py with 18 tests

mohamedelabbas1996
and
claude
committed
fix(pack): handle BrokenPipeError cleanly when sqfstar is OOM killed

mohamedelabbas1996
and
claude
committed
feat(pack): harden stream_chunks_to_tar against all real failure modes

mohamedelabbas1996
and
claude
committed
feat(pack): rename stream_chunks_to_tar → merge_sqfs_chunks, full hardening

mohamedelabbas1996
and
claude
committed
test(pack): rename and update tests for merge_sqfs_chunks.py

mohamedelabbas1996
and
claude
committed
feat(download): merge failed status into training_images

mohamedelabbas1996
and
claude
committed
feat(pack): add job_bq_pack_per_task.sh with verify-then-delete

mohamedelabbas1996
and
claude
committed
Add BQ training_images dedup clean script

mohamedelabbas1996
and
claude
committed
Add test suite for clean.py (68 tests)

mohamedelabbas1996
and
claude
committed

Commits on Jun 11, 2026