Skip to content

Add selective mirror mode with SBOM input #20

@andrew

Description

@andrew

The proxy currently operates in a lazy caching mode where packages are only fetched when a client requests them. This means the proxy always needs upstream connectivity to serve metadata, and artifacts are only cached after first access.

A mirror mode would let you pre-populate the cache from an SBOM so the proxy can serve everything a build needs without hitting upstream.

How it would work

Run proxy mirror sbom.json where the input is a CycloneDX or SPDX SBOM. The command parses PURLs from the SBOM, then for each package/version:

  1. Fetches metadata from upstream and stores it locally
  2. Downloads the artifact and caches it (skipping already-cached artifacts)
  3. Reports progress as it goes

CycloneDX is the easier format since components carry PURLs directly. SPDX would need external reference parsing.

Metadata caching

This is the part that doesn't exist today. The database tracks packages/versions/artifacts but not raw metadata responses. Handlers always fetch metadata live from upstream.

For mirror mode to be useful, cached metadata needs to be stored and servable. The simplest approach is storing raw metadata blobs in storage alongside artifacts (e.g. npm/_metadata/lodash.json, cargo/_metadata/serde.json). Handlers would check for cached metadata before hitting upstream, or fall back to it when upstream is unreachable.

Starting point

One-shot command only. Periodic sync, lockfile parsing, or other input formats can come later.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions