Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/build-web/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ runs:
if [ "$(wasm-bindgen --version 2>/dev/null | awk '{print $2}')" != "$V" ]; then
cargo install wasm-bindgen-cli --version "$V" --locked --force
fi
- uses: actions/setup-node@v4
- uses: actions/setup-node@v7
with:
node-version: 22
cache: npm
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ jobs:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Fetch the schema fixtures (NeTEx)
- name: Fetch the schema fixtures (NeTEx, OSDM)
shell: bash
run: scripts/fetch-fixtures.sh
- name: Build
Expand All @@ -35,7 +35,7 @@ jobs:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
Expand All @@ -49,7 +49,7 @@ jobs:
name: Web viewer
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: ./.github/actions/build-web
with:
lib: "true"
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
name: Build image
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7

- name: Use the release version
if: github.event_name == 'release'
Expand All @@ -37,21 +37,21 @@ jobs:
# requests and main build amd64 only.
- name: Set up QEMU
if: github.event_name == 'release'
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4

- uses: docker/setup-buildx-action@v3
- uses: docker/setup-buildx-action@v4

- name: Log in to GitHub Container Registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Image tags
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: ${{ env.IMAGE }}
tags: |
Expand All @@ -61,7 +61,7 @@ jobs:
type=raw,value=latest,enable=${{ github.event_name == 'release' }}

- name: Build and push
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
platforms: ${{ github.event_name == 'release' && 'linux/amd64,linux/arm64' || 'linux/amd64' }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: ./.github/actions/build-web
- name: Cache the example downloads
uses: actions/cache@v4
uses: actions/cache@v6
with:
path: tests/fixtures/downloads
key: examples-${{ hashFiles('scripts/build-examples.sh') }}
Expand All @@ -42,7 +42,7 @@ jobs:
run: |
cp THIRD_PARTY_NOTICES.md web/dist/
touch web/dist/.nojekyll
- uses: actions/upload-pages-artifact@v3
- uses: actions/upload-pages-artifact@v5
with:
path: web/dist

Expand All @@ -54,4 +54,4 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@v5
22 changes: 11 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,17 @@ jobs:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Use the release version
run: scripts/set-version.sh "${GITHUB_REF#refs/tags/v}"
- uses: ./.github/actions/build-web
with:
lib: "true"
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@v7
with:
name: web-dist
path: web/dist
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@v7
with:
name: web-dist-lib
path: web/dist-lib
Expand All @@ -46,12 +46,12 @@ jobs:
env:
HAS_CARGO_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN != '' }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Use the release version
run: scripts/set-version.sh "${GITHUB_REF#refs/tags/v}"
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
with:
name: web-dist
path: crates/cli/web-dist
Expand Down Expand Up @@ -86,7 +86,7 @@ jobs:
- { target: aarch64-apple-darwin, os: macos-latest, archive: tar.gz }
- { target: x86_64-pc-windows-msvc, os: windows-latest, archive: zip }
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
Expand All @@ -100,7 +100,7 @@ jobs:
sed -i.bak -E "s/^version = \"[^\"]+\"/version = \"$V\"/" Cargo.toml
sed -i.bak -E "s/(schema-explorer-core = \{ path = \"[^\"]+\", version = \")[^\"]+\"/\1$V\"/" Cargo.toml crates/wasm/Cargo.toml
rm -f Cargo.toml.bak crates/wasm/Cargo.toml.bak
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
with:
name: web-dist
path: web/dist
Expand All @@ -116,7 +116,7 @@ jobs:
if [ "${{ matrix.archive }}" = zip ]; then 7z a "$NAME.zip" "$NAME" >/dev/null; echo "ASSET=$NAME.zip" >> "$GITHUB_ENV"
else tar czf "$NAME.tar.gz" "$NAME"; echo "ASSET=$NAME.tar.gz" >> "$GITHUB_ENV"; fi
- name: Attach to the release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
files: ${{ env.ASSET }}

Expand All @@ -127,14 +127,14 @@ jobs:
permissions:
id-token: write # npm trusted publishing and provenance
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: 22
registry-url: https://registry.npmjs.org
- name: Trusted publishing needs npm 11.5 or later
run: npm install -g npm@latest
- uses: actions/download-artifact@v4
- uses: actions/download-artifact@v8
with:
name: web-dist-lib
path: web/dist-lib
Expand Down
26 changes: 26 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

61 changes: 48 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,42 @@
# schema-explorer

XML Schema (XSD) as UML-like diagrams, readable type pages and version diffs.
XML Schema (XSD) and JSON Schema (also inside OpenAPI and AsyncAPI) as
UML-like diagrams, readable type pages and version diffs.

Large schema sets such as NeTEx or railML are hard to read as hundreds of
`.xsd` files. schema-explorer reads a whole set, follows its imports and
includes, and shows it the way people are used to: class diagrams, content
Large schema sets such as NeTEx, railML or an OpenAPI specification are hard
to read as hundreds of `.xsd`, `.json` or `.yaml` files. schema-explorer
reads a whole set, follows its imports, includes and `$ref`s, and shows it the way people are used to: class diagrams, content
trees, and one page per type and element with every child, attribute,
cardinality and allowed value. It also compares two versions of a set and
says what changed and what breaks. It is the schema counterpart of
[ontology-explorer](https://github.com/Matdata-eu/ontology-explorer).

It runs entirely in the browser, so it can be published on GitHub Pages, and
a small command-line tool pre-indexes large sets, serves the viewer locally
and writes diff reports. JSON Schema support is planned.
and writes diff reports.

**Try it:** <https://matdata-eu.github.io/schema-explorer/> with NeTEx,
ERA TAF and TAP TSI, OASIS UBL 2.4 and the W3C schema for schemas, or drop
your own files on it.
ERA TAF and TAP TSI, OASIS UBL 2.4, the W3C schema for schemas, the UIC OSDM
OpenAPI specification and the JSON Schema 2020-12 meta-schemas, or drop your
own files on it.

![The page of a NeTEx complex type: derivation chain and class diagram](docs/screenshot-type.png)

## Features

- **Open** `.xsd` files or whole folders by drag and drop or the file picker:
- **XML Schema** 1.0 (tolerating 1.1 constructs) and **JSON Schema**
draft-04, -06, -07, 2019-09 and 2020-12 (picked by `$schema`), in JSON or
YAML, including the schemas of **OpenAPI** 3.0/3.1 (`components.schemas`)
and **AsyncAPI** 2/3 documents (schemas and message payloads). JSON Schema
maps onto the same model and is shown in JSON terms: named schemas
(`$defs`, `definitions`, components) are object or scalar definitions,
properties have a cardinality from `required` and `minItems`/`maxItems`,
a `$ref` in `allOf` is a generalisation, `oneOf`/`anyOf` are choices,
`enum`/`const` are enumerations, `format`, `pattern`, `minimum` and the
other validation keywords are facets, and `patternProperties` and
`additionalProperties` are listed. `$ref` is resolved across files, JSON
pointers, `$id` and `$anchor`.
- **Open** `.xsd`, `.json`, `.yaml` files or whole folders by drag and drop or the file picker:
they are indexed in the browser (WebAssembly, in a worker), nothing is
uploaded. Or open a pre-built bundle by its address (`?bundle=<url>`), or
pick an example.
Expand All @@ -35,7 +49,9 @@ your own files on it.
`sequence` / `choice` / `all` and groups, inherited rows grouped per base
type and foldable), attributes (type, use, default or fixed), facets and
enumerations with their documentation, identity constraints, where used,
the source file and line, and the XSD of the component with linked names.
the source file and line, and the XSD of the component with linked names
(for JSON Schema: the component's schema as JSON, with `$ref`s linked,
its examples and deprecated/read-only flags).
- **Element pages** (global and local): type, substitution group head and
members, nillable, abstract, default or fixed, where used.
- **Simple type pages**: restriction facets, enumeration table, list item and
Expand Down Expand Up @@ -64,6 +80,7 @@ your own files on it.

```sh
schema-explorer serve path/to/schemas/ # every .xsd file in the folder
schema-explorer serve openapi.yaml # JSON Schema, OpenAPI, AsyncAPI
schema-explorer serve main.xsd --bind 0.0.0.0:8080 --no-open
```

Expand All @@ -78,17 +95,25 @@ release. Until then, build from source with `cargo build --release -p schema-exp

## Indexing a schema set

The core reads an XSD set (XSD 1.0, tolerating 1.1 constructs) into one JSON
bundle for the viewer: every global and local component keyed by qualified
The core reads an XSD set (XSD 1.0, tolerating 1.1 constructs) or a JSON
Schema set (JSON Schema, OpenAPI or AsyncAPI files) into one JSON bundle for the viewer: every global and local component keyed by qualified
name and source file and line, with documentation, references resolved to
ids, "where used" back-references, derived types, substitution group
members, effective content models and a search index.

```sh
schema-explorer index path/to/NeTEx_publication.xsd -o bundle.json
schema-explorer index schemas/ --compact -o bundle.json # directories are read recursively
schema-explorer index OSDM-online-api.yml -o bundle.json # OpenAPI with its schema files
```

A directory gives its `.xsd` files, or every `.json`, `.yaml` and `.yml`
file when it has no `.xsd` file. JSON Schema bundles say
`"language": "jsonSchema"` and carry a `json` block per component (its JSON
kind, JSON pointer, title, examples, flags); `$ref`s to other files are
followed from disk, and a `$ref` that cannot be resolved becomes an
`unresolved-ref` diagnostic.

Imports and includes are followed from disk; a location that cannot be
found falls back to an input file with the imported namespace or the same
file name, and otherwise becomes an `unresolved-import` diagnostic in the
Expand Down Expand Up @@ -116,8 +141,14 @@ turns that off).
schema-explorer diff railml-3.2/schema/ railml-3.3/schema/ --format md -o CHANGES.md
schema-explorer diff old.json new.json -o diff.json # bundles work too
schema-explorer diff a.xsd,b.xsd c.xsd,d.xsd --format html -o diff.html --fail-on-breaking
schema-explorer diff api-v1.yaml api-v2.yaml --format md # JSON Schema / OpenAPI
```

For JSON Schema, a property that becomes required, a removed property or
enum value, a new `additionalProperties: false`, a tighter `maximum` or
`maxItems`, or a dropped `nullable` are breaking; property order is not
compared.

Several paths for one side are joined with commas. The JSON diff format is
versioned (`format`, see `crates/core/src/diff.rs`). In the viewer, "Compare
two versions" on the open page takes two schema sets; a link
Expand All @@ -139,14 +170,17 @@ and changed (~, amber).
`scripts/build-examples.sh <cli> <dist>` shows how the website is built: it
pre-indexes a schema set into `bundle.json` and puts its `.xsd` files next
to it (for the source view). Any static host works:
`index.html?bundle=examples/netex/bundle.json&sources=examples/netex/xsd/`.
`index.html?bundle=examples/netex/bundle.json&sources=examples/netex/xsd/`
(JSON Schema examples keep their files under `schema/`).
Only publish schemas whose licence allows it.

The website's examples are fetched from their publishers at build time and
keep their own terms: NeTEx (GPL-3.0, NeTEx-CEN); TAF TSI and TAP TSI B.10
schemas, source: European Union Agency for Railways (reproduction authorised,
source acknowledged); UBL 2.4, copyright © OASIS Open 2024 (OASIS IPR
Policy); XMLSchema.xsd and xml.xsd, copyright © W3C (W3C Document License).
Policy); XMLSchema.xsd and xml.xsd, copyright © W3C (W3C Document License);
OSDM, UIC (Apache-2.0); JSON Schema 2020-12 meta-schemas, copyright © 2022
JSON Schema Specification Authors (BSD-3-Clause).
See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).

## Development
Expand All @@ -163,6 +197,7 @@ cd web && npm ci && npm run build
Tests on real-world schemas read them from the directory in `SCHEMA_FIXTURES`
(default `tests/fixtures`, gitignored) and are skipped when absent.
`scripts/fetch-fixtures.sh` fetches the public [NeTEx](https://github.com/NeTEx-CEN/NeTEx)
and [OSDM](https://github.com/UnionInternationalCheminsdeFer/OSDM) (OpenAPI)
schemas there; the tests also look for `railml/<version>/{schema,codelists}`
and `telematics/{netex,taf,taf-v3.5.1,b5,b10}`. For timings, run
`cargo test --release --test fixtures -- --nocapture`. Schemas whose licence forbids redistribution (such as railML)
Expand Down
6 changes: 6 additions & 0 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,9 @@ they are not part of this repository or the binary:
with <https://www.w3.org/2001/xml.xsd>. Copyright © World Wide Web
Consortium. Used under the
[W3C Document License](https://www.w3.org/copyright/document-license-2023/).
- **OSDM**, the UIC Open Sales and Distribution Model specification (OpenAPI
and JSON schemas), from
<https://github.com/UnionInternationalCheminsdeFer/OSDM>, Apache-2.0.
- **JSON Schema 2020-12** meta-schemas, <https://json-schema.org/draft/2020-12>.
Copyright © 2022 JSON Schema Specification Authors. BSD-3-Clause
(<https://github.com/json-schema-org/json-schema-spec>).
Loading
Loading