Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .claude/settings.local.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"permissions": {
"allow": ["Bash(npx vitest:*)", "Bash(bash:*)", "Bash(node test-debug.js:*)"]
},
"enableAllProjectMcpServers": false
}
662 changes: 331 additions & 331 deletions .yarn/releases/yarn-4.9.4.cjs → .yarn/releases/yarn-4.11.0.cjs

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion .yarnrc.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
nodeLinker: node-modules

yarnPath: .yarn/releases/yarn-4.9.4.cjs
yarnPath: .yarn/releases/yarn-4.11.0.cjs
152 changes: 152 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Development Commands

### Building
```bash
yarn build # Compile TypeScript to dist/
yarn typecheck # Run TypeScript type checking without emitting files
```

### Testing
```bash
yarn test # Run all tests with coverage (Vitest)
yarn test:node # Run tests in Node.js environment
yarn test:browser # Run tests in browser environment (jsdom)
yarn test:watch # Run tests in watch mode
yarn test:update # Update test snapshots
```

### Code Quality
```bash
yarn lint # Run ESLint on lib/ directory
yarn prettier # Format all code files
```

### Running Individual Tests
To run a single test file:
```bash
npx vitest test/specs/circular/circular.spec.ts
```

To run tests matching a pattern:
```bash
npx vitest --grep "circular"
```

## Architecture Overview

### Core Purpose
This library parses, resolves, and dereferences JSON Schema `$ref` pointers. It handles references to:
- External files (local filesystem)
- HTTP/HTTPS URLs
- Internal JSON pointers within schemas
- Mixed JSON and YAML formats
- Circular/recursive references

### Key Architecture Components

#### 1. $RefParser (lib/index.ts)
The main entry point and orchestrator class. Provides four primary operations:
- **parse()**: Reads a single schema file (JSON/YAML) without resolving references
- **resolve()**: Parses and resolves all `$ref` pointers, returns a `$Refs` object mapping references to values
- **bundle()**: Converts external `$ref` pointers to internal ones (single file output)
- **dereference()**: Replaces all `$ref` pointers with their actual values (fully expanded schema)

All methods support both callback and Promise-based APIs, with multiple overload signatures.

#### 2. $Refs (lib/refs.ts)
A map/registry of all resolved JSON references and their values. Tracks:
- All file paths/URLs encountered
- Circular reference detection
- Helper methods to query references by type (file, http, etc.)

#### 3. Pointer (lib/pointer.ts)
Represents a single JSON pointer (`#/definitions/person`) and implements JSON Pointer RFC 6901 spec:
- Parses JSON pointer syntax (`/`, `~0`, `~1` escaping)
- Resolves pointers to actual values within objects
- Handles edge cases (null values, missing properties)

#### 4. $Ref (lib/ref.ts)
Wraps a single reference with metadata:
- The reference path/URL
- The resolved value
- Path type (file, http, etc.)
- Error information (when continueOnError is enabled)

#### 5. Plugin System
Two types of plugins, both configurable via options:

**Parsers** (lib/parsers/):
- JSON parser (json.ts)
- YAML parser (yaml.ts) - uses js-yaml
- Text parser (text.ts)
- Binary parser (binary.ts)
- Execute in order based on `order` property and `canParse()` matching

**Resolvers** (lib/resolvers/):
- File resolver (file.ts) - reads from filesystem (Node.js only)
- HTTP resolver (http.ts) - fetches from URLs using native fetch
- Custom resolvers can be added via options
- Execute in order based on `order` property and `canRead()` matching

#### 6. Core Operations

**lib/parse.ts**: Entry point for parsing a single schema file
**lib/resolve-external.ts**: Crawls schema to find and resolve external `$ref` pointers
**lib/bundle.ts**: Replaces external refs with internal refs
**lib/dereference.ts**: Replaces all `$ref` pointers with actual values, handles circular references

#### 7. Options System (lib/options.ts)
Hierarchical configuration with defaults for:
- Which parsers/resolvers to enable
- Circular reference handling (boolean or "ignore")
- External reference resolution (relative vs root)
- Continue on error mode (collect all errors vs fail fast)
- Bundle/dereference callbacks and matchers
- Input mutation control (mutateInputSchema)

### Key Design Patterns

1. **Flexible Arguments**: normalizeArgs() (lib/normalize-args.ts) unifies various call signatures into consistent internal format

2. **Path Handling**: Automatic conversion between filesystem paths and file:// URLs. Cross-platform support via util/url.ts and util/convert-path-to-posix.ts

3. **Error Handling**:
- Fail-fast by default
- Optional continueOnError mode collects errors in JSONParserErrorGroup
- Specific error types: JSONParserError, InvalidPointerError, MissingPointerError, ResolverError, ParserError

4. **Circular Reference Management**:
- Detected during dereference
- Can throw error, ignore, or handle via dereference.circular option
- Reference equality maintained (same `$ref` → same object instance)

5. **Browser/Node Compatibility**:
- Uses native fetch (requires Node 18+)
- File resolver disabled in browser builds (package.json browser field)
- Tests run in both environments

## Testing Strategy

Tests are organized in test/specs/ by scenario:
- Each scenario has test files (*.spec.ts) and fixture data
- Tests validate parse, resolve, bundle, and dereference operations
- Extensive coverage of edge cases: circular refs, deep nesting, special characters in paths
- Browser-specific tests use test/fixtures/server.ts for HTTP mocking

Test utilities:
- test/utils/helper.js: Common test patterns
- test/utils/path.js: Path handling for cross-platform tests
- test/utils/serializeJson.ts: Custom snapshot serializer

## Important Constraints

1. **TypeScript Strict Mode**: Project uses strict TypeScript including exactOptionalPropertyTypes
2. **JSON Schema Support**: Compatible with JSON Schema v4, v6, and v7
3. **Minimum Node Version**: Requires Node >= 20 (for native fetch support)
4. **Circular JSON**: Dereferenced schemas may contain circular references (not JSON.stringify safe)
5. **Path Normalization**: Always converts filesystem paths to POSIX format internally
6. **URL Safety**: HTTP resolver has safeUrlResolver option to block internal URLs (default: unsafe allowed)
4 changes: 0 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,15 @@ format. Maybe some of the files contain cross-references to each other.
{
"definitions": {
"person": {
// references an external file
"$ref": "schemas/people/Bruce-Wayne.json"
},
"place": {
// references a sub-schema in an external file
"$ref": "schemas/places.yaml#/definitions/Gotham-City"
},
"thing": {
// references a URL
"$ref": "http://wayne-enterprises.com/things/batmobile"
},
"color": {
// references a value in an external file via an internal reference
"$ref": "#/definitions/thing/properties/colors/black-as-the-night"
}
}
Expand Down
21 changes: 12 additions & 9 deletions lib/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ import $Ref from "./ref.js";
import Pointer from "./pointer.js";
import * as url from "./util/url.js";
import type $Refs from "./refs.js";
import type $RefParser from "./index";
import type { ParserOptions } from "./index";
import type { JSONSchema } from "./index";
import type { BundleOptions } from "./options";
import type $RefParser from "./index.js";
import type { ParserOptions } from "./index.js";
import type { JSONSchema } from "./index.js";
import type { BundleOptions } from "./options.js";

export interface InventoryEntry {
$ref: any;
Expand Down Expand Up @@ -40,7 +40,7 @@ function bundle<S extends object = JSONSchema, O extends ParserOptions<S> = Pars
crawl<S, O>(parser, "schema", parser.$refs._root$Ref.path + "#", "#", 0, inventory, parser.$refs, options);

// Remap all $ref pointers
remap(inventory);
remap<S, O>(inventory, options);
}

/**
Expand Down Expand Up @@ -203,7 +203,10 @@ function inventory$Ref<S extends object = JSONSchema, O extends ParserOptions<S>
*
* @param inventory
*/
function remap(inventory: InventoryEntry[]) {
function remap<S extends object = JSONSchema, O extends ParserOptions<S> = ParserOptions<S>>(
inventory: InventoryEntry[],
options: O,
) {
// Group & sort all the $ref pointers, so they're in the order that we need to dereference/remap them
inventory.sort((a: InventoryEntry, b: InventoryEntry) => {
if (a.file !== b.file) {
Expand Down Expand Up @@ -254,10 +257,10 @@ function remap(inventory: InventoryEntry[]) {
// This $ref already resolves to the main JSON Schema file
entry.$ref.$ref = entry.hash;
} else if (entry.file === file && entry.hash === hash) {
// This $ref points to the same value as the prevous $ref, so remap it to the same path
// This $ref points to the same value as the previous $ref, so remap it to the same path
entry.$ref.$ref = pathFromRoot;
} else if (entry.file === file && entry.hash.indexOf(hash + "/") === 0) {
// This $ref points to a sub-value of the prevous $ref, so remap it beneath that path
// This $ref points to a sub-value of the previous $ref, so remap it beneath that path
entry.$ref.$ref = Pointer.join(pathFromRoot, Pointer.parse(entry.hash.replace(hash, "#")));
} else {
// We've moved to a new file or new hash
Expand All @@ -267,7 +270,7 @@ function remap(inventory: InventoryEntry[]) {

// This is the first $ref to point to this value, so dereference the value.
// Any other $refs that point to the same value will point to this $ref instead
entry.$ref = entry.parent[entry.key] = $Ref.dereference(entry.$ref, entry.value);
entry.$ref = entry.parent[entry.key] = $Ref.dereference(entry.$ref, entry.value, options);

if (entry.circular) {
// This $ref points to itself
Expand Down
7 changes: 3 additions & 4 deletions lib/dereference.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@ import Pointer from "./pointer.js";
import * as url from "./util/url.js";
import type $Refs from "./refs.js";
import type { DereferenceOptions, ParserOptions } from "./options.js";
import type { JSONSchema } from "./types";
import type $RefParser from "./index";
import { TimeoutError } from "./util/errors";
import { type $RefParser, type JSONSchema } from "./index.js";
import { TimeoutError } from "./util/errors.js";

export default dereference;

Expand Down Expand Up @@ -278,7 +277,7 @@ function dereference$Ref<S extends object = JSONSchema, O extends ParserOptions<
}

// Dereference the JSON reference
let dereferencedValue = $Ref.dereference($ref, pointer.value);
let dereferencedValue = $Ref.dereference($ref, pointer.value, options);

// Crawl the dereferenced value (unless it's circular)
if (!circular) {
Expand Down
2 changes: 1 addition & 1 deletion lib/normalize-args.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import type { Options, ParserOptions } from "./options.js";
import { getNewOptions } from "./options.js";
import type { JSONSchema, SchemaCallback } from "./types";
import type { JSONSchema, SchemaCallback } from "./index.js";

// I really dislike this function and the way it's written. It's not clear what it's doing, and it's way too flexible
// In the future, I'd like to deprecate the api and accept only named parameters in index.ts
Expand Down
8 changes: 8 additions & 0 deletions lib/options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ export interface DereferenceOptions {
* Default: `relative`
*/
externalReferenceResolution?: "relative" | "root";

/**
* Whether duplicate keys should be merged when dereferencing objects.
*
* Default: `true`
*/
mergeKeys?: boolean;
}

/**
Expand Down Expand Up @@ -229,6 +236,7 @@ export const getJsonSchemaRefParserDefaultOptions = () => {
*/
excludedPathMatcher: () => false,
referenceResolution: "relative",
mergeKeys: true,
},

mutateInputSchema: true,
Expand Down
Loading
Loading