Missing `spec/` Directory and `generate_spec.sh` Script

[dcguim]

Summary

The sdk/python/generate_models.sh and generate_ts_schema_types.js scripts both expect a spec/ directory containing pre-resolved JSON schemas. However:

1. spec/ is gitignored (line 3 of .gitignore)
2. No script exists to generate spec/ from source/schemas/
3. Running the generators fails with FileNotFoundError: /path/to/spec

Background

UCP schemas in source/schemas/ contain ucp_request and ucp_response annotations that control field visibility per direction and operation. These annotations must be resolved before generating SDK models.

Per CONTRIBUTING.md:

"Schemas live in source/ and are published with ucp_* annotations intact. Agents use ucp-schema to resolve annotations for specific operations at runtime."

Current State of Scripts

Script	Input	Output	Status
main.py (MkDocs)	source/schemas/	Resolves at runtime	Works - calls ucp-schema resolve on-demand
generate_ts_schema_types.js	spec/	generated/schema-types.ts	Broken - spec/ doesn't exist
sdk/python/generate_models.sh	../../spec/	src/ucp_sdk/models/	Broken - spec/ doesn't exist

How main.py Resolves Schemas (lines 88-137)

cmd = [
    "ucp-schema",
    "resolve",
    str(schema_path),
    "--request" if direction == "request" else "--response",
    "--op",
    operation,
]
if bundle:
    cmd.append("--bundle")

result = subprocess.run(cmd, capture_output=True, text=True, check=False)

This works for documentation because it resolves at runtime. But the SDK generators need pre-resolved schemas in spec/.

Missing Models

Comparing source/schemas/shopping/ with models/schemas/shopping/:

UCP Schema	Models Generated
cart.json	None - completely missing

All other schemas (checkout.json, discount.json, fulfillment.json, buyer_consent.json, ap2_mandate.json, order.json, payment.json) have corresponding models.

Proposed Solution: generate_spec.sh

Create a script that populates spec/ from source/schemas/ using ucp-schema resolve. This script should:

1. Resolve annotated schemas for all directions/operations
2. Bundle $ref pointers for self-contained schemas
3. Mirror the structure expected by existing generators

Suggested Implementation

#!/bin/bash
# generate_spec.sh - Generate resolved schemas into spec/ directory
#
# This script uses ucp-schema to resolve UCP annotations from source/schemas/
# and outputs self-contained JSON schemas to spec/ for SDK generators.

set -e

cd "$(dirname "$0")"

# Directories
SOURCE_DIR="source/schemas"
SPEC_DIR="spec"

# Schemas with UCP annotations
ANNOTATED_SCHEMAS=(
    "shopping/checkout.json"
    "shopping/cart.json"
    "shopping/discount.json"
    "shopping/fulfillment.json"
    "shopping/ap2_mandate.json"
    "shopping/buyer_consent.json"
)

# Schemas without UCP annotations (just bundle refs)
PLAIN_SCHEMAS=(
    "shopping/order.json"
    "shopping/payment.json"
)

# Operations for request direction (matching existing models)
REQUEST_OPS=("create" "update")

# Check for ucp-schema
if ! command -v ucp-schema &> /dev/null; then
    echo "Error: ucp-schema not found."
    echo "Install with: cargo install ucp-schema"
    exit 1
fi

# Setup directories
echo "Setting up spec/ directory..."
rm -rf "$SPEC_DIR"
mkdir -p "$SPEC_DIR/schemas/shopping"
mkdir -p "$SPEC_DIR/handlers"

# Resolve annotated schemas
echo "Resolving annotated schemas..."
for schema in "${ANNOTATED_SCHEMAS[@]}"; do
    basename=$(basename "$schema" .json)
    echo "  Processing $basename..."

    # Response schema (read operation)
    ucp-schema resolve "$SOURCE_DIR/$schema" \
        --response --op read --bundle --pretty \
        > "$SPEC_DIR/schemas/shopping/${basename}_resp.json"

    # Request schemas for each operation
    for op in "${REQUEST_OPS[@]}"; do
        ucp-schema resolve "$SOURCE_DIR/$schema" \
            --request --op "$op" --bundle --pretty \
            > "$SPEC_DIR/schemas/shopping/${basename}_${op}_req.json"
    done
done

# Bundle plain schemas
echo "Bundling plain schemas..."
for schema in "${PLAIN_SCHEMAS[@]}"; do
    basename=$(basename "$schema" .json)
    echo "  Bundling $basename..."
    ucp-schema resolve "$SOURCE_DIR/$schema" \
        --response --op read --bundle --pretty \
        > "$SPEC_DIR/schemas/shopping/${basename}.json"
done

# Copy types (no resolution needed)
echo "Copying type schemas..."
mkdir -p "$SPEC_DIR/schemas/shopping/types"
cp "$SOURCE_DIR/shopping/types/"*.json "$SPEC_DIR/schemas/shopping/types/"

echo ""
echo "Done! Resolved schemas written to $SPEC_DIR/"
find "$SPEC_DIR" -name "*.json" | wc -l | xargs echo "Total JSON files:"

Updated Workflow

After adding generate_spec.sh:

source/schemas/  --(generate_spec.sh)-->  spec/  --(generate_models.sh)-->  models/
                         |                  |
                         |                  +--(generate_ts_schema_types.js)--> generated/
                         |
                   uses ucp-schema resolve

CI Integration

Add to .github/workflows/docs.yml or create a new workflow:

- name: Generate spec/ from source/
  run: ./generate_spec.sh

- name: Generate Python models
  run: bash sdk/python/generate_models.sh

- name: Generate TypeScript types
  run: node generate_ts_schema_types.js

Previous Attempt

I attempted to create a workaround script at /Users/dguim/localwork/ucp/generate_pydantic_models.sh that:

1. Used ucp-schema resolve to generate resolved schemas
2. Ran datamodel-code-generator on the output

However, this produced problematic Pydantic models. For example, in the generated response/checkout.py:

class Instrument(Checkout):
    """A payment instrument with selection state."""

    model_config = ConfigDict(
        extra="allow",
    )
    selected: bool | None = None

This incorrectly has Instrument inheriting from Checkout, which doesn't make semantic sense - a payment instrument should not be a full checkout object. This appears to be an artifact of how datamodel-code-generator interprets the allOf composition in the bundled JSON Schema.

This suggests that either:

1. The schema bundling strategy needs adjustment
2. datamodel-code-generator options need tuning
3. The spec/ structure should be different from what I attempted

Questions for Maintainers

1. Is this the intended approach? Should spec/ be generated from source/ using ucp-schema?

2. Naming convention: Should resolved schemas use {name}_resp.json / {name}_{op}_req.json format, or a different structure?

3. Why is Cart missing? cart.json exists in source/schemas/shopping/ but has no generated models.

4. Handler schemas: Should source/handlers/ also be resolved into spec/handlers/?

Related Files

• .gitignore line 3: /spec/
• main.py lines 88-137: Runtime resolution for docs
• generate_ts_schema_types.js line 5: SOURCE_ROOT = path.resolve(__dirname, 'spec')
• sdk/python/generate_models.sh line 11: SCHEMA_DIR="../../spec/"
• ucp-schema CLI: https://github.com/universal-commerce-protocol/ucp-schema