Skip to content

Conversation

@mattpodwysocki
Copy link
Contributor

Description

Makes output schema validation non-fatal for tools, matching the pattern established in
mapbox/mcp-server#74.

Changes

Added validateOutput helper to BaseTool (src/tools/BaseTool.ts:111-132)

  • New protected method that validates output data against schemas
  • Logs warnings instead of throwing errors when validation fails
  • Returns raw data as graceful fallback, allowing tools to continue functioning

Updated 3 tools to use graceful validation:

  • ListStylesTool - Replaced error-returning safeParse with validateOutput helper
  • CreateTokenTool - Replaced error-returning safeParse with validateOutput helper
  • ListTokensTool - Replaced error-returning safeParse with validateOutput helper

Impact

Before: When API responses deviated from expected schemas (but were still valid), tools would
fail with schema validation errors.

After: Tools log warnings and continue operating with raw data, improving resilience while
maintaining debuggability through logged validation issues.

Key Benefits

Non-fatal validation - Schema mismatches log warnings instead of failing tool operations
Better resilience - Tools work even when API responses change slightly
Maintained debuggability - Validation failures still logged at warning level
Consistent pattern - All tools follow the same graceful validation approach

Tests

New Unit Tests

Added 3 unit tests verifying graceful schema validation fallback:

  1. ListStylesTool (test/tools/list-styles-tool/ListStylesTool.test.ts:271-313)

    • Verifies tool handles missing required fields (owner, created, modified)
    • Confirms warning is logged with "Output schema validation failed"
    • Ensures raw data is returned including unexpected fields
  2. CreateTokenTool (test/tools/create-token-tool/CreateTokenTool.test.ts:366-402)

    • Verifies tool handles missing required fields (id, created, modified)
    • Confirms warning is logged with "Output schema validation failed"
    • Ensures raw data is returned including unexpected fields
  3. ListTokensTool (test/tools/list-tokens-tool/ListTokensTool.test.ts:568-605)

    • Verifies tool handles missing required fields (usage, client, scopes)
    • Confirms warning is logged with "Output schema validation failed"
    • Ensures raw data is returned including unexpected fields

Test Results

✅ All 286 tests passing (283 original + 3 new)
✅ Build successful
✅ No new linting issues

Each test verifies three critical behaviors:

  1. ✅ No errors thrown (graceful degradation)
  2. ✅ Warning logged for debugging
  3. ✅ Raw data returned successfully

Checklist

  • Code has been tested locally
  • Unit tests have been added or updated
  • Documentation has been updated if needed

Additional Notes

@mattpodwysocki mattpodwysocki requested a review from a team as a code owner November 24, 2025 12:51
@mattpodwysocki mattpodwysocki merged commit 67099df into main Nov 24, 2025
1 check passed
@mattpodwysocki mattpodwysocki deleted the schema_fixes branch November 24, 2025 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants