Skip to content

Suggestion: add pydantic versioned schema migration + base DLC schemas #48

Description

@C-Achard

This is a reminder/centralized post regarding where to host DLC config/project schemas and related tools.

Several of our libraries in the software stack could benefit from having access to:

  • DLC project layout/file structure schemas
  • DLC config schemas (available in main DLC)
  • Versioned schema migration code (available in main DLC)
  • DLC keypoint/data spec schema(s)

and potentially other pieces of code.

Criteria for inclusion in DLCLibrary (requires discussion, this is an outline):

  • Lighweight dependencies
    • No core DLC
  • Useful to at least two DLC repos in a concrete way
    • e.g. repo-specific migration logic should be hosted at the lowest stage where needed only
    • Adding code "for later" should be strictly documented and motivated, and ideally have an associated WIP PR in involved repos
  • Should have one central domain-correct behavior
    • Centralizing prevents incompatible interpretations across repositories
    • DLCLibrary should be a source of truth, not a copy
    • Existing implementations should migrate to importing it, adding here should have corresponding migration logic ready downstream ideally
    • Must not become a drawer of loosely useful helpers that may be repo specific/general-purpose code
    • In general adding code here should be heavily scrutinized, only the bare required minimum should be accepted
  • Testing can also be done in this repo on a different level:
    • Public schemas and migration behavior can be versioned and tested
    • Breaking changes require an explicit release and migration path
    • Round-trip and compatibility tests
    • Unknown or newer schema versions must fail clearly if broken/problematic

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementNew feature or request

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions