Skip to content

Build bulk response rows without validating each through Pydantic #915

Open

Description

@bencap

Summary

Whole-set endpoints build each row as a dict and then validate it through a Pydantic view model before serializing, for example VariantDetail.model_validate(detail) in the /variant-details stream. On a 500k-variant score set that validation runs half a million times on data the server just built. This issue serializes the built rows directly and moves the shape guarantee into tests.

Scope

  • Serialize rows straight to JSON in the /variant-details stream (_stream_score_set_variant_details, routers/score_sets.py), the lean view (get_lean_score_set_variants), the /annotated-variants/* streams, and wherever else the profiling shows validation cost.
  • Keep the Pydantic models as the OpenAPI schema and as the single-object routes' response models.
  • Add a test per endpoint that validates every emitted row against its model on a seeded score set.

Acceptance criteria

  • The listed endpoints don't call model_validate per row.
  • Each endpoint's output validates against its view model in tests.
  • Each endpoint's output is byte-identical to the validated output on the test score set.
  • Time per 10k rows drops by at least the validation share measured in the profiling issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backend

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions