[TS PBT] Build a reproducible property benchmark and mutant corpus

[CaelmBleidd]

Goal

Build a reproducible benchmark of Kotlin-defined TypeScript properties and controlled program mutants for evaluating concrete PBT backends and the hybrid USVM pipeline.

Why

Coverage-only samples are not sufficient to evaluate property-based testing or property-directed symbolic execution.

The paper needs a documented corpus with meaningful properties, known seeded faults, fixed revisions, and reproducible run configurations. The benchmark must be defined before runtime-derived hints are evaluated to avoid selecting cases around the observed results.

The benchmark must use the common Kotlin property model so the same corpus can be projected into fast-check, future PBT backends, and USVM without separate property declarations.

Scope

• Define benchmark selection criteria.
• Include representative property categories:
  • round-trip;
  • idempotence;
  • reference implementation;
  • relational and metamorphic;
  • collection invariants;
  • supported stateful or model-based properties where possible.
• Include both curated minimal fixtures and suitable open-source TypeScript code.
• Record the source repository, revision, license, and selected files for external projects.
• Define domains and TypeScript entry-point references using the common Kotlin property abstraction.
• Keep predicate and precondition implementations in the original TypeScript runtime.
• Record exact, approximate, and unsupported capability for every selected backend and property.
• Add controlled mutants with:
  • mutation description;
  • changed source location;
  • expected violated property;
  • stable mutant identifier.
• Freeze benchmark manifests, dependencies, selected backend IDs and versions, seeds, and default budgets.
• Provide commands for validating and running the corpus through the Kotlin pipeline with a selected PBT backend.
• Keep benchmark construction separate from final experimental conclusions.

Definition of Done

• The benchmark contains at least one validated property from every supported property category.
• Both correct and deliberately mutated implementations are represented.
• Every property has a documented rationale and expected behavior.
• Every external source has a pinned revision and recorded license.
• Every mutant is traceable to one source change and one expected property violation.
• Capability is recorded per backend without silently excluding unsupported cases.
• Benchmark manifests, backend versions, and default run configurations are versioned.
• A validation command checks Kotlin property loading, selected-backend execution, expected fixtures, and artifact structure.
• The benchmark uses the same property definitions consumed by all pipeline modes without a separate fast-check representation.
• The selection protocol and known corpus limitations are documented.
• The corpus and validation tooling are delivered in a dedicated PR linked to this issue.