Reduce repeated table metadata preparation during inference

[autumnust]

Problem identified

Table preprocessing repeatedly rebuilds column-location dictionaries for identical schemas. Numeric category alignment also sorts and searches vocabularies one column at a time, uses a dynamically sized nonzero result, and remaps codes through separate per-column operations. These costs recur in ordinary fitted prediction pipelines, especially across wide tables and ensemble preprocessing.

Current status

This change is stacked on PR #1168, branch codex/autoresearch-query-scaling-preparation at ba793705be3cbeb25a6600f4e4bacf4ea38d2c96. That parent fuses query scaling and includes #1152's fitted-cache pinning fix. The table-metadata costs remain in this parent. All reported gains are incremental against it; model weights, precision, estimator count, recipe, data and benchmark settings are unchanged.

Solution

Reuse column-location preparation through a bounded cache of 128 immutable schema keys. The cache retains no tensors or fitted model state; every table receives its own dictionary, and constructor validation remains intact.

Batch numeric vocabulary lookups by compatible widths, dtypes and devices. Replace dynamic nonzero indexing with fixed-shape gather/where operations, then remap compatible columns together. The existing memory-sizing utility bounds temporary batches. String-category handling remains on its existing path.

Both parts reduce repeated table-metadata preparation through public SDM operations. Compatibility grouping follows the operation's requirements, with no benchmark identifiers or special-case dataset dimensions. Each constituent was measured independently before evaluating this combination; the combined gain is measured directly, not inferred by adding percentages.

All 440 relevant public checks pass, with three optional-dependency skips, and all pre-commit hooks pass. Coverage includes independent vocabularies, missing/unknown codes, constrained temporary-memory batches, GPU synchronization checks, schema ownership, column ordering, constructor errors and compiled preprocessing. Direct component checks also exercise independent CPU/CUDA table widths with the schema cache cleared before each complete operation.

Quantifiable improvement

3.87% lower aggregate prediction latency across six cases, including 5.08% on the initial four-case suite. Three complete alternating pairs were run per case. All 18 full prediction arrays and official metric errors match exactly; every observed prediction pair is faster. Polish Companies Bankruptcy is effectively flat, so this does not establish a useful speedup for every workload.

Kumo-Tabular Large v1.0.1, 16 estimators, FP16 autocast, one RTX PRO 6000 Blackwell Server Edition GPU and CPUs 0–1. Unprofiled measurements use the upstream validation → cached refit → formal prediction lifecycle and timer boundaries. Times are separate medians; percentage reductions use median paired ratios. Ranges are observed spread, not confidence intervals.

Case	Prediction ms, parent → candidate	Saved ms	Reduction, median [range]	Fit s, parent → candidate	Peak GPU MiB, P / C	Peak host GiB, P / C
airfoil_self_noise	57.79 → 55.36	2.43	4.79% [3.67–7.37%]	1.87 → 1.83	3757 / 3757	3.103 / 3.083
MIC	229.16 → 219.92	9.25	4.16% [3.71–4.59%]	8.12 → 8.00	4089 / 4089	13.498 / 13.499
coffee_rating_prediction	233.55 → 221.97	11.59	5.20% [4.90–5.44%]	19.29 → 19.20	7445 / 7445	28.464 / 28.464
Bioresponse	1909.41 → 1791.47	117.95	6.15% [6.05–6.25%]	54.01 → 51.81	9953 / 9953	52.642 / 52.651
credit-g	66.07 → 64.29	1.78	2.63% [2.19–3.02%]	2.46 → 2.44	5459 / 5459	4.299 / 4.300
polish_companies_bankruptcy	440.18 → 439.35	0.83	0.14% [0.08–0.25%]	12.53 → 12.49	5585 / 5585	11.069 / 11.071

Credit-g and Polish Companies Bankruptcy were reserved from development. Fit medians decrease on all six cases; one Airfoil fit observation is 3.4% slower, and all observations are retained. The earlier one-pair MIC screen is excluded from these headline aggregates.

Maximum framework allocated and reserved memory are unchanged on every case. The fixed GPU allowance is 81,569,985,331 bytes (75.97 GiB), with a 64 GiB host limit and no swap. Every worker passes resource and cleanup checks. Device peaks are sampled total usage including driver-reserved memory; allocator peaks and the protection artifacts are retained. Temporary category batches use the existing memory-sizing policy, and schema reuse adds a bounded CPU metadata cache.

Reproduction

Final tested head: 317dc1049cb12fc224c1f604a6b41c04705e0894; parent: ba793705be3cbeb25a6600f4e4bacf4ea38d2c96. All combined model and public checks use the final source. PyTorch 2.14.1+cu130; driver 595.71.05; TabArena a334f8e33d337a3c51df18ec3ab35c37d5e37256; model checkpoint revision 3c3e10bbdb590ace29e7026847f92db3c603096d.

Evidence is retained on 4u8g-tur-0037 under /raid/leisun/repos/structured_dm/execution-notes/inference_efficiency_auto_research/experiments/r3-table-metadata-preparation/: verified-review.json, public-review.json, precommit-1.log, source-317dc1049.bundle, and official runs 3/4. Commands, input hashes, source identities, runtime packages, resource samples, exact prediction arrays and fit times are retained with each run. Public checks enable category validation; the frozen benchmark configuration is unchanged.

From paired-run harness 40ff0ead30a372073dc904f668a1dbc215cf8b77, using official protected harness 2b2051b and the recorded GPU 0 policy:

python -m measurement.kumo_tabular_arena.paired_runs \
  --baseline "$SDM_PARENT" --candidate "$SDM_CANDIDATE" \
  --harness "$OFFICIAL_HARNESS" --policy "$GPU0_POLICY" \
  --datasets airfoil_self_noise MIC coffee_rating_prediction Bioresponse \
  --pairs 3 --output "$FRESH_OUTPUT"

Repeat with --datasets credit-g polish_companies_bankruptcy --pairs 3. The policy is retained at experiments/r3-gpu0-qualification/runs/000001/results/policy.json under the research root. Requalify hardware-specific limits on another host. No cumulative percentage is inferred from earlier PRs.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.