feat(wan22): add VBench accuracy scorer for WAN 2.2 video generation

[wu6u3tw]

Summary

• Adds VBenchScorer (scorer_id="vbench") that scores video-generation outputs on six VBench dimensions (subject_consistency, background_consistency, motion_smoothness, dynamic_degree, appearance_style, scene) and returns the mean of the per-dimension aggregates.
• Runs VBench in an isolated uv subproject under examples/09_Wan22_VideoGen_Example/accuracy/ (vbench pins transformers==4.33.2 + numpy<2, incompatible with the parent env). VBenchScorer.score() shells out to vbench_runner.py via uv run --project, so the benchmark process never imports vbench.
• VideoGenAdapter.decode_response now mirrors video_path into response_output so the event log carries it to the scorer.
• Wires ScorerMethod.VBENCH into the config enum and adds offline_wan22_accuracy.yaml as a peer to the existing perf example.

Test plan

• Unit: tests/unit/evaluation/test_scoring.py covers registration, mean-of-6-dims with mocked subprocess.run, missing-subproject FileNotFoundError.
• Unit: tests/unit/videogen/test_adapter.py + integration adapter test updated for the new response_output contract.
• examples/09_Wan22_VideoGen_Example/offline_wan22_accuracy.yaml validates as OfflineBenchmarkConfig.
• Subproject uv lock resolves (115 packages).
• pre-commit run --all-files clean.
• End-to-end VBench run on a GPU host with uv sync in the accuracy subproject — not yet executed.

Draft for code review.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Code Review

This pull request introduces the VBenchScorer to evaluate video generation accuracy across six MLPerf dimensions. To manage conflicting dependencies, VBench is executed within an isolated uv subproject using a standalone runner script. The changes include a comprehensive runbook, configuration templates, and updates to the VideoGenAdapter to pass video paths to the scorer. Review feedback highlights the need for more robust error handling when parsing VBench results and ensuring that symlink creation is resilient to prompts containing directory separators.

[gemini-code-assist]

This block can be improved in two ways:\n1. Use read_bytes() for more efficient JSON decoding with msgspec.\n2. Add explicit error handling for missing dimension keys in the results JSON. If VBench fails to score a dimension (e.g., due to prompt mismatch), a raw KeyError is raised, which is difficult to debug.

Suggested change

	with results_path.open() as f:
	results = msgspec.json.decode(f.read())
	per_dim_scores: list[float] = [
	float(results[dim][0]) for dim in self.dimensions
	]
	results = msgspec.json.decode(results_path.read_bytes())
	try:
	per_dim_scores: list[float] = [
	float(results[dim][0]) for dim in self.dimensions
	]
	except KeyError as e:
	raise KeyError(
	f"Dimension {e} missing from VBench results at {results_path}. "
	"Ensure prompts match VBench's standard suite for all dimensions."
	) from e

[gemini-code-assist]

Prompts can contain characters like slashes (/) which are interpreted as directory separators in file paths. If a prompt contains a slash, dst.symlink_to will fail because the nested parent directory won't exist. Adding dst.parent.mkdir ensures the symlink can be created regardless of the prompt content.

Suggested change

	dst = staged_dir / f"{prompt}-{idx}{src.suffix or '.mp4'}"
	dst.unlink(missing_ok=True)
	dst.symlink_to(src.resolve())
	dst = staged_dir / f"{prompt}-{idx}{src.suffix or '.mp4'}"
	dst.parent.mkdir(parents=True, exist_ok=True)
	dst.unlink(missing_ok=True)
	dst.symlink_to(src.resolve())