Reproducibility question: Claude Code version, number of runs, and score discrepancy (we get 40+ vs. reported ~30)

[tc-yue]

Hi, thanks for the great work and for releasing the benchmark.

We're trying to reproduce the reported numbers and are seeing a consistent discrepancy, so I have a few questions about the exact evaluation setup:

1. Claude Code version — Which version of Claude Code did you use as the agent scaffold for the reported results? We'd like to match it exactly, since scaffold behavior changes across versions.
2. Number of runs — How many times was each configuration run, and how were the reported scores aggregated (e.g., mean over N runs, pass@1, best-of-N)? Knowing N would help us account for run-to-run variance.
3. Score discrepancy — In our runs using Claude Code 2.140.0 + GPT-5.5, we consistently score 40+, across multiple independent runs, which is noticeably higher than the ~30 reported in the paper for this configuration. Could you help us understand what might cause this gap? Specifically:
   - Is the reported score for this model produced with Claude Code, or with a different native scaffold? (The paper mentions using the provider's native scaffold — e.g., Codex CLI for OpenAI models — so we want to confirm which harness was used for GPT-5.5.)
   - Were there any differences in reasoning effort, user-simulator model, max-turns/steps, or task subset that we should replicate?

Any details you can share on the exact configuration used to produce the reported numbers would be very helpful for a faithful reproduction. Thanks!