PYTHON-6070 Improve help output for just run-server and just setup-tests by blink1073 · Pull Request #3035 · mongodb/mongo-python-driver

blink1073 · 2026-09-02T14:46:31Z

Changes in this PR

just run-server -h now forwards to run-mongodb.sh start -h instead of printing generic argparse help for run_server.py.
just setup-tests <test_name> -h shows the valid sub_test_name choices for test names where they're enumerable (kms, auth_aws, auth_oidc, mod_wsgi, perf).
Merged SUB_TEST_REQUIRED into SUB_TEST_NAME_MAP so the sub-test-required check and the choice list share one source of truth.
Grouped the less common setup-tests flags (--test-min-deps, --green-framework, --compressor, --crypt-shared, --no-ext, --mongodb-api-version, --disable-test-commands) under an "other options" help section.
Trimmed the test_name/sub_test_name usage/help output so the full choice list appears once instead of being repeated in both the usage line and the arguments section.

Example: just setup-tests kms -h

usage: setup_tests.py kms [-h] [--verbose] [--quiet] [--auth] [--ssl]
                          [--test-min-deps] [--debug-log] [--cov]
                          [--green-framework {gevent}]
                          [--compressor {zlib,zstd,snappy}] [--crypt-shared]
                          [--no-ext] [--mongodb-api-version {1}]
                          [--disable-test-commands]
                          [sub_test_name]

Set up the test environment and services for 'kms'.

positional arguments:
  sub_test_name         The optional sub test name, for example 'azure'. One
                        of: azure, azure-remote, azure-fail, gcp, gcp-remote,
                        gcp-fail.

options:
  -h, --help            show this help message and exit
  --verbose, -v         Whether to log at the DEBUG level.
  --quiet, -q           Whether to log at the WARNING level.
  --auth                Whether to add authentication.
  --ssl                 Whether to add TLS configuration.
  --debug-log           Enable pymongo standard logging.
  --cov                 Add test coverage.

other options:
  --test-min-deps       Test against minimum dependency versions
  --green-framework {gevent}
                        Optional green framework to test against.
  --compressor {zlib,zstd,snappy}
                        Optional compression algorithm.
  --crypt-shared        Test with crypt_shared.
  --no-ext              Run without c extensions.
  --mongodb-api-version {1}
                        MongoDB stable API version to use.
  --disable-test-commands
                        Disable test commands.

Example: just run-server -h

run_server.py now forwards straight to the drivers-tools script instead of printing its own help:

$ just run-server -h
usage: drivers-orchestration [-h] [--verbose] [--quiet] [--version VERSION]
                             [--topology {standalone,replica_set,sharded_cluster}]
                             [--auth] [--ssl] [--local-atlas]
                             [--mongodb-runner]
                             [--orchestration-file ORCHESTRATION_FILE]
                             [--load-balancer] [--auth-aws]
                             [--skip-crypt-shared] [--install-legacy-shell]
                             [--disable-test-commands]
                             [--storage-engine {,mmapv1,wiredtiger,inmemory}]
                             [--require-api-version]
                             [--existing-binaries-dir EXISTING_BINARIES_DIR]
                             [--tls-pem-key-file TLS_PEM_KEY_FILE]
                             [--tls-ca-file TLS_CA_FILE]
                             [--tls-allow-invalid-certificates] [--arch ARCH]
                             [--mongo-orchestration-home MONGO_ORCHESTRATION_HOME]
                             [--mongodb-binaries MONGODB_BINARIES]
                             [--tls-cert-key-file TLS_CERT_KEY_FILE]

Run mongo-orchestration and launch a deployment.

Use '--help' for more information.

options:
  -h, --help            show this help message and exit
  --verbose, -v         Whether to log at the DEBUG level
  --quiet, -q           Whether to log at the WARNING level
  --version VERSION     The version to download. Use "latest" to download the
                        newest available version (including release
                        candidates).
  --topology {standalone,replica_set,sharded_cluster}
                        The topology of the server deployment (defaults to
                        standalone unless another flag like load_balancer is
                        set)
  --auth                Whether to add authentication
  --ssl                 Whether to add TLS configuration
  --local-atlas         Whether to use mongodb-atlas-local to start the server
  --mongodb-runner      Whether to use mongodb-runner to start the server
  --orchestration-file ORCHESTRATION_FILE
                        The name of the orchestration config file

Other options:
  --load-balancer       Whether to use a load balancer
  --auth-aws            Whether to use MONGODB-AWS auth
  --skip-crypt-shared   Whether to skip installing crypt_shared lib
  --install-legacy-shell
                        Whether to install the legacy shell
  --disable-test-commands
                        Whether to disable test commands
  --storage-engine {,mmapv1,wiredtiger,inmemory}
                        The storage engine to use
  --require-api-version
                        Whether to set requireApiVersion
  --existing-binaries-dir EXISTING_BINARIES_DIR
                        A directory containing existing mongodb binaries to
                        use instead of downloading new ones
  --tls-pem-key-file TLS_PEM_KEY_FILE
                        A .pem file that contains the TLS certificate and key
                        for the server
  --tls-ca-file TLS_CA_FILE
                        A .pem file that contains the root certificate chain
                        for the server
  --tls-allow-invalid-certificates
                        Whether to pass --tlsAllowInvalidCertificates to
                        mongod
  --arch ARCH           the architecture. if unspecified, the arch will be
                        inferred.
  --mongo-orchestration-home MONGO_ORCHESTRATION_HOME
                        The path to mongo-orchestration home
  --mongodb-binaries MONGODB_BINARIES
                        The path to store the MongoDB binaries
  --tls-cert-key-file TLS_CERT_KEY_FILE
                        A .pem to be used as the tlsCertificateKeyFile option
                        in mongo-orchestration

(This help text comes from run-mongodb.sh in drivers-evergreen-tools, not from this repo.)

Test Plan

Manually invoked get_test_options with kms, auth_aws, auth_oidc, mod_wsgi, perf, and no test name, confirming help output and validation for each.
Manually invoked just run-server -h against a local checkout of drivers-evergreen-tools to confirm it forwards to and prints run-mongodb.sh start -h's own output.
Ran just lint-manual — all hooks pass.
Ran just typing — pre-existing unrelated failures in pymongo/ocsp_support.py, no new errors.
Submitted an Evergreen patch tagged pr (all 47 pr-tagged variants) — completed with no task failures.

Checklist

Checklist for Author

Did you update the changelog (if necessary)?
Is there test coverage?
Is any followup work tracked in a JIRA ticket? If so, add link(s).

Checklist for Reviewer

Does the title of the PR reference a JIRA Ticket?
Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
Is all relevant documentation (README or docstring) updated?

just run-server -h now forwards to run-mongodb.sh start -h instead of printing generic argparse help, and just setup-tests <name> -h shows the valid sub_test_name choices for test names where they're enumerable (kms, auth_aws, auth_oidc, mod_wsgi, perf).

Copilot

🟡 Changes recommended

The updated argparse configuration can still produce misleading/duplicated help output and should be adjusted to match the PR’s stated help-output goals.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR (PYTHON-6070) improves the developer UX of Evergreen helper scripts by refining CLI help output and making just run-server -h delegate help to the underlying drivers-tools script.

Changes:

Enhanced setup-tests help output by enumerating valid sub_test_name choices for specific test suites and reducing repeated/verbose choice listings.
Consolidated “sub-test required” logic into a single SUB_TEST_NAME_MAP source of truth.
Forwarded run-server -h/--help to run-mongodb.sh start -h instead of printing the Python script’s argparse help.

File summaries

File	Description
.evergreen/scripts/utils.py	Refactors CLI option parsing/help text for `setup-tests`, adds `SUB_TEST_NAME_MAP`, and reorganizes help sections.
.evergreen/scripts/run_server.py	Adds `-h/--help` forwarding behavior to drivers-tools `run-mongodb.sh`.

Review details

Suppressed comments (1)

.evergreen/scripts/utils.py:142

sub_test_name is validated as required for tests in SUB_TEST_NAME_MAP, but the generated help text always calls it "optional". For just setup-tests <name> -h this is misleading because the command will error without a sub-test for these suites; the help can mention that it’s required for the pinned test name.

            help_text = (
                f"The optional sub test name, for example {example_sub_test_name!r}. "
                f"One of: {', '.join(sub_test_choices)}."
            )

Files reviewed: 2/2 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Pass cwd=DRIVERS_TOOLS when forwarding -h to run-mongodb.sh, matching the normal start path. Clarify that sub_test_name is required, not optional, for test names in SUB_TEST_NAME_MAP.

Copilot

🟡 Changes recommended

The current argv-scanning logic in get_test_options() can mis-detect the intended test_name and break valid invocations, and run_server.py -h can crash when DRIVERS_TOOLS is unset instead of providing usable help.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 2/2 changed files
Comments generated: 2
Review effort level: Lite

Copilot

🟡 Changes recommended

The run-server -h/--help forwarding currently goes through run_command, which adds extra INFO log lines and prevents the help output from being forwarded verbatim as shown in the PR description.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 2/2 changed files
Comments generated: 1
Review effort level: Lite

Only pin the test_name argument's choices from the first positional argument, not any token in argv, so a sub_test_name that happens to match a SUB_TEST_NAME_MAP key no longer gets misidentified as the test suite. Fail fast with a clear error when DRIVERS_TOOLS is unset instead of a confusing FileNotFoundError.

run_command logs "Running command..." before and after every call, which wrapped the forwarded run-mongodb.sh start -h output and defeated the point of showing it verbatim. Call subprocess.run directly for the -h path instead.

Copilot

🟡 Changes recommended

just run-server -h/--help currently fails when DRIVERS_TOOLS is unset due to an early env check that prevents reaching any help output.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 2/2 changed files
Comments generated: 1
Review effort level: Lite

just run-server -h/--help was raising before reaching the help path when DRIVERS_TOOLS was unset. Only raise for the real start path; -h without DRIVERS_TOOLS now falls back to get_test_options' own argparse help instead of erroring.

Copilot

🔵 Needs a closer look

The new known_test_name inference can mis-detect the suite name when options with values are placed before the positional test name, leading to incorrect/help output behavior.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

.evergreen/scripts/utils.py:111

The logic that infers known_test_name treats any non-flag token as positional. This mis-detects the test name when an option that takes a value is provided before the test name (e.g. --green-framework gevent kms -h), because gevent will be treated as the first positional argument and the help output won’t be specialized (and could become wrong if an option value ever matches a test name). Consider skipping known option values when identifying the first positional argument.

Files reviewed: 2/2 changed files
Comments generated: 0 new
Review effort level: Lite

codecov · 2026-09-02T17:07:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

An option that takes a value (e.g. --green-framework gevent) placed before the test_name positional was mistaken for it, silently skipping the help specialization. Skip known value-taking options' values when scanning for the first positional argument.

Copilot

🔵 Needs a closer look

run-server -h can currently fail if DRIVERS_TOOLS is set to a stale/invalid path, instead of falling back to local argparse help.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

.evergreen/scripts/run_server.py:25

When -h/--help is requested and DRIVERS_TOOLS is set but points to a non-existent checkout (or run-mongodb.sh is missing), this will raise an exception instead of falling back to the local argparse help. Consider verifying DRIVERS_TOOLS/run-mongodb.sh exist before forwarding so help remains reliable even with a stale env var.

Files reviewed: 2/2 changed files
Comments generated: 0 new
Review effort level: Lite

A stale or invalid DRIVERS_TOOLS (pointing at a missing checkout or missing run-mongodb.sh) crashed with a confusing FileNotFoundError instead of falling back to the local argparse help. Only forward -h when run-mongodb.sh actually exists.

Copilot

🟢 Approval recommended

The changes are localized to developer/Evergreen scripts, align with the PR’s stated behavior, and do not introduce any verified functional regressions in argument parsing or validation.

Review details

Files reviewed: 2/2 changed files
Comments generated: 0 new
Review effort level: Lite

blink1073 requested a lite review from Copilot September 2, 2026 14:51

Copilot started reviewing on behalf of blink1073 September 2, 2026 14:52 View session