Add AuTest for connect_attempts_max_retries_down_server

[masaori335]

This is an effort to verify connect_attempts_max_retries_down_server behavior.

The doc for proxy.config.http.connect_attempts_max_retries_down_server says below.

Maximum number of connection attempts Traffic Server can make while an origin is marked down per request.

Also doc of proxy.config.http.connect_attempts_max_retries says below

Once the maximum number of retries is reached, the origin is marked down. After this, the setting proxy.config.http.connect_attempts_max_retries_down_server is used to limit the number of retry attempts to the known down origin.

However, I don't see ATS makes connection attempts against down servers. From my understanding, HostDBRecord::select_best_http doesn't return HostDBInfo which is in down state. How HttpSM can try to down servers?

[Copilot]

Pull request overview

Adds a new AuTest (ATSReplay-based gold test) intended to exercise proxy.config.http.connect_attempts_max_retries_down_server, and extends the ATSReplay harness to allow DNS records to be specified directly in replay YAML.

Changes:

• Added a new replay scenario connect_attempts_rr_down_server.replay.yaml under tests/gold_tests/dns/replay/.
• Added a new gold test driver tests/gold_tests/dns/connect_attempts.test.py to run the replay.
• Extended ATSReplayTest to accept autest.dns.records and apply them to the DNS server process via addRecords.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
tests/gold_tests/dns/replay/connect_attempts_rr_down_server.replay.yaml	New replay scenario configuring connect-attempt retry records and expected proxy responses.
tests/gold_tests/dns/connect_attempts.test.py	New gold test entrypoint invoking the replay via Test.ATSReplayTest.
tests/gold_tests/autest-site/ats_replay.test.ext	ATSReplay harness enhancement: allow replay YAML to inject DNS records into the test DNS server.

[Copilot]

The header comment says this replay assumes caching is enabled, but autest.ats.process_config.enable_cache is set to false. Please update/remove the comment to match the actual configuration (or enable cache if that was intended).

Suggested change

	# This replay file assumes that caching is enabled.
	# This replay file runs with caching disabled.

[Copilot]

Typo in comment: immidiately should be immediately.

Suggested change

	# request expected hit down_server cache and immidiately get 500
	# request expected hit down_server cache and immediately get 500

[Copilot]

This transaction expects a 500 response, but ATS's "origin down" path returns 502 (see HttpTransact::OriginDown), and with proxy.config.http.connect_attempts_max_retries_down_server: 1 ATS will still attempt a connection to a marked-down origin rather than short-circuiting. The expected proxy-response.status here should be updated accordingly (likely 502), and the accompanying comment should be adjusted to reflect what behavior is being asserted.

[masaori335]

I think I got what I'm missing. To test this config, ATS needs to go this line in HttpTransact::handle_response_from_server, but current test is trying connect attempt against un-exist servers and ATS doesn't comes here.

trafficserver/src/proxy/http/HttpTransact.cc

Line 3912 in 5b32738

max_connect_retries = s->txn_conf->connect_attempts_max_retries_down_server - 1;

[bryancall]

[approve ci ubuntu autest 0]

[bryancall]

AuTest 0of4 is a real failure in the new connect_attempts test — not flaky. It failed on both the original run and a retrigger (build 35759). The proxy-verifier client exits with code 1, meaning the response from ATS didn't match the replay expectations. Server, DNS, and ATS processes all pass fine.

FAILED TESTS:
  - connect_attempts

Process: Default: Failed
  Test : Checking that ReturnCode == 0 - Failed
     Reason: Returned Value 1 != 0

Ubuntu passed on retrigger — that one was flaky. Just this test remaining.