[ParidelPooya]

Fixes #797.

Summary

After a suspend that the service refused for now, the MicroVM worker now waits at least 30 seconds before its next SuspendMicrovm call. The docs now say which errors reach "refused for now" with the default client. The JavaScript worker made the same change in aws/aws-durable-execution-sdk-js#1040.

Why

1. feat(microvm-worker): act on lifecycle hooks #782 added the worker's own suspend when idle. A first-attempt error that the service answers before it acts is "refused for now". The worker then accepts jobs again at once and starts the idle time again.
2. Some of these errors repeat on every call. A wrong secret key gives a signature error each time. A static session token gives ExpiredTokenException each time after it expires.
3. The run hook accepts any positive autoSuspendIdleSeconds. So an idle MicroVM made one call per idle time, with no lower bound, and wrote two log lines per call.
4. A probe with an idle time of 0.0001 s counted 15 calls in 3 s of real time. The count depends on the machine.
5. The docs said that a first-attempt throttle is the usual case for "refused for now". The default client uses botocore's standard retry mode with 3 attempts, and that mode retries ThrottlingException. A probe showed that a 429 on every attempt ends with RetryAttempts 2. So the error is classified unknown.
6. botocore retries no signature, clock-skew or expired-token error. So those errors stay "refused for now" with the default client.

Changes

The 30-second minimum

1. _start_idle_time(after_refusal=True) uses max(autoSuspendIdleSeconds, 30 s) for the next idle time.
2. The refused-for-now branch runs while the call is still counted in _suspend_calls. So the request waits in _idle_after_call, and _idle_after_call_floor records the minimum. _suspend_call_ended applies it.
3. A later request during the same call overwrites the recorded minimum. So the latest request decides.
4. A resume hook or a job that ends during the call does not shorten the wait, because the refused outcome arrives after it.
5. A job or a resume hook after the refusal starts the normal idle time.
6. While no job ends and no lifecycle hook arrives, the worker makes at most one call per 30 seconds.

Warnings

The refused-for-now warning now has two texts. The worker chooses the text under the lock, from the same decision that sets the minimum.

• Without a suspend hook during the call: the worker tries again after the next idle time. That idle time lasts at least 30 seconds, unless a job or a resume hook starts the idle time again.
• With a suspend hook during the call: the hook's refusal of jobs stays. It ends when the resume hook arrives, or 30 seconds after the suspend hook. The idle time then starts again.

Docs

• README item 7 and the docstrings of SuspendOutcome, classify_suspend_error, create_microvms_client and the listener say that the default client retries a throttle. So a throttle usually ends as unknown. "Refused for now" covers expired credentials and a signature error, with or without a clock offset. A throttle counts only from a client that does not retry it.
• The JavaScript default client also retries clock-skew errors. So there, a clock-skew error usually ends as unknown. README item 7 states this difference.
• The create_microvms_client option doc says that a caller client that retries must report its retries in ResponseMetadata.RetryAttempts on each ClientError. Otherwise a retried throttle counts as a first attempt.

Testing

• Package: 411 tests pass. worker.py and self_suspend.py are at 100 % line and branch coverage. Typecheck, hatch fmt --check and the commit lint pass.
• New tests with fake timers:
  • the minimum at idle times of 1 s, 10 s and 45 s;
  • one call per 30 s while the refusal lasts;
  • the normal idle time after a job or a resume hook that follows the refusal;
  • the minimum after a resume hook, or a job that starts and ends, during the call;
  • the normal idle time after a job that runs across the refused outcome;
  • both warning texts.
• A test with the real default client checks the retry count and the outcome for a throttle, a skewed signature error, a signature mismatch and an expired token.
• Six mutations of the minimum each fail at least one test. Examples: no minimum, min instead of max, the minimum on every start, the first request during the call deciding instead of the latest.
• A loop probe over 3 s of real time made 1 call for every idle time and error after the change. Before the change it made up to 15.
• The package suite passed 3 of 3 runs with 8 parallel workers.
• Three rounds of review. The review docs are not part of the PR.
• Not run against the live service.