Conversation
e266280 to
a60db3a
Compare
cachebag
changed the title
|
@rami3l Re: #3937; this PR is intentionally limited to replacing rustup’s own executable and does not modify toolchain transactions, but it introduces a self-update-specific staging directory, lock, and cleanup lifecycle. Does that overlap with the primitives you are developing for the process-safety goal, or do you think keeping self-update isolated here is reasonable? |
@cachebag Actually I think it can be done quite separately from the transactional semantics. As I added at the final minute comment when trying to merge the project goal, I specifically said that it would be only about concurrency problems of the Rust installations rather than those of the rustup installation (which this PR falls under): rust-lang/goals#731 (comment) |
2cdfb67 to
3b4cc2b
Compare
cachebag
changed the title
|
CI seems to have crapped out on us due to a GitHub issue. @rami3l does it let you re-run the failed jobs? |
bc36468 to
af1b039
Compare
This comment has been minimized.
This comment has been minimized.
af1b039 to
c7ebab5
Compare
This comment has been minimized.
This comment has been minimized.
c7ebab5 to
c1d76d8
Compare
@djc Re: a separate PR; I suppose that is fine...ab0d89a and 1ca3033 can be by themselves, though what should I do with fc00f5f? The whole point of that test is that it fails without the fix. |
6bd5c62 to
f5c76de
Compare
This comment has been minimized.
This comment has been minimized.
@cachebag Sorry for the late reply! As per our current contribution guidelines, each commit is supposed to pass the test suite on its own, so we encourage adding a test upfront demonstrating the current behavior (even the current behavior is an error being thrown) upfront, and when the fix arrives, the test case is updated accordingly. |
f5c76de to
1aba886
Compare
This comment has been minimized.
This comment has been minimized.
1aba886 to
ecfbdd4
Compare
|
@rami3l understood, thank you for the feedback that makes a lot of sense. this is how i've decided to split it out then:
i ran the race test at each of those commits locally to confirm it passes at every step. no rush of course: but i've left some threads open with questions/comments on your review(s). |
I don't really disagree with anything @rami3l wrote there, but I think in the execution "logically independent commits" can be up to interpretation. Here are some guidelines that I like to keep in mind: specifically when you're mainly adding code, until you've actually achieved a desirable outcome the intermediate outcomes (that don't achieve a desirable outcome yet) aren't as valuable, because the state of the code really only makes sense when you achieve the desirable outcome. For adding code, splitting up the additions into multiple code often doesn't really make it more reviewable (which is very different from when you're adding code and changing existing code in the same commit). So in that light do you think the current commit history could be simplified somewhat? @rami3l thoughts? |
This comment has been minimized.
This comment has been minimized.
|
@djc I agree with you that if one single change is adding a large chunk of stuff then there is generally no use in splitting it into multiple commits. However, my previous point was just that since the previous version of that change ( @cachebag The new split looks pretty good to me, so I'd consider that concern to have been addressed. Nice work on that matter! Frankly speaking, I think the current state of this PR looks pretty polished already (probably modulo some minor concerns). I'd love to do another round once you have done with the rebase and I'm looking forward to merging it soon :) |
ecfbdd4 to
0d0a8de
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
A process parked at a checkpoint could only be killed by the test driver. Poll for the checkpoint marker instead of sleeping, and add `ParkedChild::resume()`, which removes the marker and waits for the process to finish. This lets a test race another command against a paused operation and then observe how the paused one completes.
Park the replacer right before it installs the new binaries and run a proxy in the meantime. The proxy's startup cleanup deletes `$CARGO_HOME/bin/rustup-init`, which is the replacer's own executable, so the replacer unlinks the installed rustup and then fails to copy itself over it, leaving no rustup behind (rust-lang#5076, rust-lang#1864). The test asserts this current behaviour so that it passes on its own; the fix later in this series updates it to assert that the replacement succeeds.
Import sibling items through `super` in the Windows module and move `DEFAULT_UPDATE_ROOT` below its users, as the coding standards prefer. No functional change.
Two concurrent `rustup self update` invocations shared one updater path and could overwrite each other's download or replacement (rust-lang#1864). `prepare_update` now takes a global self-update lock before downloading and hands it to `run_update` inside a `PreparedUpdater`, which releases it only once the replacer has been spawned. The replacer takes the same lock before replacing rustup, so `install_bins` becomes a method on the lock and can only run while it is held. The lock file lives under `$RUSTUP_HOME/self-update/` and is released by the OS when the owning process exits, so a crash can never leave it held.
0d0a8de to
adafa09
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
…come Every rustup or proxy invocation deleted `$CARGO_HOME/bin/rustup-init` during startup cleanup. A proxy starting while the updater was still replacing rustup could therefore delete the updater out from under it (rust-lang#5076). The updater now lives at `$RUSTUP_HOME/self-update/rustup-init`, and the replacer records a `complete` or `failed` marker next to it once it is done. Startup cleanup removes the managed updater only when such a marker exists and the self-update lock is free, so an update in progress is never touched. The legacy path is still cleaned as before. The proxy cleanup race test now asserts that the replacement succeeds and the installed rustup is updated.
An updater whose replacer never ran, or crashed before recording an outcome, has no marker and was left behind forever. A legacy `$CARGO_HOME/bin/rustup-init` may still belong to an older rustup that is running it, so deleting it on sight is the very race being fixed. Both are now removed only after they have gone untouched for a day.
Replacement used to unlink the installed rustup and then copy the updater over the freed path. Any failure in between, such as the updater having been deleted meanwhile, left `$CARGO_HOME/bin` without a rustup at all. The new binary is now copied to a `.rustup-pending-*` sibling, synced to disk, and then renamed over the installed rustup. `std::fs::rename` replaces an existing destination in one step on every platform, so a failure before publication leaves the existing rustup untouched.
A crash between staging and publishing leaves a `.rustup-pending-*` file in `$CARGO_HOME/bin`. Startup cleanup now removes such files once they have gone untouched for a day, the same threshold used for abandoned updaters.
After spawning the replacer, the parent ran the updater a second time with `--version` and wrote the result to the uninstall registry entry. The registry could therefore claim a version that was never installed if the replacer went on to fail. The replacer is the new rustup and knows its own version, so it now updates `DisplayVersion` right after installing the binaries, under the same self-update lock. The test waits for the completion marker because the registry is now written after `rustup self update` has returned.
adafa09 to
83ae15f
Compare
| fn wait_for_completed_update(rustup_home: &Path) { | ||
| let stage = rustup_home.join("self-update"); | ||
| retry(Fibonacci::from_millis(1).map(jitter).take(23), || { | ||
| if stage.join("complete").is_file() { | ||
| Ok(()) | ||
| } else if stage.join("failed").is_file() { | ||
| Err("self-update failed") | ||
| } else { | ||
| Err("self-update has not completed") | ||
| } | ||
| }) | ||
| .unwrap(); | ||
| } | ||
|
|
||
| fn managed_updater(rustup_home: &Path) -> PathBuf { | ||
| rustup_home | ||
| .join("self-update") | ||
| .join(format!("rustup-init{EXE_SUFFIX}")) | ||
| } | ||
|
|
There was a problem hiding this comment.
Nit: mod self_update is public so I guess you can reuse some constants/helpers from over there.
| let markers = [Marker::Complete, Marker::Failed]; | ||
| let finished = markers | ||
| .iter() | ||
| .any(|marker| marker.path(&lock.directory).is_file()); | ||
| if finished && utils::remove_file_best_effort("self-updater", &updater) { | ||
| for marker in markers { | ||
| utils::remove_file_best_effort( | ||
| "self-update status marker", | ||
| &marker.path(&lock.directory), | ||
| ); | ||
| } | ||
| } |
There was a problem hiding this comment.
The lock only says whether an update is running right now. After the parent exits and before the replacer takes the lock, cleanup sees a free lock and an updater on disk and can’t tell the difference between being finished or queued to run. Deleting it in the second case is the bug.
Please correct me if I have misread anything, I am just trying to understand the full story. In the following snippet that you have written:
let result = process.cargo_home().and_then(...);
stage::mark_result(result.is_ok(), process);
result?;If the process crashes in let result = ..., then the cleanup never happens, because the proxy still thinks that the updater is queued to run, so not deleting the replacer in this case is (at this point) still a bug, but this bug is mitigated by cleaning it up when it becomes stale in a later commit, because a self update can't possibly take that long. Is that correct?
| .and_then(|cargo_home| { | ||
| self_update_lock.install_bins(&cargo_home.join("bin"), super::force_hard_links(process)) | ||
| }) | ||
| .and_then(|()| { | ||
| update_uninstall_registry_display_version(env!("CARGO_PKG_VERSION"), process) | ||
| }); |
There was a problem hiding this comment.
Nit: Maybe we can do both steps in one callback?
This branch has not been deployed
5 participants
Self-update currently stages every updater at the shared
$CARGO_HOME/bin/rustup-initpath. Any rustup proxy started while replacement is in progress can unlink that path during startup cleanup. On Unix, the replacement process then removes the installedrustup, fails to reopen its own deleted path, and leaves rustup missing. This appears to explain the failures reported in #1864, #4648, and #4777.The first commit adds a deterministic test for that race. It fails on
demo/self-update-race-beforeand the identical test passes with this branch.This PR uses a single managed updater under
$RUSTUP_HOME, guarded by an OS-released global self-update lock. It fully prepares and syncs a sibling binary before publishing it with rename on Unix or ReplaceFileW on Windows. Completed, failed, and abandoned artifacts are cleaned without racing an active update.The core result of this fix is that now, a failure before publication will leave the existing rustup untouched.
Fixes #5076
Fixes #1864