fix(run): purge local stores from the correct directory - #1404
Open
l2ysho wants to merge 2 commits into
Open
Conversation
l2ysho
marked this pull request as draft
September 8, 2026 22:24
`apify run --purge` hands the purge to crawlee unless a temp input file is in play; templates now ship an input schema, so that file is always written and the CLI has to purge by itself. That path was broken in two places: - `useModuleVersion` probed for crawlee without passing `cwd`, so the subprocess looked in whatever directory the CLI process sat in - the `purgeDefault*` helpers resolved storage paths against the OS working directory, which silently made them no-ops under the test cwd mock Both now resolve against `process.cwd()`. Adds unit tests for the helpers and a test for the branch taken when crawlee is absent; the run tests write their input schema to `.actor/`, where templates keep it, so it is not shadowed on case-insensitive file systems. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up from review of the purge fix. Same class of defect in the code around it, plus the coverage that was missing. - `checkIfStorageIsEmpty` fed backslash paths to glob on Windows, which reads them as escapes, so the `!.../INPUT.*` negations never matched and every `apify run` without `--purge` warned about leftover state - the legacy `apify_storage` rename and the input/temp-input paths in `validateAndStoreInput` resolve against the command's directory - `getLocalInput` used `join`, which breaks on an absolute `APIFY_LOCAL_STORAGE_DIR` - dropped the redundant storage-dir `existsSync` guards from the purge helpers Adds `useModuleVersion` tests, `checkIfStorageIsEmpty` coverage, and drops the template's input schema in the run input tests so one schema exists on every file system. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
l2ysho
force-pushed
the
claude/master-pipeline-check-failures-2dbef0
branch
from
September 9, 2026 14:27
6c144be to
f3ad9d8
Compare
l2ysho
marked this pull request as ready for review
September 9, 2026 14:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
After we merged apify/actor-templates#866 this popped up. Actually it is missing scenario in
run --purgenot covered by tests and also a bug. + one windows bug found in review.Why master is red
apify/actor-templates #866 added
.actor/input_schema.jsonto every template on 2026-09-08 15:15 UTC. The tests download the live template, so the change hit CI with no commit here — the run before it was green.With a schema present,
apify run --purgewrites a merged temp input file, so it turns crawlee's purge-on-start off (that purge only preservesINPUT*and would delete the temp file) and purges by itself instead. That path was broken:useModuleVersionran the crawlee version probe withoutcwd, so under the test cwd mock it looked in the repo root instead of the project. Production is unaffected — execa defaults toprocess.cwd()— but nothing could exercise the branch.purgeDefault*resolved storage paths against the OS working directory, so under the same mock they returned early and deleted nothing — a silent no-op that reads exactly like a pass.Two user-facing fixes
Found while reviewing the code around the above, both pre-existing:
checkIfStorageIsEmptyinterpolatedstorage\key_value_stores\defaultinto glob patterns, which reads backslashes as escapes. The!…/INPUT.*negations never matched, soapify runwithout--purgealways warned "the storage directory contains a previous state" — even with nothing but the input file. Confirmed empirically against the pinned tinyglobby.APIFY_LOCAL_STORAGE_DIR:getLocalInputand the input paths invalidateAndStoreInputusedjoin(cwd, …), producing<cwd>/<abs>. Nowresolve, matching the purge helpers.Changes
src/lib/utils.tsgetLocalInputresolve against the command's directory; redundantexistsSyncguards droppedsrc/commands/run.tssrc/lib/hooks/useModuleVersion.tscwdto the version probetest/local/lib/utils-purge-storages.test.tscheckIfStorageIsEmpty, directlytest/local/lib/useModuleVersion.test.tstest/local/commands/run-without-crawlee.test.tstest/local/commands/run.test.tsThe last one fixes the Windows/macOS half of the failure.
.actor/INPUT_SCHEMA.jsonis first inDEFAULT_INPUT_SCHEMA_PATHS, and on case-insensitive file systems it matched the template's.actor/input_schema.json, so an empty schema won over the test's own. The tests now delete the template's file, so exactly one schema exists on every platform.Coverage
Every change here has a test that fails without it.
checkIfStorageIsEmptyhad no coverage at all before. The no-crawlee test asserts a marker file the Actor writes — without it, "the Actor never ran" and "the purge worked" both leave an empty storage folder, which is how the original bug stayed hidden.pnpm run test:local: 550 passed, 4 skipped, 0 failed. Lint, format,tsc --noEmit, build clean. No dependency changes, so no install-size impact.Merge order
Node 20 legs fail here on
create.test.ts > should skip installing optional dependencies— red on master since 2026-09-04, four days before the templates change, and unrelated to this PR. #1403 drops those legs; its own legs fail on the bug this PR fixes. Merging #1403 first lets this one get a fully green run before it lands.🤖 Generated with Claude Code