feat(envs): remove core envs from the manifest and their sources from the workspace - #10465
Draft
davidfirst wants to merge 286 commits into
Draft
feat(envs): remove core envs from the manifest and their sources from the workspace#10465davidfirst wants to merge 286 commits into
davidfirst wants to merge 286 commits into
Conversation
PR Summary by QodoLoad former core envs as regular registry envs with legacy version pinning
AI Description
Diagram
High-Level Assessment
Files changed (15)
|
Code Review by Qodo
1. Unfixable NonLoadedEnv remediation
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
|
Code review by qodo was updated up to the latest commit c7dd1a7 |
Code Review by QodoNew Review StartedThis review has been superseded by a new analysis |
|
Code review by qodo was updated up to the latest commit e6418b9 |
|
Code review by qodo was updated up to the latest commit c9eca3d |
|
Code review by qodo was updated up to the latest commit 3f5c24e |
|
Code review by qodo was updated up to the latest commit 0607c7d |
|
Code review by qodo was updated up to the latest commit b23b273 |
|
Code review by qodo was updated up to the latest commit 94eddce |
|
Code review by qodo was updated up to the latest commit ac4b7d0 |
|
Code review by qodo was updated up to the latest commit ab21e34 |
|
Code review by qodo was updated up to the latest commit 66fd06b |
Code Review by QodoNew Review StartedThis review has been superseded by a new analysis |
|
Code review by qodo was updated up to the latest commit 684cdf6 |
|
Code review by qodo was updated up to the latest commit 3df0fcd |
|
Code review by qodo was updated up to the latest commit eedfdcd |
…the ones nobody claims Three problems with the previous commit, all from review: - BundleUiTask built both roots with `Promise.all`, which rejects as soon as one fails - the `finally` then closed compilers while the other root was still bundling, the exact hazard the deferral exists to avoid. It now waits with `allSettled` and surfaces the first failure afterwards. - `build()` pushed every compiler it created onto `openBuildCompilers`, but only BundleUiTask ever drains that list. `bit start`'s pre-bundle and watch rebuilds call `build()` too, so a module graph per build would have stayed referenced for the life of the process - worse than before. Tracking is now opt-in via `deferClose`; every other caller gets its compilers closed in `build()`'s finally. - Cleanup ran unguarded from a `finally`, so a throw from `compiler.close()` could replace the build error that sent us there. It is now per-compiler try/catch that logs and moves on, the same contract as the webpack bundler's cleanup.
…preview bundlers Review point: the predicate was defined twice, once per bundler, so the two could drift into handling TypeScript-in-node_modules differently. Both configs already import from @teambit/webpack (and nothing in that aspect imports back), so it moves there as a single export with the rationale attached.
…modules paths
Review point: the previous wording ('lets TypeScript through from anywhere, declaration
files excluded') could be read as excluding .d.ts globally. The predicate only ever
returns true for paths under node_modules; outside it, nothing is excluded and the
rule's own `test` decides.
…nent The helper now lives in its own component, published as @teambit/webpack.modules.exclude-node-modules-js. Both bundler configs import it from there and the copy that lived in the webpack aspect is gone, along with its re-export - nothing outside those two configs used it. Pinned at 0.0.1 in the workspace policy. The lockfile is deliberately untouched: regenerating it here rewrites ~130k lines of unrelated drift, which belongs in its own chore commit.
Adds @teambit/webpack.modules.exclude-node-modules-js@0.0.1, pinned in the previous commit's policy entry. The diff is far larger than that one package: this is the first `bit install` to reconcile the lockfile with the branch's `workspace.jsonc`, so it also drops entries the branch had already removed - `@teambit/aspect` from the importers, and the `@teambit/legacy@2.1.0` peer suffixes that trail through most resolutions.
Review: BundleUiTask found the first rejected root, threw its reason, and the catch
immediately replaced it with `new Error("Generating UI bundle failed")`. The actual
error only ever reached the debug log - diagnosing a failed BundleUI in CI meant
downloading the log artifact to find out what rspack actually said.
Now the thrown message names the roots that failed, every failing root is logged
(not only the one that ends the task), and the first reason is attached as `cause`.
The reason itself stays out of the message on purpose: for an rspack failure it is
the entire stats output, megabytes of it, which is why the message was generic in
the first place.
Same issue as BundleUI, found by review: `buildPreBundlePreview` creates an rspack compiler, runs it, and never closes it - so the module graph and rspack native side stay resident. Both callers keep the process alive well past that point: the PreBundlePreview build task runs once per env inside a `bit build` that runs every task in one process, and `bit start` pre-bundles on demand in a server that then keeps running. Closed in a `finally`, so the error paths release it too, and the close itself is guarded - cleanup must not replace the build error that sent us there. No deferral needed here, unlike BundleUiTask: there is one compiler and nothing else bundling alongside it. Checked the other rspack call site while here: `ui-server.ts` hands its compiler to RspackDevServer, which needs it alive to watch, so it stays as is.
zkochan
added a commit
that referenced
this pull request
Aug 14, 2026
…ng on lane-only components (#10611) A `bit ci pr` on a branch that had removed a component from the workspace failed on every run, in three successive places. All three come from the same gap: a lane can carry components the workspace does not track, and neither the switch nor the sync handled that. ### `--workspace-only` was accepted and never used `LaneSwitcher` takes `existingOnWorkspaceOnly` and does nothing with it, so `bit lane switch --workspace-only` ("checkout only the components in the workspace to the selected lane") checked out everything on the lane, writing components back into `.bitmap` and onto disk. `bit ci pr` relies on that flag: there the git checkout is the source of truth and the lane only supplies version pointers. Now the switch filters its ids by `.bitmap` when the flag is set. ### `applyVersion` threw for a component that isn't on disk Under `--force-ours`, `applyVersion` threw `applyVersion expect to get componentFromFS` for a lane component absent from the workspace. "Keep ours" only means something when there are local files to keep; with none - which `isLane` explicitly allows - it now falls through and writes the component from the model instead of throwing. ### The config sync staged main's config for components that can't be snapped `syncConfigFromMain` wrote an unmerged entry for every lane component whose head on main had moved. `bit snap` collects during-merge components from those scope-level entries and resolves each against `.bitmap`, so an entry for a component the workspace doesn't track aborted the snap with `MissingBitMapComponent`. The sync exists to feed that snap, so a component that cannot be snapped is skipped before the model load. ### And the flag's other caller Making `workspaceOnly` effective exposed that `bit ci sync`'s `materializeLane` inherited it from `switchToLane`'s defaults, where it is wrong: mirroring a lane onto a branch has to write the whole lane, including components this workspace doesn't track yet. `e2e/harmony/ci-sync.e2e.ts` covers exactly that with a lane-only `comp3`; it now passes `workspaceOnly: false`. ## Testing Developed against #10465, where this set is what got `bit ci pr` past the lane switch and through to a successful snap and export of 171 components. The full e2e suite (40 shards, `ci-sync` and the lane suites included) has run green on that branch with these changes four consecutive times.
zkochan
added a commit
that referenced
this pull request
Aug 14, 2026
…ipt that resolves from node_modules (#10612) Two independent problems in the bundling tasks, both found while getting `bit ci pr` through a build on #10465. ### The UI bundle compilers were never released `BundleUiTask` bundles both UI roots (scope and workspace) and `ui.build()` never closed either rspack compiler. A compiler keeps its whole module graph alive - plus rspack's native side of it - and `bit build` runs every task in one process, so both UI graphs stayed resident through every task that follows, a preview bundle per env among them. The process was OOM-killed partway through those bundles. The compilers are now closed once the task is done, rather than inside `build()`. The webpack bundler in this repo closes its compiler after each run the same way. Why at the end of the task and not per build, stated with the confidence the evidence actually supports: an earlier revision closed each compiler inside `build()`, and that run had the second root fail to resolve a module the first had resolved, while the same code with no closes bundled fine. But that run was also bundling a component's raw sources, because the component's `dist` was missing - a #10465-only condition (eight components had landed on a compiler-less default env) that has since been fixed. With `dist` present the bundle does not walk into component source directories at all, so I would not treat "closing mid-task breaks the other root" as established. Deferring is the conservative choice while there are two compilers in flight; it is not a constraint to design around. Which matters for #10596: if the two roots become one rspack build with two entry points, there is no "other root still bundling", and the tracking added here (`openBuildCompilers` + `deferClose`) collapses into a single close in `build()`'s `finally`. This PR reduces what is held *after* the task; the peak *during* it is still two full compilations of the same app, which is that issue's territory. ### `.ts` reaching the bundler unhandled Both rspack configs excluded `node_modules` from the swc loader, so a bundle that reached TypeScript through `node_modules` died on the first `export type` with `Module parse failed`. That happens for real: a bit component consumed through `node_modules` can resolve to its sources, because an injected pnpm copy is taken from the package directory before the compile fills its `dist`, and the package entry then falls back to `index.ts`. Excluding `node_modules` is about not re-processing already-transpiled third-party JavaScript. A `.ts` file is never valid bundler input wherever it resolves from, so the exclusion now applies to JavaScript only, with `.d.ts` still excluded. ## Testing On #10465 the compiler release is part of what took the measured container peak of the `bit ci pr` build from 16373MB to 11695MB (sampled from the cgroup, 16384MB limit), and `BundleUI` succeeds with both changes in place. The full e2e suite has run green with them four consecutive times. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conflict resolutions: - .circleci/config.yml: master's revert of --keep-lane (#10615) applied inside this branch's memory-sampler wrapper for the `bit ci pr` step. - scopes/react/react/react.env.ts, react.main.runtime.ts: stay deleted - this branch removes the core env sources from the workspace, so #10610's edits to them are moot. - scopes/preview/preview/rspack/rspack.config.ts: keep the mdxOptions parameter. The import master re-adds is what this branch removed on purpose: @teambit/mdx.modules.mdx-v3-options is ESM-only and has to reach the config through a native import() done by the caller. - scopes/preview/preview/pre-bundle.ts: keep master's closeRspackCompiler, which logs a close failure instead of swallowing it (#10612), and drop this branch's earlier copy. - workspace.jsonc: both sides - master's webpack-bundler/webpack-dev-server entries plus this branch's exclude-node-modules-js. - pnpm-lock.yaml: regenerated from master's lockfile with `bit install`, so the full-re-resolution lockfile this branch carried is gone. @teambit/react is now the only added package (1.0.1099, consumed from the registry); everything else is importers and duplicate versions dropping out with the core env sources. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…om-manifest # Conflicts: # .bitmap
@teambit/node, react, aspect, env: 1.0.1042 -> 1.0.1102 @teambit/mdx, readme: 1.0.1043 -> 1.0.1103
…om-manifest # Conflicts: # .bitmap # .circleci/config.yml # pnpm-lock.yaml # scopes/harmony/aspect/aspect.env.ts # scopes/react/react/apps/web/react-app-options.ts # scopes/react/react/apps/web/react.application.ts # scopes/react/react/apps/web/webpack/mutators.ts # scopes/react/react/react.env.ts # scopes/react/react/react.main.runtime.ts # scopes/react/react/webpack/webpack.config.env.dev.ts
@teambit/node, react, aspect, env: 1.0.1102 -> 1.0.1105 @teambit/mdx, readme: 1.0.1103 -> 1.0.1106 react/aspect no longer depend on @teambit/webpack after #10620.
… the core aspect WebpackMain.createBundler/createDevServer, the WebpackBundler/WebpackDevServer classes, and their config factories duplicated what @teambit/webpack.webpack-bundler and @teambit/webpack.webpack-dev-server now do (react/aspect/node envs build through those packages directly, not this aspect - see #10620). Nothing in the codebase calls createBundler/createDevServer anymore. Also drops generateExposePeersTransformer (already marked dead code) and its only helpers (get-exposed-rules, resolve-peer). Kept: the type exports, WebpackConfigMutator/Configuration re-exports, the thin transformer/plugin/fallback re-exports, and the WebpackMain class itself (gutted, so the aspect still loads for backward compatibility).
…apture check_circular_dependencies was the only bit-executing job left on the hub-stg (staging) default while bit_pr/bit_merge use production hub; building the workspace graph here can trigger a live scope import of uncached component objects, and staging appears to be the likely cause of the sporadic 10m no-output CI hangs. Also add a script-level timeout so a stall fails fast with a clear message, print the resolved bit binary/version/hub_domain for diagnostics, and persist debug.log as a CI artifact.
Mirrors e2e's --bit_bin. Lets us pin/compare which bit binary these scripts run against (repo build vs a bvm-linked release) instead of relying on whatever "bit" resolves to on PATH - useful for narrowing down the CI hang, which turned out to reproduce locally too: the repo's own binary took ~3x longer than a released bvm build on the same workspace (5m50s vs 1m57s for `insights circular --json`).
An auto-merge with bit-ci's version-bump commit reintroduced 7 top-level .bitmap entries (aspect, babel, env, mdx, node, react, readme) for legacy core envs this branch already removed from tracking - their source no longer exists at those paths, so bbit install failed with ComponentNotFoundInPath. The bot's commit predates this branch's removal work, so a naive 3-way merge kept its stale entries instead of the deletion. Removed exactly those 7 blocks; confirmed via diff against merge-base/mine/bot that nothing else in .bitmap needed touching (the remaining changes are the legitimate 2.0.80->2.0.81 version bump). bit status loads cleanly now.
…ce-cycle Verified against two real CI runs and a local repro: execSync's timeout error here doesn't set error.killed, it's a raw ETIMEDOUT from spawnSync, so the clearer message never fired. Also correct the message itself - the CI evidence points at a local force-write storm during aspect loading (every dist file of every loaded aspect written to every duplicate .pnpm-hash variant), not a network/registry stall.
Diagnostic addition to narrow down the perf regression found in this check: run it once with the repo's own binary (gating, as before) and once with the bvm-linked nightly release already fetched by setup_harmony (informational only, never fails the job). Reuses the existing install_bvm/bvm_upgrade commands and the same .bvm cache key setup_harmony populates this pipeline run, so it should mostly reuse the cached bundle rather than re-download. Remove once the regression in the repo binary's aspect-loading path is root-caused.
CircleCI skips steps after a failed one unless that specific step is
marked when:always - it doesn't propagate through a whole reusable
command's inner steps. install_bvm/bvm_upgrade sat after the repo-bit
check step, so when that step failed/timed out (as expected), bvm
setup never ran and the diagnostic comparison silently no-opped
("bbit: command not found", swallowed by the || true). Move the bvm
setup earlier so it runs unconditionally, independent of the repo-bit
check's outcome.
We already know the repo-bit check is slow/fails; running the known- good bvm baseline first gives that signal without waiting on the repo-bit run, and it stays non-blocking (|| true) so it can't skip the repo-bit gate either way.
Repo bit has a confirmed, reproducible perf regression on this check (~5min timeout vs ~1-2min on a bvm release, across 4 environments) - see the new CI-HANG-INVESTIGATION.md for the full root-cause writeup (narrowed to workspace.ts's self-as-aspect recursion, unlocked by isCoreAspect() now returning false for the envs this branch removed from the core manifest). Disabling the repo-bit step until it's fixed so it can't block merges on a broken check; the non-blocking bvm-bit comparison stays as a sanity signal. Re-enable once fixed.
Runtime-instrumented the workspace.ts self-as-aspect branch in a disposable /tmp clone - it's never entered during the hang, disproving the earlier hypothesis. Re-reading the captured debug.log's INFO-level loadAspects lines (not the much noisier DEBUG-level file-write lines) shows the real shape: one single trace root recurses for the entire 5-minute window, alternating consumer-fs-load/extension-merge calls into workspace.loadAspects without ever terminating - not fan-out across many components. Found a near-exact match already diagnosed and fixed (unmerged) on origin/refactor/component-loading-v2-take-3-stage2: commits f9ae003 and its follow-up 1213c36 describe and fix the identical mechanism in WorkspaceAspectsLoader.loadAspects (concurrent calls for different root aspects independently re-isolating a shared env dependency, because isAspectLoaded only flips true after a load completes). Their fix serializes loadAspects through a single queue; measured similarly (13:54 -> 10s on a 311-component workspace). Porting that approach is now the primary recommended next step.
…om-manifest # Conflicts: # .bitmap # pnpm-lock.yaml
@teambit/node, react, aspect, env: 1.0.1105 -> 1.0.1107 @teambit/mdx, readme: 1.0.1106 -> 1.0.1108
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes the env aspects (
teambit.react/react,teambit.harmony/node,teambit.harmony/aspect,teambit.envs/env,teambit.mdx/mdx,teambit.mdx/readme) from the core manifest to slim Bit. They now act like any other env, installed from the registry.New default env:
teambit.harmony/empty-env(core). A totally empty env - no compiler, no tester, no preview, no dependency policy. Components with no env configured use it and work fully offline out of the box (add → compile no-op → tag/snap → export). Since it has no behavior, it has nothing to drift when bit itself changes - the one env that is safe to keep core (and versionless in models) forever. To get a dev experience, users configure a real env (bit createflows already do).teambit.harmony/aspectandteambit.envs/envare removed like the rest, with zero behavior change. Their implementation is untouched (react-based, preview and all) - users get the exact released behavior afterbit install(the pinned-version machinery auto-installs them). New envs are created from the bitdev env packages (bit create react-envetc.), so these built-in envs are legacy surface. Thebit-aspecttemplate and the harmony starters moved to the core generator aspect, sobit create bit-aspectandbit newkeep working out of the box (the created aspect needsbit installbefore it loads, like any env).Versionless by design. Config entries for the removed env ids are persisted by name, without a version - exactly as they were when core (registered as core-extension names). Keeping them versionless is deliberate on two counts. First, it keeps the env from becoming a dependency edge of its own components; otherwise an env such as react, whose dependency closure includes components that use it as their env, creates circular TS project references and breaks lane/tag builds. Second, it preserves forward compatibility: a re-tag under the new bit keeps the env id versionless, so a teammate who has not upgraded yet (whose bit still ships these as core) can import the re-tagged component and resolve the env - instead of receiving a versioned id their bit has no component for. The alternative (showing the component as modified and pinning the env on the next tag) would silently break not-yet-upgraded consumers.
Backward compatibility. Old components have the removed envs saved without a version.
legacy-core-envs.tsmaps them to pinned versions, applied only at the resolution/loading/install level - stored objects are never mutated. Versionless legacy ids match the env slot ignoring version,bit installauto-adds their packages, and single-instance semantics are enforced (a loaded version is reused rather than loading another copy). Not-installed legacy envs fail fast with aNonLoadedEnvissue suggestingbit install- no scope-capsule isolation in workspace context (which used to take minutes). Old components load without being reported as modified, and re-tagging keeps the env versionless - covered end-to-end bye2e/harmony/legacy-core-env-back-compat.e2e.ts, which imports a component exported by a pre-removal bit (env saved versionless) and asserts it is not modified and stays versionless after a re-tag.Relocated core wiring: the
bit aspectCLI command moved toteambit.workspace/workspace;validateBeforePersistHookmoved toteambit.dependencies/dependency-resolver; the dead@teambit/legacylink is now skipped instead of crashing.Also fixes latent issues this path exposed: versionless seeders filtering out all manifests in
loadExtensionsByManifests, circular env chains causing infinite component-load recursion, versioned core-aspect ids escaping core filters anddoRequiremutating shared core manifests, stack overflows from recursive graph traversal, and a spuriousMissingDistsissue for compiler-less envs.Verified locally: fresh workspace (JS and TS components) - clean status in ~1s, tag/snap/export offline,
bit envs/bit testgraceful; this repo's workspace - status/insights/list-core clean; the seven repo components that relied on the default env are now explicitly set to the node env.bit create <template> --env <removed-env>loads the env's templates on demand from the global scope (pinned version); this path also loads the full manifest graph, and binds manifest deps of legacy envs to their pinned versions (models built when these envs were core don't list them as dependencies). The e2esetCustomEnvhelper installs the env package the fixture imports (e.g.@teambit/node).Also removes the former-core env sources from this repo's workspace (
scopes/harmony/node,scopes/react/react,scopes/harmony/aspect,scopes/envs/env,scopes/mdx/mdx,scopes/docs/readme) - bit now dogfoods them as installed packages like any consumer, and the source-vs-installed duality is gone. Making this pass end-to-end surfaced several general fixes that ride along:.docs.mdximports are detected even when the mdx aspect isn't loaded (latent gap once mdx is no longer core - without it, docs deps silently drop from dependency computation and preview bundling fails).Module._extensionsrequire hooks are restored after each build task. An in-process tester leaves@babel/register's pirates hook installed; the hook claims all.jsfiles (including node_modules, regardless of babelignoreconfig) and breaksrequire()of ESM-only packages in every later task in the process (pirates drops theformatarg node >=22.12 uses to routerequire(esm)).import()instead of a top-level require, immune to the same stale-hook hazard.@bit-no-check; timings manifest covers the split spec files so shard balancing accounts for the heavier env-install suites.