perf(cpa): use AST for templates #14648
Open
+15,518
−4,722
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement AST-based detection of buildConfig call structure in Payload config files using TDD approach. Detects: - buildConfig CallExpression - Import declarations - db property assignment - plugins array All tests pass (3/3).
Implements removeSharp function using TDD to remove sharp import and property from payload config files. This completes Phase 3 - Transformation Implementation.
Refactored configure-payload-config.ts to use AST-based transformations instead of string manipulation: - Replaced string manipulation logic with calls to configurePayloadConfig() and updatePackageJson() from AST modules - Added helper functions: mapDbType(), mapStorageAdapter(), and getEnvVarName() - Enhanced AST payload-config module with removeCommentMarkers() to clean up template markers - Fixed quote style conversion (double to single quotes) - Added indentation normalization to handle ts-morph's formatting quirks - Updated database and storage adapter templates with proper indentation - All existing integration tests pass
Removed AST comment markers from all template payload.config.ts files: - storage-adapter-import-placeholder - database-adapter-import - database-adapter-config-start - database-adapter-config-end - storage-adapter-placeholder - sharp-import Normalized template structures with clean imports and empty plugins arrays where applicable. Templates cleaned: - templates/_template/src/payload.config.ts - templates/blank/src/payload.config.ts - templates/website/src/payload.config.ts - templates/ecommerce/src/payload.config.ts - templates/with-cloudflare-d1/src/payload.config.ts - templates/with-vercel-website/src/payload.config.ts
Removed string-based replacement utilities that were superseded by AST-based config manipulation. Updated test to verify adapter configuration directly instead of using replacement templates.
Fixed .js extension in AST utils import and updated pnpm-lock.yaml with prettier version.
Add integration tests for sampled template × database × storage combinations. Tests verify AST transformations are correctly applied to payload.config.ts and package.json for representative combinations: - blank + mongodb + localDisk - blank + postgres + vercelBlobStorage - website + mongodb + s3Storage - website + postgres + localDisk - ecommerce + mongodb + localDisk - ecommerce + postgres + r2Storage Tests copy template to temp directory, apply transformations via configurePayloadConfig(), and verify imports/config properties are correctly modified. Skip TypeScript compilation for speed.
- Add .js extension to types import in package-json.ts - Add type assertion for DB_PACKAGE_NAMES lookup - Import QuoteKind enum from ts-morph - Add null checks for regex match results in indentation normalization - Remove invalid indentationText setting
…, azureStorage, gcsStorage, uploadthingStorage)
Convert all interfaces to types in AST implementation:
- types.ts: 7 interfaces (DetectionError, PayloadConfigStructures, etc.)
- adapter-config.ts: 2 interfaces (DatabaseAdapterConfig, StorageAdapterConfig)
- utils.ts: 2 internal interfaces (FormatErrorOptions, AddImportOptions)
- package-json.ts: 2 internal interfaces (PackageJsonTransformOptions, PackageJsonStructure)
ConfigureOptions now uses intersection type (WriteOptions & {...}) instead of extends.
…ations Add debug logging throughout AST transformation pipeline when --debug flag is passed: **payload-config.ts:** - Detection phase: buildConfig found/missing, import counts, structure details - Transformations: adapter types, imports added/removed, special imports (vercel-postgres, d1-sqlite) - Validation: structure validation results - Write phase: quote/indent normalization, prettier status, file operations **utils.ts:** - Import operations: additions to existing vs new imports, removals **package-json.ts:** - Dependency updates: adapter changes, sharp removal, package name updates **Integration:** - configure-payload-config.ts: accepts debugMode parameter - create-project.ts: passes cliArgs['--debug'] to AST functions All debug messages use [AST] prefix with status indicators (✓ success, ✗ failure, ⚠ warning).
… code - Set process.env.DEBUG in main.ts when --debug flag is passed - Update debug() function to check process.env.DEBUG internally - Remove debugMode parameters from all AST functions (56 if-blocks eliminated) - Update all call sites to remove debugMode arguments - Remove debugMode from WriteOptions and ConfigureOptions types This eliminates ~150 LOC and improves code readability by removing conditional debug blocks throughout the codebase.
… with 2+ params
- addDatabaseAdapter: (sourceFile, adapter, envVarName) → (sourceFile, { adapter, envVarName })
- addStorageAdapter: (sourceFile, adapter) → (sourceFile, { adapter })
- findImportDeclaration: (sourceFile, moduleSpecifier) → (sourceFile, { moduleSpecifier })
- removeImportDeclaration: (sourceFile, moduleSpecifier) → (sourceFile, { moduleSpecifier })
Benefits:
- Improved backwards-compatibility (can add new options without breaking changes)
- More explicit at call sites
- Consistent with existing functions (addImportDeclaration, configurePayloadConfig)
All 64 tests passing.
… AST code Replace dynamic prettier import with CLI execution to resolve Jest/ESM issues. - Add detectPackageManager() for pnpm/npm/yarn/bun detection - Refactor writeTransformedFile() to call prettier CLI after save - Remove 50+ lines of manual normalization code - Enable formatWithPrettier by default - Update tests to use prettier (remove formatWithPrettier: false)
Contributor
📦 esbuild Bundle Analysis for payloadThis analysis was generated by esbuild-bundle-analyzer. 🤖
Largest pathsThese visualization shows top 20 largest paths in the bundle.Meta file: packages/next/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js
Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js
DetailsNext to the size is how much the size has increased or decreased compared with the base branch of this PR.
|
denolfe
changed the title
denolfe
changed the title
denolfe
changed the title
Member
Author
|
Manual testing notes:
|
Derive DbType and StorageAdapterType from ast/types.ts arrays, eliminating duplicate definitions and unsafe `as` casts.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Refactor
create-payload-appfrom string manipulation to AST-based transformations. This will allow us to be more flexible in the future if we want to add more functionality.Key Changes
AST transformation engine (
packages/create-payload-app/src/lib/ast/)Template normalization
Improved reliability
Design Decisions
AST over string manipulation
String-based replacements were brittle and error-prone with variations in formatting, import order, and code
structure. Using
ts-morphprovides:Adapter configuration centralization
Database and storage adapter configs are now centralized in
adapter-config.tsrather than scattered acrosstemplate files and replacement logic. This reduces duplication and makes adding new adapters
straightforward.
Template cleanup
Removed comment markers (e.g.,
// database-adapter-import-start) from templates since AST can locateinsertion points structurally. Templates now serve as clean, production-ready code without scaffolding
artifacts.
Overall Flow
sequenceDiagram participant CLI as create-payload-app participant AST as AST Engine participant TS as ts-morph participant Template as Template Files CLI->>Template: Copy template CLI->>AST: configurePayloadConfig(options) AST->>TS: Parse payload.config.ts AST->>TS: Transform config structure Note over AST,TS: Replace DB/storage adapters, add/remove plugins, update imports TS->>AST: Modified AST AST->>AST: Format with Prettier AST->>Template: Write updated file AST->>Template: Update package.json