Draft
Conversation
Phase 3.1: Add "shortcodes" to PATH_VALUED_KEYS in extension/read.rs so per-format shortcode paths are marked as ConfigValueKind::Path during extension metadata parsing. Phase 3.2: Create LuaShortcodeEngine in pampa/src/lua/shortcode.rs - Lua state setup with pandoc/quarto globals (WASM-aware) - Script loading via both conventions (return-table and env-function) - Handler dispatch with TS Quarto calling convention (args, kwargs, meta, raw_args, context) - Return value classification (nil→error, string→text, userdata→ inline/block, table→classify) - quarto.shortcode.read_arg() and error_output() API - pub(crate) extract helpers in filter.rs for userdata unwrapping
- Add ShortcodeResult::Blocks variant for block-level returns - Add ResolutionContext enum (Block/Inline) to ShortcodeHandler::resolve() - Rewrite resolve_blocks() with index-based iteration and two-pass logic: Para/Plain with single shortcode → block context, otherwise inline - Graceful degradation: Blocks in inline context → flatten_blocks_to_inlines - Remove unused AnalysisContext import (re-added as actually needed)
Connect LuaShortcodeEngine to ShortcodeResolveTransform, enabling extension and user-defined Lua shortcode handlers alongside built-in Rust handlers. Key changes: - Add with_lua_support() constructor storing extensions, runtime, format, and shortcode paths as Send+Sync owned data - Create LuaShortcodeEngine on the stack in transform() (it's !Send+!Sync) - Thread lua_engine parameter through all resolution functions - Add name-based extension lookup: unknown shortcodes trigger on-demand loading of matching extension's shortcode scripts - Move pipeline construction from AstTransformsStage::new() to run(), using StageContext data (extensions, runtime, format) for the shortcode transform - Add extract_shortcode_paths() for reading paths from merged metadata - Add helper functions for Lua dispatch and result conversion Handler priority: built-in Rust > loaded Lua > extension name lookup (matches TS Quarto behavior). 5 new integration tests covering Lua shortcode dispatch, extension name lookup, Rust handler override priority, unknown shortcode errors, and block-context extension shortcodes.
… 3.5-3.7) - Add metadata merge test verifying extension shortcode paths preserve ConfigValueKind::Path through build_extension_metadata_layer - Add RawBlock integration test for block-context Lua shortcodes - Add smoke test: inline shortcode extension (hello → text output) - Add smoke test: block shortcode extension (break → RawBlock <hr>) - Fix unused_imports warning in pampa lua/mod.rs for shortcode exports - All 6919 workspace tests pass, cargo xtask verify build passes
- Add smoke test for per-format extension shortcodes (contributes.formats. html.shortcodes path, flows through metadata merge) - Mark Phase 3 complete in both detail plan and master plan - Remove lipsum smoke test item (future built-in shortcode, not infra)
Real-world Quarto extensions (like lipsum) expect a quarto.* global namespace with utility functions beyond warn/error. This adds: - quarto.json: alias of pandoc.json (decode/encode/null) - quarto.log: stderr logging with level-gated output/error/warning/ info/debug/trace functions and setloglevel - quarto.utils.resolve_path: resolves paths relative to the calling script's directory (tracked per handler in shortcode engine) - quarto.utils.type: alias of pandoc.utils.type The shortcode engine tracks script directories per handler name so that resolve_path works correctly when multiple extensions are loaded. The filter engine sets the script dir to the filter's parent directory. New file: crates/pampa/src/lua/quarto_api.rs (22 unit tests) Modified: shortcode.rs (4 new integration tests including multi-extension) Modified: filter.rs, filter_tests.rs (2 new integration tests)
- Rewrite waitForPreviewRender to poll with fast-fail on fatal WASM errors - Collect console errors, page errors, and HTTP 500s for diagnostic context - Sync QMD render target last so extension files land in VFS before first render
- Re-enable MetadataMergeStage in parse_qmd_to_ast so format is injected into merged metadata (needed by formatDetection tests and PreviewRouter). Remove the commented-out AstTransformsStage entirely — parse_qmd_to_ast intentionally returns a pre-transform AST for Elliot's debug renderer. - Delete stale PreviewRouter.test.ts (file moved to render/ subdirectory, test was never visible in CI since test:ci was disabled). - Uncomment test:ci step in ts-test-suite.yml so hub-client tests run in CI.
gordonwoodhull
force-pushed
the
feature/shortcode-extensions
branch
from
bebd2c6 to
5694ba6
Compare
Collaborator
Author
Note for developers: some hub-client tests running in CI for the first timeSpecifically, the hub-client unit/integration/WASM tests. Not the playwright tests yet; those must be run manually. This PR enables
None of these were running in CI before today. The most significant newly-covered test is the WASM smoke-all runner, since it exercises the full render pipeline including extension shortcode support introduced in this PR. As a side effect of enabling this, we found and fixed two pre-existing issues:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Lua-based shortcode support for Quarto extensions, allowing extensions to define custom shortcodes via Lua scripts (e.g.,
{{< hello >}}). This is Phase 3 of the extensions roadmap, building on the extension discovery/metadata (Phase 1) and filter resolution (Phase 2) work already merged to main.What's new
Lua shortcode engine (
pampa/src/lua/shortcode.rs): Loads and dispatches Lua shortcode handlers. Supports bothreturn-tableandenv-functionLua conventions, matches TS Quarto's calling convention (args,kwargs,meta,raw_args,context), and handles inline and block-level return values.Block-level shortcode results: Added
ShortcodeResult::Blocksvariant with two-pass resolution logic — aPara/Plaincontaining a single shortcode gets block context; otherwise falls back to inline with graceful flattening.Pipeline wiring:
AstTransformsStagenow builds the transform pipeline JIT inrun()(moved fromnew()) so it has access toStageContext— extensions, runtime, and format — for the shortcode transform. Unknown shortcode names trigger on-demand loading of matching extension Lua scripts.quarto.*Lua API (pampa/src/lua/quarto_api.rs): Addsquarto.json,quarto.log, andquarto.utilsnamespaces expected by real-world extensions (e.g., lipsum). Includesquarto.utils.resolve_pathwith per-handler script directory tracking.Per-format shortcode paths: Extension metadata
shortcodeskeys under format-specific sections are now marked asConfigValueKind::Pathso they resolve correctly relative to the extension directory.Smoke tests: Three new smoke-all fixtures covering inline shortcodes, block shortcodes, and per-format shortcode dispatch.
CI fix
MetadataMergeStageinparse_qmd_to_ast(was inadvertently commented out alongsideAstTransformsStage). This restores format injection into merged metadata, fixing 4formatDetection.wasm.test.tsfailures.AstTransformsStagefromparse_qmd_to_astentirely — the function intentionally returns a pre-transform AST for the React debug renderer.test:ciints-test-suite.ymlso hub-client tests actually run in CI.PreviewRouter.test.ts(component moved torender/subdirectory; test was never visible in CI).Test plan
test-suite.yml— Rust nextest including smoke-all)ts-test-suite.yml— hub-client test:ci, now enabled for the first time)hub-client-e2e.yml— Playwright smoke-all)shortcode-extension,filter-extension) returning 401 — under investigation