Skip to content

36002 workflow fire convert block editor markdown to prosemirror json on save server side#36253

Open
hassandotcms wants to merge 3 commits into
mainfrom
36002-workflow-fire-convert-block-editor-htmlmarkdown-to-prosemirror-json-on-save-server-side
Open

36002 workflow fire convert block editor markdown to prosemirror json on save server side#36253
hassandotcms wants to merge 3 commits into
mainfrom
36002-workflow-fire-convert-block-editor-htmlmarkdown-to-prosemirror-json-on-save-server-side

Conversation

@hassandotcms

Copy link
Copy Markdown
Member

What

Converts Story Block (Block Editor) field values supplied as Markdown to Tiptap/ProseMirror JSON server-side, on the shared content save path. Non-interactive clients (AI agents, headless imports) no longer need a human to open and re-save the contentlet for the field to read back as structured content.

Closes #36002 (Markdown scope; see Scope below).

How

  • TiptapMarkdown.isTiptapDoc / isMarkdownRepresentable — pure discriminators.
  • MapToContentletPopulator.fillFields — the seam shared by the workflow fire endpoints and the content REST API. For a Story Block value:
    • starts with { (JSON) or < (HTML) → stored unchanged;
    • otherwise (Markdown) → converted via TiptapMarkdown.toTiptap.
  • Guards: already-valid JSON passes through untouched; a Markdown update is rejected (400) when the existing stored doc contains rich blocks Markdown can't represent (dotContent, dotVideo, grid, …) instead of silently destroying them; a conversion failure never blocks the save (stores raw + logs).
  • OpenAPI fire note corrected to reflect automatic server-side conversion (regenerated).

Scope

  • Markdown only. HTML is deferred to a follow-up PR. HTML continues to pass through unchanged (no regression).

Behavior change

  • A Markdown overwrite of rich content now returns 400 instead of a silent success that
    corrupted the field. Additive and rollback-safe (stored JSON is read natively by N-1).

Testing

  • Unit (65): TiptapMarkdownDocDetectionTest (13) + existing TiptapMarkdownTest /
    RoundTripContractTest.
  • Integration (5): StoryBlockMarkdownPopulatorTest — convert + GraphQL read-back, JSON
    passthrough, HTML passthrough, primitive replace, rich-overwrite reject.
  • Regression re-run clean: MapToContentletPopulatorTest (20), StoryBlockValidationTest (28).

…minators

Add two pure helpers to TiptapMarkdown that the save path needs to safely
ingest Story Block values:

- isTiptapDoc(String): cheap detector for an already-valid Tiptap/ProseMirror
  document (peeks the first non-whitespace char before parsing), so editor-
  authored JSON can be stored unchanged instead of re-parsed as Markdown.
- isMarkdownRepresentable(String): true only when every block is Markdown-
  expressible, used to refuse a Markdown overwrite that would silently drop
  rich blocks (dotContent, dotVideo, grid, etc.). Marks are ignored on purpose
  (losing a mark loses styling, not content).

Covered by TiptapMarkdownDocDetectionTest (13 cases incl. nested rich blocks,
marks-only docs, malformed/empty/null input).

Refs #36002
…t save path

Wire the converter into MapToContentletPopulator.fillFields, the shared seam
that the workflow fire endpoints and the content REST API all funnel through.
For a Story Block field whose incoming value is Markdown (begins with neither
'{' nor '<'), convert it to a ProseMirror JSON document and store that, so non-
interactive clients (AI agents, headless imports) no longer require a human to
open and re-save the contentlet.

Guards:
- Already-valid Tiptap JSON and (deferred) HTML are stored unchanged.
- A Markdown update is refused when the existing stored document contains rich
  blocks Markdown cannot represent, rather than silently destroying them.
- A conversion failure never blocks the save: the raw value is stored and a
  warning logged (graceful degradation, consistent with #35728).

The converter stays pure; conversion and guards live at the ingestion seam.

Covered by StoryBlockMarkdownPopulatorTest (convert + GraphQL read-back, JSON
passthrough, HTML passthrough, primitive replace, rich-overwrite reject);
registered in MainSuite1b.

Refs #36002
…wn conversion

The fire endpoints' Block Editor note promised Markdown/HTML acceptance but
admitted it only took effect after a human re-saved in the editor — documenting
the exact bug #36002 fixes. Update the shared @operation note to state that
Markdown is converted to ProseMirror JSON automatically on save (and already-
valid JSON is stored unchanged), drop the "converted when opened in the editor"
caveat, and use a Markdown example. Regenerate openapi.yaml (all 6 fire
operations share the constant).

Refs #36002
@claude

claude Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Claude finished @hassandotcms's task in 1m 35s —— View job


Rollback Safety Analysis

  • Gather context from PR description and changed files
  • Read rollback-unsafe categories reference
  • Get full PR diff
  • Analyze diff against every unsafe category
  • Apply appropriate label and post findings

Verdict: ✅ Safe To Rollback

Label applied: AI: Safe To Rollback


Analysis

I analyzed the diff against all categories in ROLLBACK_UNSAFE_CATEGORIES.md. No unsafe patterns were found.

Files changed:

File Change
MapToContentletPopulator.java New processStoryBlockField() method — converts Markdown to Tiptap JSON on save
TiptapMarkdown.java Two new static methods: isTiptapDoc() and isMarkdownRepresentable()
WorkflowResource.java Updated OpenAPI BLOCK_EDITOR_FIELD_NOTE string constant
openapi.yaml Regenerated doc text — description-only change
TiptapMarkdownDocDetectionTest.java New unit test class
StoryBlockMarkdownPopulatorTest.java New integration test class
MainSuite1b.java Test suite wired up to include new integration test

Category-by-category review:

  • C-1 (Structural Data Model): No runonce tasks, no schema mutations. The change only alters the in-flight Java string written to an existing Story Block field's existing column. ✅
  • C-2 (Elasticsearch Mapping): No changes to ESMappingAPIImpl, ESMappingUtilHelper, or ESMappingConstants. No reindex task. ✅
  • C-3 (Content JSON Model Version): No changes to CURRENT_MODEL_VERSION, ImmutableContentlet, or ContentletJsonAPIImpl. ✅
  • C-4 (DROP TABLE/COLUMN): No DDL of any kind. ✅
  • H-1 (One-Way Data Migration): No runonce task, no UPDATE … SET across rows. The conversion only applies to new saves on the ingest path. Existing stored data is never touched. ✅
  • H-2 (RENAME TABLE/COLUMN): None. ✅
  • H-3 (PK Restructuring): None. ✅
  • H-4 (New Field Type): No new field type registered. The STORY_BLOCK_FIELD type already exists in N-1. ✅
  • H-5 (Storage Provider Change): None. ✅
  • H-6 (DROP PROCEDURE/FUNCTION): None. ✅
  • H-7 (NOT NULL without default): No schema changes. ✅
  • H-8 (VTL Viewtool Contract): No viewtool classes modified. The new methods are on TiptapMarkdown (a non-viewtool utility class). ✅
  • M-1 (Column Type Change): None. ✅
  • M-2 (Push Publishing Bundle Format): No *Bundler.java or *Handler.java changes. ✅
  • M-3 (REST/GraphQL API Contract): The OpenAPI description text changed but the endpoint signatures, HTTP method, path, request/response shape, and field names are all identical. API consumers that send JSON already work; those that send Markdown now get better behavior. The new 400 rejection path (for Markdown overwrites of rich content) is additive: N-1 never raised this 400 (it silently stored corrupted data), so rolling back to N-1 restores the prior silent behavior — no client is newly broken by the rollback. ✅
  • M-4 (OSGi Interface Change): No OSGi interface modifications. ✅

Additional rollback-safety note: The PR explicitly states the stored format is additive and rollback-safe — "Additive and rollback-safe (stored JSON is read natively by N-1)" — because a Tiptap JSON document stored by N is a valid JSON string that N-1 would have stored and read back in the same way. No migration of existing rows occurs.

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

🤖 Bedrock Review — deepseek.v3.2

[🟠 High] dotCMS/src/main/java/com/dotcms/rest/MapToContentletPopulator.java:259 — The processStoryBlockField method is called for String values, but the earlier condition value instanceof Map for binary fields could be skipped incorrectly. If a Story Block field value is a Map (unlikely but possible), it will be processed as binary, causing data corruption. Add an explicit check for FieldType.STORY_BLOCK_FIELD before the binary field condition.

[🟡 Medium] dotCMS/src/main/java/com/dotcms/rest/MapToContentletPopulator.java:292 — The check trimmed.isEmpty() || trimmed.charAt(0) == '{' || trimmed.charAt(0) == '<' returns the original value for empty strings, but an empty string should be stored as-is without conversion. This is correct, but note that empty JSON ({}) or malformed HTML (<) could be incorrectly passed through. Ensure the logic aligns with the contract: only valid Tiptap JSON or HTML should be preserved.

[🟡 Medium] dotCMS/src/main/java/com/dotcms/rest/MapToContentletPopulator.java:299 — The method contentlet.getStringProperty(field.getVelocityVarName()) may return null for a new contentlet, causing TiptapMarkdown.isTiptapDoc(existing) to return false. This is fine, but the subsequent TiptapMarkdown.isMarkdownRepresentable(existing) will return true (as per its contract for null), allowing the Markdown conversion. This is acceptable behavior for new content.

[🟡 Medium] dotCMS/src/main/java/com/dotcms/tiptap/TiptapMarkdown.java:121 — The isTiptapDoc method uses value.stripLeading() which is available from Java 11. dotCMS uses Java 25, so it's fine, but ensure consistency with other string trimming in the codebase (e.g., StringUtil.trim()).

[🟡 Medium] dotCMS/src/main/java/com/dotcms/tiptap/TiptapMarkdown.java:130 — The method catches java.io.IOException and returns false. This is correct, but note that other runtime exceptions (e.g., OutOfMemoryError) are not caught. This is acceptable as they should propagate.

[🟡 Medium] dotCMS/src/main/java/com/dotcms/tiptap/TiptapMarkdown.java:172 — The isMarkdownRepresentable method recursively checks nodes but does not guard against circular references or extremely deep nesting, which could cause a StackOverflowError. Since the JSON is generated by dotCMS, this risk is low, but consider adding a depth limit for safety.

[🟡 Medium] dotcms-integration/src/test/java/com/dotcms/rest/StoryBlockMarkdownPopulatorTest.java:176 — The test markdown_overwrite_of_rich_content_is_rejected expects an IllegalArgumentException but does not verify that the exception is thrown by the correct code path (i.e., from toStoryBlockJson). This is acceptable for integration testing, but ensure unit tests cover the specific method.

[🟡 Medium] dotcms-integration/src/test/java/com/dotcms/rest/StoryBlockMarkdownPopulatorTest.java:90 — The test uses Mockito to mock DataFetchingEnvironment, but the mock setup is minimal. Ensure the mock does not affect the behavior of StoryBlockFieldDataFetcher in a way that masks bugs. This is acceptable for integration.

[🟡 Medium] dotCMS/src/main/webapp/WEB-INF/openapi/openapi.yaml — Multiple lines updated to change the contract from HTML/Markdown to Markdown-only. This is a breaking change for clients sending HTML. Ensure backward compatibility is maintained (the code still accepts HTML via the < check), but the documentation no longer mentions HTML, which could confuse users. Update the documentation to clarify HTML is still accepted but not recommended.

[🟡 Medium] dotCMS/src/main/java/com/dotcms/rest/api/v1/workflow/WorkflowResource.java:231 — The updated documentation states "send the value as a Markdown string" and de-emphasizes HTML. This aligns with the code change but may mislead users who rely on HTML input. Ensure the documentation reflects that HTML is still accepted (stored as-is) but Markdown is preferred.

The PR is otherwise clean and well-tested.


Run: #27958629180 · tokens: in: 8834 · out: 1000 · total: 9834

@hassandotcms hassandotcms requested review from fmontes and wezell June 22, 2026 14:39
@hassandotcms hassandotcms marked this pull request as ready for review June 22, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Workflow fire: convert Block Editor HTML/Markdown to ProseMirror JSON on save (server-side)

1 participant