feat(telemetry): per-tag, provenance, and failure-taxonomy properties on render events - #4004
Merged
Merged
Conversation
…lete
countElementTags's single regex scan (the ~83% of renders that get no
live probe session) is extended into scanElementTags, which returns a
per-tag breakdown and an a-roll video count alongside the same total,
all from one pass over the same script/style-stripped markup — the
total is the sum of the per-tag map by construction, so the two
properties can't drift apart. countElementTags itself is removed; it
had zero production callers after this change.
New render_complete properties: composition_element_tags ({tag: count},
raw lowercase tag names, capped at the top 50 by count with the rest
folded into an "other" key) and aroll_video_count (<video
data-aroll="true"> elements from the same scan). Both are static-scan-
only, mirroring the existing composition_element_count_source split —
the live probe-DOM path never runs this scan.
Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
heygenVideoCount joins arollVideoCount/byTag as a third property derived from scanElementTags's single pass: <video data-media-source="heygen"> elements. media-use's resolve.md now documents that the skill stamps this attribute on a mounted video only when the ledger record's provider is "heygen.video" (the mount itself is always the agent's own composition edit, never written by resolve scripts) — the ledger stays the single source of truth for provenance, the HTML attribute is a projection of it, and no other provider value is invented beyond this one signal. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
error_name (error.name, e.g. one of the ~20 typed producer error classes) and failed_stage_code (job.currentStage/failedStage normalized to a stable snake_case code via normalizeStageCode) join the existing free-text error_message/failed_stage, so a fleet query can bucket failures without parsing prose. Error-conditional by nature — render_complete has no equivalent, since a successful render has no error to name or stage to blame. Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
miga-heygen
changed the title
miguel-heygen
approved these changes
Sep 16, 2026
Merged
miga-heygen
added a commit
that referenced
this pull request
Sep 16, 2026
- Portrait compositions render the full frame when the project was scaffolded at a different size: html/body are sized to the composition root at runtime; new lint warning root_dimensions_mismatch (#4005, fixes #4001) - render_complete telemetry adds composition_element_tags, aroll_video_count, heygen_video_count; render_error adds error_name and failed_stage_code (#4004) - Catalog docs "Source" accordion titles are plain strings, so docs search no longer shows [object Object] (#4006) Co-Authored-By: Miguel Ángel <miguel.sierra@heygen.com>
miguel-heygen
added a commit
that referenced
this pull request
Sep 16, 2026
- Portrait compositions render the full frame when the project was scaffolded at a different size: html/body are sized to the composition root at runtime; new lint warning root_dimensions_mismatch (#4005, fixes #4001) - render_complete telemetry adds composition_element_tags, aroll_video_count, heygen_video_count; render_error adds error_name and failed_stage_code (#4004) - Catalog docs "Source" accordion titles are plain strings, so docs search no longer shows [object Object] (#4006) Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
render_completetelemetry reportscomposition_element_count(an integer total) on essentially every render, but nothing names which elements make it up.registry_blocks_usedonly fires for catalog-sourced blocks, so hand-authored elements — the overwhelming majority — have no identity in the wire format.countElementTags(the regex scan already run on every render without a live probe session) is replaced byscanElementTags, which returns{total, byTag, arollVideoCount, heygenVideoCount}from the same single pass over the same script/style-stripped markup:totalis the sum ofbyTag's values by construction, so the properties cannot drift apart.countElementTagsitself is removed — after this change it had zero production callers.render_completeproperties from that one scan:composition_element_tags: an object,{tag: count}, raw lowercase tag name — native elements likevideo/img/divand custom elements likehf-audio-groupare both kept as-is, no grouping — capped at the top 50 tags by count with the remainder folded into anotherkey so a pathological composition's distinct-tag cardinality can't inflate the event payload.aroll_video_count:<video data-aroll="true">elements.data-aroll="true"can be stamped on any media element type the composition generator emits (video/image/audio), but this count is scoped to<video>only, matching its name — a-roll is a video-editing term for primary-take footage.heygen_video_count:<video data-media-source="heygen">elements. Themedia-useskill stamps this attribute on a mounted video only when the ledger record's provider traces toheygen.video— the ledger is the single source of truth for provenance, the HTML attribute is a projection of it. Mounting a resolved asset into a composition is always the calling agent's own edit (the skill'sresolvecommand never writes composition HTML itself), so the stamping instruction lives in the skill's own reference doc rather than in an HTML-emitting function.composition_element_count_sourcesplit.render_errorgets a bucketable failure taxonomy alongside the existing free-text fields:error_name(error.name— one of the producer's ~20 typed error classes, e.g.CaptureFailure,DrawElementCaptureError,SwiftShaderAssertionError) andfailed_stage_code(the samejob.currentStage/failedStagevalue normalized to a stable snake_case code via a small map with a slugified fallback for anything unrecognized, so a future stage string still gets a distinct code without needing the map updated first). These are error-conditional by nature —render_completehas no equivalent, since a successful render has no error to name or stage to blame.Not included
Everything below would need a browser-probe change (touching the live
page.evaluateinit call), cross-PR coordination, or was judged lower value relative to the risk of rushing it in the same pass as the above — left for a focused follow-up:HF_*env-override names — same category: clearly specified, no PostHog schema change needed, but each needs its own careful wiring + test pass and didn't fit in this round without risking quality.<body>) landing in parallel — the property needs to keep reporting the raw measured mismatch accurately regardless of which lands first, not assume the other's outcome.Test plan
hf-*tag grouping, case-normalization, the cap-overflow-into-otherscenario, a count/map consistency invariant (sum(byTag) === total), the<video data-aroll>/<video data-media-source="heygen">scope (audio/img/other-provider values are not counted), and zero-match cases reporting0/{}rather thanundefined.resolveCompositionElementCounttests updated to assert the new fields are present on every "static" result and absent on the "live" result.normalizeStageCode: every knownupdateJobStatusstage string maps to its code, an unrecognized string slugifies instead of bucketing into "unknown", and empty/punctuation-only input falls back to "unknown".packages/producer/src/services/renderOrchestrator.test.ts,packages/cli/src/telemetry/events.test.ts,packages/cli/src/telemetry/renderObservability.test.ts,packages/cli/src/commands/render.test.ts— all passing. Plain unit tests with no browser/ffmpeg dependency, running in the existingproducer-source-tests(unit lane) / CLI test jobs with no additional CI setup needed.typecheck,oxlint,oxfmt --checkclean on every touched file.