Skip to content

Reject invalid YAML frontmatter in a single, self-identifying test#5264

Open
ahpook with Copilot wants to merge 3 commits into
mainfrom
copilot/analyze-actions-logs-issue
Open

Reject invalid YAML frontmatter in a single, self-identifying test#5264
ahpook with Copilot wants to merge 3 commits into
mainfrom
copilot/analyze-actions-logs-issue

Conversation

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

A single malformed topic frontmatter file (e.g. display_name:zork missing the space after the colon) produced ~825 identical, non-specific invalid YAML: ... at line 3 column 18 failures, because metadata_for flunked on parse errors and the alias cross-check test re-parses every other topic — so one bad file flunked once per topic, never naming the culprit.

Changes

  • Decouple parsing from assertion (test/test_helper.rb): metadata_for now only parses, returning nil on Psych::SyntaxError instead of calling flunk. This stops the cascade through tests that read other items' metadata.
  • Self-identifying YAML check: new yaml_syntax_error_for returns a path-qualified message (invalid YAML in topics/zork/index.md: ...) or nil.
  • Isolated per-item test: added it "has valid YAML frontmatter" to topics_test.rb and collections_test.rb, each parsing only its own index.md.
  • Dedup: extracted a shared frontmatter_for helper used by both metadata_for and yaml_syntax_error_for.

Result

A broken zork/index.md now fails exactly one test, pointing straight at the file:

topics::zork topic#test_0001_has valid YAML frontmatter [test/topics_test.rb]:
invalid YAML in topics/zork/index.md: (<unknown>): mapping values are not allowed in this context at line 3 column 18

Other items' cross-checks treat an unparseable foreign file as having no aliases (its own YAML test already reports the failure), so no incidental noise.

@ahpook
ahpook marked this pull request as ready for review July 24, 2026 18:30
@ahpook
ahpook requested a review from a team as a code owner July 24, 2026 18:30
Copilot AI review requested due to automatic review settings July 24, 2026 18:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds targeted YAML frontmatter validation while preventing cross-topic parse-error cascades.

Changes:

  • Extracts shared frontmatter parsing.
  • Adds path-qualified YAML syntax errors.
  • Adds per-topic and per-collection syntax tests.
Show a summary per file
File Description
test/test_helper.rb Separates parsing from syntax reporting.
test/topics_test.rb Adds topic YAML validation.
test/collections_test.rb Adds collection YAML validation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread test/test_helper.rb
Comment on lines +254 to +255
rescue Psych::SyntaxError
nil
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants