Skip to content

Add Doc Detective + Promptless blog post#300

Open
InlinePizza wants to merge 5 commits intomainfrom
blog/doc-detective
Open

Add Doc Detective + Promptless blog post#300
InlinePizza wants to merge 5 commits intomainfrom
blog/doc-detective

Conversation

@InlinePizza
Copy link
Contributor

Summary

  • Adds a new product updates blog post announcing Doc Detective support in Promptless
  • Content sourced from the Notion draft and adapted to the site's MDX blog format
  • No images included yet — the Notion source image needs to be uploaded to S3

Test plan

  • Verify the post renders correctly on the local dev server
  • Confirm frontmatter fields (title, date, section) display properly
  • Review content for any needed edits before publishing
  • Upload and add the hero image once available

Announce Doc Detective support in Promptless as a product update.
@vercel
Copy link

vercel bot commented Mar 12, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
promptless-docs Ready Ready Preview, Comment Mar 17, 2026 9:15pm

Request Review

Copy link

@hawkeyexl hawkeyexl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it overall, but it doesn't address why you might need Doc Detective is you already have Promptless, how they're two sides of the same coin. Promptless reduces drift, but Doc Detective verified that any content, human-created or AI-generated, is accurate.


It handles the messiness of real web applications too. You can configure wait conditions that monitor network activity and DOM mutations, so the test only proceeds once the page has actually finished loading. Useful when your single-page app is still fetching data three seconds after navigation.

The screenshot capability goes further than just capturing images. Doc Detective compares screenshots against reference images pixel by pixel. You set a variation threshold (say 5%), and if the current state of the UI has drifted beyond that, the test flags it. So when someone changes the product and the docs still show the old screenshots, you find out from your test suite, not from a confused user filing a support ticket.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repeated in its own section later on.


A specification contains tests, each test is a sequence of steps, and each step is an action:

```json

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole object is completely wrong. We haven't used the "action" key since v2.

}
```

You can also embed tests directly in Markdown as comments, or use automatic markup detection. Doc Detective can parse instructions like "Go to [the dashboard](https://app.example.com/)" and generate the corresponding test action from the Markdown itself.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're likely going to need code syntax here so the markdown doesn't render.


## Where does it fit in your workflow?

Doc Detective runs from the command line. Put it in your CI/CD pipeline. Every time someone changes the docs or the product, the doc tests run. Failures show up alongside your code test failures. The coupling between docs and product becomes something you actively monitor, not something that silently degrades.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Doc test failures show up..."

Comment on lines +26 to +46
## If you have Promptless, why do you need Doc Detective?

Promptless fixes docs drift. When a pull request renames an API field or changes
a workflow, Promptless identifies what needs to be updated in the docs before the
code ships. It monitors pull requests, Slack, Discord, and support channels so
your writers do not have to manually track every engineering change. The most
valuable thing Promptless does is catch drift before it reaches your customers.

But products are complex and teams move fast. Not every change that affects the
docs will be caught before it ships. Doc Detective is your safety net. It takes
the procedures your docs describe ("click this button," "send this request,"
"run this command") and executes them against the real product. If the result
does not match what the docs say, the test fails. No interpretation, no
inference. It is a test suite for your documentation, closer to Cucumber than to
an AI agent, and cheap enough to run on every PR or on a schedule.

When a Doc Detective test fails, that is definitive proof that your docs have
drifted. Promptless picks up that failure, figures out what the documentation
needs to say to match the behavior Doc Detective actually observed, and makes
the fix. Promptless also uses your product directly to write new Doc Detective
tests, so your test coverage grows as your docs grow.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## If you have Promptless, why do you need Doc Detective?
Promptless fixes docs drift. When a pull request renames an API field or changes
a workflow, Promptless identifies what needs to be updated in the docs before the
code ships. It monitors pull requests, Slack, Discord, and support channels so
your writers do not have to manually track every engineering change. The most
valuable thing Promptless does is catch drift before it reaches your customers.
But products are complex and teams move fast. Not every change that affects the
docs will be caught before it ships. Doc Detective is your safety net. It takes
the procedures your docs describe ("click this button," "send this request,"
"run this command") and executes them against the real product. If the result
does not match what the docs say, the test fails. No interpretation, no
inference. It is a test suite for your documentation, closer to Cucumber than to
an AI agent, and cheap enough to run on every PR or on a schedule.
When a Doc Detective test fails, that is definitive proof that your docs have
drifted. Promptless picks up that failure, figures out what the documentation
needs to say to match the behavior Doc Detective actually observed, and makes
the fix. Promptless also uses your product directly to write new Doc Detective
tests, so your test coverage grows as your docs grow.
We see Promptless and Doc Detective as two sides of the same coin. Promptless helps prevent and remediate docs drift, and Doc Detective _deterministically validates_ that any content—whether written by Promptless or a human—is faithful to the product that it documents.
Companies that are the most serious about guaranteeing the accuracy of their docs should use Promptless and Doc Detective together, and now Promptless users can get Doc Detective tests created automatically with every Promptless suggestion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants