Ai manpage - #4899
Open
bdbaddog wants to merge 5 commits into
Open
Conversation
The scons(1) man page is too large for a language model to read in full: the roff scons.1 is roughly 230k tokens and the generated HTML about 337k. Add doc/man/scons-ai.md, a condensed Markdown reference of roughly 113k tokens - about half the roff man page - which keeps every documented entry (each option, tool, builder, function, construction variable, configure context method and node attribute, with its description) but drops markup, cross-reference chrome, code examples, Note blocks and version history. The generator, bin/scons-man-to-ai.py, reads the checked-in DocBook source doc/man/scons.xml rather than built documentation, so it needs no doc build and works from a clean checkout. It resolves the parameter entities and XIncludes that supply the bulk of the content, which requires lxml (already a development dependency). The generated man page HTML is also accepted via --input for comparison against the documentation toolchain; both inputs produce the same set of entries. Wire it into doc/SConscript as the 'ai-manpage' alias, also reachable from the 'doc' alias. Because it builds from checked-in sources it is generated even when SKIP_DOC is set and the rest of the doc build is skipped. Also unwrap two <variablelist> elements in the Decider documentation that were nested inside a <para>. That nesting is schema-valid but is a 2-of-214 anomaly, and the DocBook XSL hoists the list out of the paragraph, leaving four empty <p></p> elements in the rendered HTML man page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZmNvpfrD52gbqknUecffc
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZmNvpfrD52gbqknUecffc
Install the condensed AI manpage alongside the other rendered documentation formats, matching the HTML/PDF/EPUB/TEXT layout, and include it in the documentation tarball. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZmNvpfrD52gbqknUecffc
Collaborator
|
Guess it's okay. Since the result isn't human-friendly, nor human-intended, it's hard to say much about it. How would this be provided? you planning to put it in doc/man even though it's "generated"? |
Contributor
Author
not sure. the .gen files are under doc/generated.. if we put it there and then referenced it in our agents.md maybe? |
Collaborator
I'd say that makes sense. |
Build the condensed AI manpage directly to build/doc/MD/scons-ai.md rather than writing it into doc/man and installing a copy. It is generated output, so it does not belong in the source tree; remove the checked-in copy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WZmNvpfrD52gbqknUecffc
Render scons-ai.md into doc/generated/scons-ai.md, matching the checked-in doc/generated/*.gen and *.mod files, instead of writing directly into the build tree. The build now installs a copy into build/doc/MD alongside the other rendered formats. Mark the generated file Precious/NoClean so 'scons -c' does not delete the checked-in copy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xrv6wmE3cfEhYmsTiZpnsZ
bdbaddog
marked this pull request as ready for review
September 6, 2026 00:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added bin/scons-man-to-ai.py which translates the xml manpage into a minimized markdown file suitable for AI to use. (reduces from 230k (scons.1) or 337k (scons-man.html) to 113k tokens with scons-ai.md
Should this be built and checked into doc/man/scons-ai.md, or only built into build/doc/MD/scons-ai.md?
In the process of this work, found that there were a few variablelist's inside para's where there was nothing else in the para's so simplified those, to also simplify the markdown generation script
(Built with Claude Opus 5)
Contributor Checklist:
CHANGES.txtandRELEASE.txt(and read theREADME.rst).