Skip to content

Hua work miscs#12

Merged
zaihuaji merged 16 commits into
mainfrom
hua-work-miscs
Mar 24, 2026
Merged

Hua work miscs#12
zaihuaji merged 16 commits into
mainfrom
hua-work-miscs

Conversation

@zaihuaji

Copy link
Copy Markdown
Collaborator

No description provided.

zaihuaji and others added 13 commits March 17, 2026 18:32
- Convert PgDOCS module-level functions into class PgDOCS in pg_docs.py
- Convert pg_rst.py to class PgRST generating RST instead of HTML:
  - All HTML tags replaced with RST equivalents (bold, links, tables, lists)
  - template_to_html renamed to template_to_rst; reads/writes .rst.temp/.rst
  - DCROOT changed to /rstdocs; TMPDIR set to ./rst_templates
  - Added docstrings to all methods
  - Fixed bugs: exmaple typo, opts.insert() None return, underline length,
    two-column table row accumulation, TMPDIR relative path after chdir,
    zero-width RST table column, and Unknown typo
- Rename rst_templates/*.temp to *.rst.temp to match template_to_rst
- Backup previous pg_rst.py as pg_rst.py.bck

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update imports: replace bare `import PgFile/PgUtil` with
  `from rda_python_common.pg_file import PgFile` and
  `from rda_python_common.pg_util import PgUtil`
- Change class declaration to `class PgRST(PgFile, PgUtil):`
  (MRO: PgRST → PgFile → PgUtil → PgLOG)
- Add `super().__init__()` in __init__ to initialise the full MRO chain
- Replace `PgFile.change_local_directory(...)` with `self.change_local_directory(...)`
  now that the method is inherited
- Fix bug: bare `Q0` in replace_option_link → `self.Q0` (NameError otherwise)
- Update class and __init__ docstrings to document the new inheritance

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add #!/usr/bin/env python3 shebang; chmod +x the file
- Add imports: inspect, argparse, importlib
- Add helper _load_opts_alias(docname):
    1. Imports rda_python_<docname> dynamically via importlib
    2. Looks for OPTS / ALIAS at module level first
    3. Falls back to the first class defined in that module that
       carries both as class-level attributes
    4. ALIAS defaults to {} when absent (it is optional)
    5. Aborts via PgLOG.pglog(LGWNEX) on import failure or missing OPTS
- Add if __name__ == '__main__' block:
    Uses argparse to accept a single positional 'docname' argument,
    calls _load_opts_alias(), then PgRST().process_docs()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ocname>.<docname>

_load_opts_alias now imports rda_python_<docname>/<docname>.py
(i.e. the <docname> submodule inside the rda_python_<docname> package)
instead of a flat rda_python_<docname>.py module.

Update all docstring and argparse help text references to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_load_opts_alias now derives origin = op.dirname(op.abspath(mod.__file__))
after importing rda_python_<docname>.<docname> and returns it as a third
value alongside opts and alias.

In __main__, pg.DOCS['ORIGIN'] is assigned to that path before
process_docs() is called, so parse_docs() looks for <docname>.usg in
the same directory as <docname>.py rather than the hard-coded
$DSSHOME/dssdb/prog_usage default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the hard-coded paths in __init__:
- ORIGIN: was PgLOG.PGLOG['DSSHOME'] + "/dssdb/prog_usage"
           now os.getcwd()
- DCROOT: was PgLOG.get_environment("WEBROOT", ...) + "/rstdocs"
           now os.getcwd()

Both can still be overridden by the caller before process_docs() is
invoked (e.g. ORIGIN is overridden by _load_opts_alias in __main__).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use op.join(op.dirname(op.abspath(__file__)), "rst_templates") so the
rst_templates/ directory is always resolved relative to pg_rst.py
regardless of the working directory at runtime.

Remove the now-redundant op.abspath(TMPDIR) re-resolution that was
previously needed to survive the change_local_directory() call in
process_docs().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old: {DSSURL}/internal/docs/{opt}
New: https://gdex-docs-{opt}.readthedocs.io/en/latest/index.html

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When --dcroot DIR is supplied, pg.DOCS['DCROOT'] is set to DIR before
process_docs() is called; output is written to <DIR>/<docname>/.
When omitted, DCROOT remains the default (current working directory).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DCROOT is removed; DOCDIR now serves as both the configurable root and
the final output directory:

- __init__: drop 'DCROOT' key and post-dict assignment; initialize
  'DOCDIR' directly to os.getcwd()
- process_docs: DOCDIR = DOCDIR/docname  (appends docname in-place)
- __main__: pg.DOCS['DOCDIR'] = args.docdir

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the DOCDIR = DOCDIR/docname assignment in process_docs so that
RST files are written directly into DOCDIR rather than a docname
subdirectory beneath it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces reStructuredText (RST) generation support for .usg help documents by adding RST templates and a new PgRST implementation, and wires a new pgrst CLI entry point.

Changes:

  • Add RST template files for index, toc, and per-section pages.
  • Add PgRST implementation to parse .usg docs and render .rst output.
  • Bump package version and add a new pgrst console script.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
src/rda_python_miscs/rst_templates/toc.rst.temp New TOC template for generated RST docs.
src/rda_python_miscs/rst_templates/section.rst.temp New per-section RST template.
src/rda_python_miscs/rst_templates/index.rst.temp New index page template for generated RST docs.
src/rda_python_miscs/pg_rst.py Adds the RST generator implementation and CLI parsing block.
src/rda_python_miscs/pg_rst.py.bck Adds a backup/alternate RST generator implementation copy.
src/rda_python_miscs/pg_docs.py Adds the HTML generator module (legacy-style) into this package.
pyproject.toml Version bump and adds the pgrst console script entry.
Comments suppressed due to low confidence (1)

src/rda_python_miscs/pg_rst.py:1236

  • pg_rst.py is used as a console script (pgrst), but the module does not define a main() function—only an if __name__ == '__main__': block. Add a main(argv=None) function that performs the current argument parsing + execution, and call it from the __main__ guard so both python -m ... and the entry point work.
if __name__ == '__main__':
   parser = argparse.ArgumentParser(
      description=(
         "Generate RST documentation from a structured .usg source document.\n\n"
         "OPTS and ALIAS are loaded from rda_python_<docname>/<docname>.py: "
         "the module is searched first for module-level OPTS/ALIAS variables, "
         "then for a class defined in that module that carries both as class "
         "attributes."
      ),
      formatter_class=argparse.RawDescriptionHelpFormatter,
   )
   parser.add_argument(
      'docname',
      help=(
         "Short document name, e.g. 'dsarch' or 'dsupdt'.  "
         "The module rda_python_<docname>/<docname>.py must be importable "
         "and must define OPTS (and optionally ALIAS) either at module "
         "level or as class attributes."
      ),
   )
   parser.add_argument(
      '--docdir',
      default=None,
      metavar='DIR',
      help=(
         "Root directory under which the per-document RST output directory "
         "is created (default: current working directory).  "
         "The final output lands in <docdir>/<docname>/."
      ),
   )
   args = parser.parse_args()

   opts, alias, origin = _load_opts_alias(args.docname)
   pg = PgRST()
   pg.DOCS['ORIGIN'] = origin
   if args.docdir is not None:
      pg.DOCS['DOCDIR'] = args.docdir
   pg.process_docs(args.docname, opts, alias)


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1233 to +1234
if args.docdir is not None:
pg.DOCS['DOCDIR'] = args.docdir

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --docdir help says output lands in <docdir>/<docname>/, but the implementation assigns pg.DOCS['DOCDIR'] = args.docdir and process_docs() writes directly into DOCDIR without creating a per-docname subdirectory. This is either misleading CLI documentation or a behavior bug that can cause different docs to overwrite each other. Either update the help text or change the code to write into os.path.join(docdir, docname) (and ensure the directory exists).

Suggested change
if args.docdir is not None:
pg.DOCS['DOCDIR'] = args.docdir
# Determine the root directory for documentation output. If --docdir is
# not provided, default to the current working directory. The final
# output directory is always <docdir>/<docname>/ as documented.
base_docdir = args.docdir if args.docdir is not None else os.getcwd()
doc_output_dir = op.join(base_docdir, args.docname)
os.makedirs(doc_output_dir, exist_ok=True)
pg.DOCS['DOCDIR'] = doc_output_dir

Copilot uses AI. Check for mistakes.
.. _section__SECID__:

__SECID__ - __TITLE__
=====================

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

section.rst.temp uses a fixed-length underline (=====================) for the section heading. In reStructuredText, the underline must be at least as long as the title text, so this will break formatting for most real section titles. Consider adding a placeholder like __ULINE__ computed in code based on the rendered title length, or switch to a directive style that doesn’t require exact underline length.

Suggested change
=====================
__ULINE__

Copilot uses AI. Check for mistakes.
Comment on lines +14 to +16
====================
__TITLE__ Guide
====================

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toc.rst.temp uses a fixed-length underline for the __TITLE__ Guide heading. For longer document titles (e.g., PUBLISH_FILELIST Guide), the underline will be shorter than the rendered title, which is invalid RST and will render incorrectly / emit warnings. Use a computed underline (via a placeholder filled in by code) or choose a heading style that avoids fixed underline sizing.

Suggested change
====================
__TITLE__ Guide
====================
.. rubric:: __TITLE__ Guide

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +30
#
###############################################################################
#
# Title : pg_rst.py
# Author : Zaihua Ji, zji@ucar.edu
# Date : 03/17/2026
# Purpose : python class to convert text help documents into
# reStructuredText (RST) format with help of rst templates
#
# Work File : $DSSHOME/lib/python/pg_rst.py
# Github : https://github.com/NCAR/rda-shared-libraries.git
#
###############################################################################
import os
import re
from os import path as op
from rda_python_common.pg_file import PgFile
from rda_python_common.pg_util import PgUtil

class PgRST(PgFile, PgUtil):
def __init__(self):
super().__init__()
self.OPTS = {}
self.ALIAS = {}
self.sections = []
self.options = {}
self.examples = []
cwd = os.getcwd()
self.DOCS = {
'ORIGIN': cwd,

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .bck backup file is checked into src/ and will be shipped in source distributions (and potentially wheels, depending on packaging settings). Since it appears to be an old/alternate implementation and is not importable as a module, it adds noise and can confuse maintainers. Please remove it from the package (or move it under docs/ or exclude it via packaging configuration).

Copilot uses AI. Check for mistakes.
for i in range(acnt):
content += "<tr>"
if i == 0:
content += " (Aliass: "

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User-visible label text has a typo: Aliass should be Alias / Aliases. This will show up in generated HTML output.

Suggested change
content += " (Aliass: "
content += " (Alias{}: ".format(s)

Copilot uses AI. Check for mistakes.
Comment thread pyproject.toml
"rdacp.py" = "rda_python_miscs.rdacp:main"
"rdakill.py" = "rda_python_miscs.rdakill:main"
"rdamod.py" = "rda_python_miscs.rdamod:main"
"pgrst" = "rda_python_miscs.pg_rst:main"

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added console script entry points to rda_python_miscs.pg_rst:main, but pg_rst.py does not define a main() function (it only has an if __name__ == '__main__': block). This will make pgrst fail at runtime when installed. Expose a main() callable (and have the __main__ block call it), or update the entry point to a function that exists.

Suggested change
"pgrst" = "rda_python_miscs.pg_rst:main"

Copilot uses AI. Check for mistakes.
Comment thread src/rda_python_miscs/pg_rst.py Outdated
elif ptype == 2:
opts = re.findall(r'(-\(*)([a-zA-Z]{2,})(\W|$)', line)
ms = re.match(r'^\s*%s(\s+[\w\.]+\s+|\s+)([a-zA-Z]{2})(\s)' % self.DOCS['DOCNAM'], line)
if ms: opts = opts.insert(0, ms.groups())

Copilot AI Mar 24, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opts.insert(0, ...) returns None, so when ms matches this line sets opts to None and breaks option-link replacement (and can raise later when iterating). Prepend the match groups without using the return value (e.g., opts = [ms.groups()] + opts).

Suggested change
if ms: opts = opts.insert(0, ms.groups())
if ms: opts = [ms.groups()] + opts

Copilot uses AI. Check for mistakes.
Comment thread src/rda_python_miscs/pg_docs.py Outdated
Comment thread src/rda_python_miscs/pg_docs.py Outdated
zaihuaji and others added 3 commits March 24, 2026 14:11
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@zaihuaji zaihuaji merged commit be2d17d into main Mar 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants