Skip to content

docs(cli): document all 106 CLI commands - #19799

Open
rangareddy wants to merge 1 commit into
apache:asf-sitefrom
rangareddy:docs-15376-cli-commands
Open

docs(cli): document all 106 CLI commands#19799
rangareddy wants to merge 1 commit into
apache:asf-sitefrom
rangareddy:docs-15376-cli-commands

Conversation

@rangareddy

@rangareddy rangareddy commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Describe the issue this Pull Request addresses

Closes #15376 (JIRA: HUDI-4689, epic
HUDI-1388), "Add documentation for all CLI commands".

hudi-cli exposes 106 commands across 26 command classes. website/docs/cli.md had two gaps:

  1. 20 commands were absent from the page entirely - not documented, and not even listed, so a reader could not
    discover they exist. The whole locks audit family, both timeline commands and both diff commands were among
    them.
  2. 56 more appeared only as a one-line entry in the embedded help output, with their options documented
    nowhere on the page.

Summary and Changelog

One file, website/docs/cli.md.

1. The help listing now carries all 106 commands. Entries were inserted at their alphabetical positions
rather than the block being regenerated, so the diff stays reviewable.

2. A new Command reference section documents every command, grouped into nine areas, with each command's
options and their defaults. It also records which entries are aliases rather than distinct commands: refresh,
metadata refresh, commits refresh, cleans refresh and savepoints refresh are five names for one method that
reloads table metadata, and temp query / temp_query, temp delete / temp_delete and
temps show / temps_show are spelling variants of three commands.

3. New worked sections for the commands that had no documentation at all:

Section Commands
Auditing Storage Locks the five locks audit commands, the layout under .hoodie/.locks, and what validate reports as an error versus a warning
Inspecting the Timeline timeline show active / incomplete and the two metadata timeline equivalents, with their output columns
Diffing a File or Partition diff file, diff partition
Repairing a Table rename partition, repair deprecated partition, repair show empty commit metadata
Changing the Meta Fields Mode table set-meta-fields-mode and its two guards

Four more are documented in the sections they belong to: trigger archival under Archived Commits,
show restore / show restores under Savepoint and Restore, kerberos kdestroy under the Kerberos section, and
metadata delete-record-index under Record Index Lookup.

Every command name, option, default and behaviour was read from the @ShellMethod and @ShellOption declarations
on master, not from the help strings, which have drifted.

Things the source read turned up that the help text does not say

  • diff file and diff partition ignore --startTs and --endTs unless --includeArchivedTimeline is also
    true.
    CLIUtils#getTimelineInRange (:46-62) applies the bounds only on the archived branch and returns the
    whole active timeline otherwise, so a narrow range silently does nothing. The help text advertises the defaults
    with no hint of the precondition.
  • repair show empty commit metadata returns void and reports through log.warn, so it renders no result
    table; its findings go to the CLI log.
  • rename partition physically removes the old partition directory after its delete_partition write
    (SparkMain#renamePartition), while repair deprecated partition leaves the emptied default directory. The two
    are otherwise the same shape.

Verification

cd website && npm run build
  -> [SUCCESS] Generated static files in "build"

docs markdown link-format checks (the ones asf-site.ci.yml runs on website/docs/**)
  -> no absolute /docs/ paths, no links missing .md, no ../ or ./ prefixes

coverage check, parsing @ShellMethod keys out of hudi-cli on master and grepping cli.md
  -> 106 source commands, 0 missing, 0 documented only in the help listing

The extraction behind the command reference was validated against the source as well: the 18 commands recorded as
taking no options were each confirmed to declare none, and the captured option count reconciles exactly with the
404 @ShellOption occurrences in the tree once the two alias methods that double-count are accounted for.

Sections and the reference were confirmed in the rendered build/docs/next/cli/index.html, including that the
escaped angle brackets in the rename partition usage string render as literal < and >.

Impact

Documentation only; no code or config changes. The page is the current (next) docs version, which tracks
master, where these commands live. Versioned copies under website/versioned_docs/ are deliberately untouched,
since several of these commands do not exist in those releases.

Risk Level

none

Documentation Update

This is the documentation update.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable
  • CI passes on my PR

@github-actions github-actions Bot added docs size:M PR with lines of changes in (100, 300] labels Aug 31, 2026
@rangareddy
rangareddy force-pushed the docs-15376-cli-commands branch from a11a0e2 to 70a311d Compare August 31, 2026 04:31
@rangareddy rangareddy changed the title docs(cli): document the 20 undocumented CLI commands and refresh the help listing docs(cli): document the undocumented CLI commands and how to set Hudi properties Aug 31, 2026
@voonhous

Copy link
Copy Markdown
Member

Is this a 2 part PR or? The description says part of. If this is a 2 part PR, please make it clear in the title this is Part 1 of N, where N should be clearly defined.

@github-actions github-actions Bot added size:L PR with lines of changes in (300, 1000] and removed size:M PR with lines of changes in (100, 300] labels Aug 31, 2026
@rangareddy
rangareddy marked this pull request as draft August 31, 2026 05:03
@rangareddy rangareddy changed the title docs(cli): document the undocumented CLI commands and how to set Hudi properties docs(cli): document all 106 CLI commands and how to set Hudi properties Aug 31, 2026
Closes apache#15376 (HUDI-4689).

hudi-cli exposes 106 commands across 26 command classes. cli.md covered 86 of them, and of
those only 30 had their options documented anywhere; the other 56 existed on the page as a
single line in the embedded help output. Twenty were absent entirely, including the whole
locks audit family, both timeline commands and both diff commands.

Every command name, option, default and behaviour below was read from the @ShellMethod and
@ShellOption declarations on master, not from the help strings, which have drifted.

The help listing now carries all 106 commands, inserted at their alphabetical positions
rather than regenerated, so the diff stays reviewable.

A new "Command reference" section documents every command, grouped into nine areas, with
each command's options and defaults. It also records which entries are aliases rather than
distinct commands: refresh, metadata refresh, commits refresh, cleans refresh and
savepoints refresh are five names for one method, and temp query / temp_query,
temp delete / temp_delete and temps show / temps_show are spelling variants.

New worked sections for what had no documentation at all:

  Auditing Storage Locks          locks audit enable / disable / status / validate /
                                  cleanup, the layout under .hoodie/.locks, and what
                                  validate reports as an error versus a warning
  Inspecting the Timeline         timeline show active / incomplete and the two metadata
                                  timeline equivalents, with the output columns
  Diffing a File or Partition     diff file, diff partition
  Repairing a Table               rename partition, repair deprecated partition,
                                  repair show empty commit metadata
  Changing the Meta Fields Mode   table set-meta-fields-mode and its two guards

Four more are documented in the sections they belong to: trigger archival under Archived
Commits, show restore / show restores under Savepoint and Restore, kerberos kdestroy under
the Kerberos section, and metadata delete-record-index under Record Index Lookup.

Three things the source read turned up that the help text does not tell you:

  - diff file and diff partition ignore --startTs and --endTs unless
    --includeArchivedTimeline is also true. CLIUtils#getTimelineInRange applies the bounds
    only on the archived branch and returns the whole active timeline otherwise, so a narrow
    range silently does nothing.
  - repair show empty commit metadata returns void and reports through log.warn, so it
    renders no result table.
  - rename partition physically removes the old partition directory after its
    delete_partition write; repair deprecated partition leaves the emptied directory.

Angle brackets in generated text are escaped: the rename partition usage string contains
<oldPartition>, which MDX otherwise parses as a JSX tag and fails the build on.
@rangareddy
rangareddy force-pushed the docs-15376-cli-commands branch from 4e1cc4c to 2dd66c7 Compare August 31, 2026 06:09
@rangareddy rangareddy changed the title docs(cli): document all 106 CLI commands and how to set Hudi properties docs(cli): document all 106 CLI commands Aug 31, 2026
@rangareddy
rangareddy marked this pull request as ready for review August 31, 2026 06:21
@rangareddy

Copy link
Copy Markdown
Collaborator Author

Hi @voonhous I have update the PR description and documentation changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs size:L PR with lines of changes in (300, 1000]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants