Skip to content

nushell-prophet/numd

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,103 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CI

numd - reproducible Nushell Markdown documents

Execute blocks of nushell code within markdown documents, write results back to your .md document, or output them to the terminal.

Quickstart

# clone the repo and `cd` into it
git clone https://github.com/nushell-prophet/numd
cd numd

# use definitions from the module
use numd

# run it on any file to check (--echo outputs to stdout without saving)
numd run z_examples/1_simple_markdown/simple_markdown.md --echo

How it works

numd run parses the initial file (example), generates a script to execute the found commands (example), executes this script in a new nushell instance, captures the results, updates the initial document accordingly, and/or outputs the resulting document into the terminal along with basic changes stats.

Experienced nushell users can understand the logic better by looking at examples. Especially, seeing numd in action describing its own commands.

Details on parsing code blocks and displaying the output

  1. numd looks for code blocks marked with ```nushell or ```nu.
  2. Code blocks are split into command groups by blank lines (double newlines). Each command group is executed separately.
  3. Output from each command group is displayed inline with # => prefix immediately after the command.
  4. Multiline commands (pipelines split across lines without blank lines) are treated as a single command group.
  5. Plain # comments are preserved; # => output lines are regenerated on each run.
  6. Use the separate-block fence option to output results in a separate code block instead of inline.

Note

This readme is a live numd document

Generate-regions

Besides code blocks, numd can maintain stretches of plain markdown. Write a one-line HTML comment with a Nushell command:

<!-- numd-gen: use numd; numd doc 'numd run' -->

On the next numd run the marker expands into a start/end pair, with the command's stdout spliced between them as raw markdown (no fences, no # =>). Every following run replaces the content with fresh output, so git diff shows the drift:

<!-- numd-gen-start: use numd; numd doc 'numd run' -->
…generated markdown…
<!-- numd-gen-end -->
  • The command runs inside the same intermediate script as ordinary code blocks, so it sees --eval config and everything defined by earlier blocks (use numd; in the marker is needed only when nothing above loaded the module).
  • Fenced code blocks inside a region are content, never executed.
  • numd clear-outputs empties region content by default (the markers stay, so the next run refills them); --keep-generated keeps it.

The command reference sections in this README — the numd run section right below, numd clear-outputs, and the rest — are such regions built on numd doc: numd doc 'numd run' renders docs for one command, numd doc numd for a whole module (--header-level sets the header depth, default 3).

numd run

Run Nushell code blocks in a markdown file, output results back to the .md, and optionally to terminal

numd run <file>    # `nothing -> string`, `nothing -> nothing`, `nothing -> record`, `nothing -> table<block_index: int, infostring: string, code: string>`

Parameters:

  • file: path — path to a .md file containing Nushell code to be executed

Flags:

  • --dry-run — return blocks that would execute (block_index, infostring, code) without executing anything
  • --echo — output resulting markdown to stdout instead of saving to file
  • --eval: string — Nushell code to prepend to the script (use open -r config.nu for file-based config)
  • --ignore-git-check — skip the check for uncommitted changes before overwriting
  • --no-fail-on-error — skip errors (markdown is never saved on error)
  • --no-stats — do not output stats of changes (is activated via --echo by default)
  • --print-block-results — print blocks one by one as they are executed, useful for long running scripts
  • --save-intermed-script: path — optional path for keeping intermediate script (useful for debugging purposes). If not set, the temporary intermediate script will be deleted.
  • --use-host-config — load host's env, config, and plugin files (default: run with nu -n for reproducibility)

Examples:

update readme

numd run README.md

preview which blocks would execute, without running them

numd run --dry-run README.md

Supported fence options

numd understands the following fence options. Several comma-separated fence options can be combined together. Fence options are placed in the infostring of the opening code fence, for example: ```nushell try, new-instance

numd list-fence-options
# => ╭───┬────────────────┬──────────────────────────────────────────────────────────────────╮
# => │ # │      long      │                           description                            │
# => ├───┼────────────────┼──────────────────────────────────────────────────────────────────┤
# => │ 0 │ no-output      │ execute code without outputting results                          │
# => │ 1 │ no-run         │ do not execute code in block                                     │
# => │ 2 │ try            │ execute block inside `try {}` for error handling                 │
# => │ 3 │ new-instance   │ execute block in new Nushell instance (useful with `try` block)  │
# => │ 4 │ separate-block │ output results in a separate code block instead of inline `# =>` │
# => │ 5 │ run-once       │ execute code block once, then set to no-run                      │
# => ╰───┴────────────────┴──────────────────────────────────────────────────────────────────╯

Stats of changes

By default, numd provides basic stats on changes made (when not using --echo).

# Running without --echo saves the file and returns stats
let path = [z_examples 1_simple_markdown simple_markdown_with_no_output.md] | path join
numd run $path --ignore-git-check
# => ╭──────────────────┬───────────────────────────────────╮
# => │ filename         │ simple_markdown_with_no_output.md │
# => │ nushell_blocks   │ 3                                 │
# => │ levenshtein_dist │ 4                                 │
# => │ diff_lines       │ 0%                                │
# => │ diff_words       │ 0%                                │
# => │ diff_chars       │ +1 (0.2%)                         │
# => ╰──────────────────┴───────────────────────────────────╯

Styling outputs

Use the --eval option to prepend Nushell code to the intermediate script. This lets you set visual settings and other configuration before your code runs.

let path = $nu.temp-dir | path join simple_nu_table.md

# let's generate some markdown and save it to the `simple_nu_table.md` file in the temp directory
"```nushell\n[[a b c]; [1 2 3]]\n```\n" | save -f $path

# let's run this file to see its outputs (--echo outputs to stdout without saving)
numd run $path --echo --no-stats --eval "
    $env.config.footer_mode = 'never'
    $env.config.table.header_on_separator = false
    $env.config.table.index_mode = 'never'
    $env.config.table.mode = 'basic_compact'
"
# => ```nushell
# => [[a b c]; [1 2 3]]
# => # => +---+---+---+
# => # => | a | b | c |
# => # => | 1 | 2 | 3 |
# => # => +---+---+---+
# => ```

numd clear-outputs

Remove numd execution outputs from the file Note: No git check here - clearing outputs is a reversible operation (just re-run numd) and users typically clear outputs intentionally before committing clean source

numd clear-outputs <file>    # `nothing -> string`, `nothing -> nothing`

Parameters:

  • file: path — path to a .md file containing numd output to be cleared

Flags:

  • --echo — output resulting markdown to stdout instead of writing to file
  • --strip-markdown — keep only Nushell script, strip all markdown tags
  • --keep-outputs — keep inline # => output lines
  • --keep-generated — keep generate-region content

numd capture

numd can use the display_output hook to write the current session prompts together with their output into a specified markdown file. There are corresponding commands numd capture start and numd capture stop.

numd capture start

start capturing commands and their outputs into a file

numd capture start (file)    # `nothing -> nothing`

Parameters:

  • file?: path

Flags:

  • --separate-blocks — create separate code blocks for each pipeline instead of inline # => output

numd capture stop

stop capturing commands and their outputs

numd capture stop    # `nothing -> nothing`

numd parse-md

Parse markdown into a table of semantic blocks (headers, paragraphs, code blocks, lists, blockquotes, frontmatter) with extracted content and metadata

numd parse-md (file)    # `string -> table`, `nothing -> table`

Parameters:

  • file?: path — optional path to markdown file (can also pipe content)

Some random familiar examples

ls z_examples | sort-by name | reject modified size
# => ╭────┬─────────────────────────────────────────┬──────╮
# => │  # │                  name                   │ type │
# => ├────┼─────────────────────────────────────────┼──────┤
# => │  0 │ z_examples/1_simple_markdown            │ dir  │
# => │  1 │ z_examples/2_numd_commands_explanations │ dir  │
# => │  2 │ z_examples/4_book_working_with_lists    │ dir  │
# => │  3 │ z_examples/5_simple_nu_table            │ dir  │
# => │  4 │ z_examples/6_edge_cases                 │ dir  │
# => │  5 │ z_examples/7_image_output               │ dir  │
# => │  6 │ z_examples/8_parse_frontmatter          │ dir  │
# => │  7 │ z_examples/999_numd_internals           │ dir  │
# => │  8 │ z_examples/99_strip_markdown            │ dir  │
# => │  9 │ z_examples/9_other                      │ dir  │
# => │ 10 │ z_examples/numd_config_example1.nu      │ file │
# => │ 11 │ z_examples/numd_config_example2.nu      │ file │
# => ╰────┴─────────────────────────────────────────┴──────╯

'hello world' | str length
# => 11

2 + 2
# => 4

git tag | lines | sort -n | last
# => 0.4.0

Real fight examples to try

# output the result of execution to terminal without updating the file (--echo implies no save)
[z_examples 1_simple_markdown simple_markdown.md]
| path join
| numd run $in --echo

Development and testing

Nushell Markdown documents used together with Git could often serve as a convenient way to test custom and built-in Nushell commands.

Testing of the numd module is done via toolkit.nu:

# Run all tests (unit + integration)
nu toolkit.nu test

# Run only unit tests (uses nutest framework)
nu toolkit.nu test-unit

# Run only integration tests (executes example markdown files)
nu toolkit.nu test-integration

Unit tests

Unit tests in tests/ use the nutest framework to test internal functions like parse-markdown-to-blocks, classify-block-action, extract-fence-options, etc.

Integration tests

Integration tests run all example files in z_examples/ through numd and report changes via Levenshtein distance. Whatever changes are made in the module - it can be easily seen if they break anything (both by the Levenshtein distance metric or by git diff of the updated example files versus their initial versions).

nu toolkit.nu test-integration
# => ╭───────────────────────────────────────────────┬─────────────────┬───────────────────┬────────────┬──────────────┬─────╮
# => │                   filename                    │ nushell_blocks  │ levenshtein_dist  │ diff_lines │  diff_words  │ ... │
# => ├───────────────────────────────────────────────┼─────────────────┼───────────────────┼────────────┼──────────────┼─────┤
# => │ types_of_data.md                              │              30 │               204 │ 0%         │ -29 (-1.1%)  │ ... │
# => │ working_with_lists.md                         │              20 │                 4 │ 0%         │ 0%           │ ... │
# => │ numd_commands_explanations.md                 │               6 │                 0 │ 0%         │ 0%           │ ... │
# => │ simple_markdown.md                            │               3 │                 0 │ 0%         │ 0%           │ ... │
# => │ error-with-try.md                             │               1 │                13 │ -1 (-4.3%) │ 0%           │ ... │
# => │ simple_markdown_first_block.md                │               3 │                 0 │ 0%         │ 0%           │ ... │
# => │ raw_strings_test.md                           │               2 │                 0 │ 0%         │ 0%           │ ... │
# => │ simple_nu_table.md                            │               3 │                 0 │ 0%         │ 0%           │ ... │
# => │ simple_nu_table_customized_width20.md         │               3 │               458 │ 0%         │ -42 (-23.7%) │ ... │
# => │ simple_nu_table_customized_example_config.md  │               3 │                56 │ 0%         │ -4 (-2.3%)   │ ... │
# => │ README.md                                     │               9 │                 0 │ 0%         │ 0%           │ ... │
# => ╰───────────────────────────────────────────────┴─────────────────┴───────────────────┴────────────┴──────────────┴─────╯

About

numd - reproducible Nushell Markdown documents

Topics

Resources

License

Stars

49 stars

Watchers

3 watching

Forks

Contributors