From c3b3c14fc449e10b6cf8509230aad18c227487ce Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 16 Nov 2026 19:40:24 +0000 Subject: [PATCH 1/3] WH013 reads the parsed workflow model; delete WorkflowYaml (#971) WH013 now reads Hypatia.Workflow, as RE001, RE002 and WH014 (#1182) already do. With that, nothing reads Hypatia.WorkflowYaml, the line reader the hardening rules shared, and it is deleted. WH013 reads a grant as a `permissions:` value (so `permissions: {contents: write}` counts, and a script's `contents: write` text does not). It reads writes and masks from string values, a credential from token/GH_TOKEN/GITHUB_TOKEN keys, and a step's own continue-on-error, in any layout. It judges flow-style jobs per job. A writing workflow the reader refuses is reported at :warn at the error line, as RE001 and WH014 do, and is skipped otherwise. Seven new WH013 tests fail on main; an eighth guards a refused workflow that names no write. Six WH014 tests carried over from the earlier rebuild pass on #1182's WH014. RE003, RE005 and WorkflowAudit's toolchain-input check still read lines. ResearchExtensions.flow_to_block/1 moves back into that module unchanged, so their findings and the fixer's line numbers do not move. Checked under Elixir 1.19.4 / OTP 27.3.4.19: - mix format --check-formatted passes, and the changed modules compile with no warnings. - Comparing main (6f3f16f) with this change, every workflow rule gives the same findings on six repository snapshots and on perturbed copies, including the copies that make WH013 fire (12 findings). - 34 related test files pass. Only workflow_hardening_test.exs differs from main: 118 tests instead of 104, all passing. CLAUDE.md's lib/ count is re-measured at 171 (172 tracked on main, less this file). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01J4QvDithtn5Xk5MbSPm4Px --- .claude/CLAUDE.md | 2 +- CHANGELOG.adoc | 43 +++ lib/hypatia/workflow_yaml.ex | 637 ------------------------------- lib/rules/research_extensions.ex | 109 +++++- lib/rules/workflow_hardening.ex | 256 +++++++++---- test/workflow_hardening_test.exs | 328 ++++++++++++++++ 6 files changed, 646 insertions(+), 729 deletions(-) delete mode 100644 lib/hypatia/workflow_yaml.ex diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 544f9285..d86fca57 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -13,7 +13,7 @@ Hypatia is the neurosymbolic CI/CD intelligence layer for the hyperpolymath ecos ## Architecture ``` -Hypatia # 170 Elixir source files under lib/ (recursive tree count, 2026-11-14) +Hypatia # 171 Elixir source files under lib/ (recursive tree count, 2026-11-16) ├── Elixir pipeline # core: pattern analysis, dispatch, learning ├── Merge orchestration # 12 modules — leases, ledger, ticker (largest subsystem) ├── Kin subsystem # 6 modules — contingency, arbiter, gate, watchdog diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index 3e87e50c..1672312a 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -12,6 +12,49 @@ https://semver.org/[Semantic Versioning]. === Changed +==== WH013 reads the canonical workflow model; `Hypatia.WorkflowYaml` is gone (2026-11-16) + +WH013 now reads `Hypatia.Workflow`, as RE001, RE002 and WH014 do. With that, +nothing reads `Hypatia.WorkflowYaml`, the line reader the hardening rules +shared, and it is deleted. Each change below has a test in +`test/workflow_hardening_test.exs` ("WH013 reads the parsed workflow model +(#971)") that fails on the line reader: + +* **A grant is a `permissions:` value.** `permissions: {contents: write}`, + the form WH013's own fix text recommends, now counts as a grant. Text that + only looks like one, such as a `run:` script that writes `contents: write` + into a file, no longer does. +* **Writes, masks and credentials are read from values and keys, not + lines.** A YAML comment naming `git push origin main` is not a write. A + `GH_TOKEN:` line that a script writes into a file is not the job's + credential. A workflow-level `env:` written after `jobs:` is inherited like + one written before it. +* **Flow-style jobs are judged per job.** The line reader could not split + `jobs: { … }`, so it judged the whole file as one body: never credentialed, + and never masked by a step's `continue-on-error`. +* **An unreadable workflow is reported, not guessed at.** As RE001 and WH014 + do, a workflow the reader refuses gets a `:warn` finding at the error's + line if its text names a write. Otherwise it is skipped. + +The tests also gain six WH014 cases (KYAML, flow `with:`, folded and quoted +run scripts, `defaults` placement, expression working directories and +unreadable files). They pass on the WH014 above. + +RE003, RE005 and `WorkflowAudit.check_sha_pinned_toolchain_without_input/1` +still read lines. Their KYAML view, `ResearchExtensions.flow_to_block/1`, +moves back into that module unchanged, so their findings, and the line +numbers the toolchain fixer rewrites by, do not move. Moving them onto the +model stays under #971. + +Measured on 2026-11-16 with Elixir 1.19.4 on OTP 27.3.4.19, comparing main +(6f3f16f) with this change on snapshots of hyperpolymath/hypatia, +rsr-template-repo, choreographic-types and mathesis, and +metadatastician/standards and kyq. Every workflow rule (WorkflowAudit, +WorkflowHardening and ResearchExtensions) gives the same findings before and +after. WH013 does not fire on these trees, so the comparison was repeated on +perturbed copies that make it fire (every `contents: write` narrowed to +`read`, 12 WH013 findings), again with the same findings. + ==== WH014 reads the canonical workflow model (2026-11-16) The masked-report/SARIF-upload rule reads jobs, steps, inputs and working diff --git a/lib/hypatia/workflow_yaml.ex b/lib/hypatia/workflow_yaml.ex deleted file mode 100644 index 3d588847..00000000 --- a/lib/hypatia/workflow_yaml.ex +++ /dev/null @@ -1,637 +0,0 @@ -# SPDX-License-Identifier: MPL-2.0 -# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) -defmodule Hypatia.WorkflowYaml do - @moduledoc """ - The workflow reader shared by the rules that look inside GitHub Actions - workflows: it reads a workflow the way YAML (and GitHub) does, as far as the - line- and indent-based rules need, while every line keeps its physical line - number so a finding points at the source. - - It was RE001's reader (`Hypatia.Rules.ResearchExtensions`) and moved here - unchanged so other rules (WH013 first) read workflows the same way instead - of each carrying its own approximation. - - * `read_workflow/1` and `flow_to_block/1` read a KYAML (flow-style) - workflow as the equivalent block layout, one line for one line. - * `active_lines/1` numbers the lines, drops comment lines and expands - YAML anchors and aliases (`expand_aliases/1`). - * `workflow_job_lines/1` splits the lines under `jobs:` into jobs, - `job_steps/1` a job into its steps, and `step_key_column/1` gives the - column of a step's own keys. - * `key_line?/2`, `direct_job_field?/2`, `job_field_indent/1`, - `strip_yaml_comment/2`, `nesting_at/2` and `indent_of/1` are the - primitives those are built from. - - It is a static text reader, not a YAML parser: it covers the shapes GitHub - workflows use, and it stays bounded on hostile input (alias expansion stops - after a fixed number of copied lines). - """ - - @doc """ - The workflow's lines as `{line, line_number}`, without comment lines, and - with YAML aliases expanded in place (each expanded line carries the alias's - line number). Physical line numbers are kept, so a finding never points at - a line that a header comment shifted. - """ - def active_lines(content) when is_binary(content) do - content - |> String.split("\n") - |> Enum.with_index(1) - |> Enum.reject(fn {line, _} -> String.starts_with?(String.trim_leading(line), "#") end) - |> expand_aliases() - end - - # YAML anchors and aliases, which GitHub expands in workflows: `*name` - # stands for the node anchored `&name`, so `env: *common`, `run: *script`, - # `<<: *defaults` or `deploy: *build` brings the anchored node into the job - # that holds the alias. Each alias is expanded in place before any job is - # read, and every line it brings carries the alias's line number, so a - # finding points at the alias: - # - # * an alias that is a key's whole value, or a list item, is replaced by - # what follows its anchor on the anchor's line (a scalar, a block - # scalar's `|`, a flow collection), and the anchor's indented lines - # follow at the column of the `*`; - # * a `<<:` merge puts the anchored mapping's entries into the mapping - # that holds the key, at the key's column, minus every entry whose key - # that mapping sets itself: YAML gives the explicit key precedence; - # * an alias inside a flow collection is followed by a copy of the - # anchored lines, deeper than the line. - # - # Anchors and aliases are recognised only where a node starts, outside - # quoted scalars and block-scalar text, so `echo "x *common"` and a `run: |` - # line `ls *common` stay text. The anchor is the nearest one of that name - # above the alias. - @anchor ~r/(?:^\s*(?:-\s+)*|:\s+|[\[{,]\s*)&([A-Za-z0-9_-]+)(?=\s|$)/ - @alias ~r/(?:^\s*(?:-\s+)*|:\s+|[\[{,]\s*)\*([A-Za-z0-9_-]+)\s*(?=$|[,\]}])/ - @merge_key ~r/^(\s*(?:-\s+)?)(?:<<|"<<"|'<<')\s*:/ - @mapping_key ~r/^\s*(?:-\s+)?(?:"([^"]*)"|'([^']*)'|([^\s"'#:][^:]*?))\s*:(?:\s|$)/ - @block_scalar_start ~r/(?:^\s*|[:-]\s+)(?:&[A-Za-z0-9_-]+\s+)?[>|][0-9+-]*\s*$/ - @block_scalar_header ~r/^[>|][0-9+-]*$/ - # Nested anchors can multiply copies exponentially (the "billion laughs" - # file), so expansion stops after this many copied lines per workflow. - @alias_budget 5_000 - - @doc """ - Expand YAML anchors and aliases in numbered lines (`{line, line_number}`), - as GitHub does, within what jobs read: the top-level `jobs:` and `env:` - sections. There each `*name` is replaced in place by the node anchored - `&name` (the nearest one above, wherever it is defined), a `<<:` merge adds - the anchored entries the mapping does not set itself, and every line an - alias brings carries the alias's line number. An alias in any other - top-level section (`on:`, `permissions:`, ...) is returned unchanged. - Anchors and aliases count only where a YAML node starts, never inside - quoted scalars or block-scalar text. Expansion stops - after #{@alias_budget} copied lines, so hostile input costs bounded work. - """ - def expand_aliases(lines) do - codes = yaml_codes(lines) - - anchors = - for {{{line, n}, code}, index} <- lines |> Enum.zip(codes) |> Enum.with_index(), - code != nil, - [_, {at, len}] <- Regex.scan(@anchor, code, return: :index), - node_start?(code, at) do - {binary_part(code, at, len), n, - anchored(lines, index, strip_yaml_comment(line), at + len)} - end - - # Only what jobs read is expanded in place: the jobs and the top-level - # env:. An anchor kept elsewhere is expanded where a job uses it. - {expanded, _} = - lines - |> Enum.zip(codes) - |> Enum.with_index() - |> Enum.flat_map_reduce({nil, @alias_budget}, fn {{{line, _} = entry, code}, index}, - {section, budget} -> - section = - case Regex.run(~r/^["']?([^\s"':]+)["']?\s*:/, line) do - [_, key] -> key - nil -> section - end - - if section in ["jobs", "env"] do - {out, budget} = expand_line(entry, code, lines, index, anchors, budget) - {out, {section, budget}} - else - {[entry], {section, budget}} - end - end) - - expanded - end - - defp expand_aliases(lines, anchors, budget) do - lines - |> Enum.zip(yaml_codes(lines)) - |> Enum.with_index() - |> Enum.flat_map_reduce(budget, fn {{entry, code}, index}, budget -> - expand_line(entry, code, lines, index, anchors, budget) - end) - end - - # One line with its aliases expanded. A line of block-scalar text (code nil) - # holds no alias. - defp expand_line(entry, nil, _lines, _index, _anchors, budget), do: {[entry], budget} - - defp expand_line(entry, _code, _lines, _index, _anchors, budget) when budget <= 0, - do: {[entry], budget} - - defp expand_line({line, n} = entry, code, lines, index, anchors, budget) do - matches = - for [_, {at, len}] <- Regex.scan(@alias, code, return: :index), - node_start?(code, at), - name = binary_part(code, at, len), - {_, _, node} <- - anchors |> Enum.filter(fn {a, m, _} -> a == name and m < n end) |> Enum.take(-1), - do: {at, len, node} - - # The alias that ends the line is the whole value of its key or item. - {value, in_flow} = - case List.last(matches) do - {at, len, _} = last -> - if String.trim(binary_part(code, at + len, byte_size(code) - at - len)) == "", - do: {last, Enum.drop(matches, -1)}, - else: {nil, matches} - - nil -> - {nil, []} - end - - {flow_copies, budget} = - Enum.flat_map_reduce(in_flow, budget, fn {at, _, {inline, block}}, budget -> - indent = indent_of(code) - inline_lines = if inline == "", do: [], else: [{indent_spaces(indent + 2) <> inline, n}] - {copy, budget} = anchored_block(inline, inline_lines ++ block, anchors, budget) - {renumber(reindent(copy, column(code, at - 1)), n), budget} - end) - - case value do - nil -> - {[entry | flow_copies], budget} - - {at, _, {inline, block}} -> - text = strip_yaml_comment(line) - head = String.trim_trailing(binary_part(text, 0, at - 1) <> inline) - {block, budget} = anchored_block(inline, block, anchors, budget - 1) - - block = - case Regex.run(@merge_key, code) do - [_, prefix] -> - merge_column = String.length(prefix) - - block - |> reindent(merge_column) - |> drop_entries(merge_column, sibling_keys(lines, index, merge_column)) - - nil -> - reindent(block, column(code, at - 1)) - end - - # The inline part may hold aliases of its own (`&a [*b, *c]`). - {head_lines, budget} = - if inline == "" or Regex.match?(@block_scalar_header, inline), - do: {[{head, n}], budget}, - else: expand_line({head, n}, yaml_code(head), [{head, n}], 0, anchors, budget) - - {head_lines ++ renumber(block, n) ++ flow_copies, budget} - end - end - - # An anchor's indented lines, their own aliases expanded. Under a block - # scalar's header they are text and are copied as they are. - defp anchored_block(inline, block, anchors, budget) do - if Regex.match?(@block_scalar_header, inline), - do: {block, budget - length(block)}, - else: expand_aliases(block, anchors, budget - length(block)) - end - - # The node an anchor names: what follows the anchor on its own line, and - # every line indented deeper than that line. - defp anchored(lines, index, text, after_name) do - inline = text |> binary_part(after_name, byte_size(text) - after_name) |> String.trim() - indent = indent_of(text) - - block = - lines - |> Enum.drop(index + 1) - |> Enum.take_while(fn {line, _} -> String.trim(line) == "" or indent_of(line) > indent end) - - {inline, block} - end - - # The keys the mapping holding line `index` sets at `column` itself. - defp sibling_keys(lines, index, column) do - {before, [_ | after_line]} = Enum.split(lines, index) - - in_mapping = fn ls -> - Enum.take_while(ls, fn {line, _} -> String.trim(line) == "" or indent_of(line) >= column end) - end - - (in_mapping.(Enum.reverse(before)) ++ in_mapping.(after_line)) - |> Enum.filter(fn {line, _} -> indent_of(line) == column end) - |> Enum.flat_map(fn {line, _} -> mapping_key(line) end) - |> Enum.reject(&(&1 == "<<")) - end - - # Merged entries, less those whose key the destination mapping sets. - defp drop_entries(lines, column, keys) do - lines - |> Enum.chunk_while( - [], - fn {line, _} = entry, acc -> - if indent_of(line) == column and String.trim(line) != "" and acc != [], - do: {:cont, Enum.reverse(acc), [entry]}, - else: {:cont, [entry | acc]} - end, - fn - [] -> {:cont, []} - acc -> {:cont, Enum.reverse(acc), []} - end - ) - |> Enum.reject(fn [{first, _} | _] -> Enum.any?(mapping_key(first), &(&1 in keys)) end) - |> Enum.concat() - end - - defp mapping_key(line) do - case Regex.run(@mapping_key, strip_yaml_comment(line)) do - nil -> [] - captures -> captures |> tl() |> Enum.reject(&(&1 == "")) |> Enum.take(1) - end - end - - # Each line's YAML structure, for finding anchors and aliases: the line - # without its comment and with quoted scalars blanked, byte offsets kept; - # nil for a line of block-scalar text. - defp yaml_codes(lines) do - {codes, _} = - Enum.map_reduce(lines, nil, fn {line, _}, scalar_indent -> - if scalar_indent && (String.trim(line) == "" or indent_of(line) > scalar_indent) do - {nil, scalar_indent} - else - code = yaml_code(line) - {code, if(Regex.match?(@block_scalar_start, code), do: indent_of(line))} - end - end) - - codes - end - - defp yaml_code(line), do: strip_yaml_comment(line, true) - - # A `,` separates nodes only inside a flow collection; in a plain scalar - # (`run: echo a, *b`) it is text. The name starts at `at`, after `*` or `&`. - defp node_start?(code, at) do - before = code |> binary_part(0, at - 1) |> String.trim_trailing() - not String.ends_with?(before, ",") or nesting_at(code, at) > 0 - end - - defp column(text, byte_offset), do: String.length(binary_part(text, 0, byte_offset)) - defp indent_spaces(n), do: String.duplicate(" ", n) - defp renumber(lines, n), do: Enum.map(lines, fn {line, _} -> {line, n} end) - - defp reindent(lines, target) do - base = - lines - |> Enum.map(&elem(&1, 0)) - |> Enum.reject(&(String.trim(&1) == "")) - |> Enum.map(&indent_of/1) - |> Enum.min(fn -> 0 end) - - Enum.map(lines, fn {line, n} -> - if String.trim(line) == "", - do: {"", n}, - else: {indent_spaces(indent_of(line) - base + target) <> String.trim_leading(line), n} - end) - end - - @doc """ - The column of a step's own properties: just past `- ` on its opening line, - or the indent of the first property under a bare `-`. A `-` followed only - by a comment (`- # publish`) is a bare `-`: YAML ignores the comment, so - the step's keys start on the next line. - """ - def step_key_column([{first, _} | rest]) do - case Regex.run(~r/^(\s*-\s+)[^\s#]/, first) do - [_, prefix] -> - String.length(prefix) - - _ -> - Enum.find_value(rest, indent_of(first) + 2, fn {line, _} -> - String.trim(line) != "" && indent_of(line) - end) - end - end - - def step_key_column([]), do: 0 - - @doc """ - The job's steps, each as its own physical lines. - """ - def job_steps(job_lines) do - with field_indent when is_integer(field_indent) <- job_field_indent(job_lines), - step_lines when step_lines != [] <- lines_in_steps(job_lines, field_indent), - step_indent when is_integer(step_indent) <- step_indent(step_lines, field_indent) do - step_lines - |> Enum.chunk_while( - [], - fn {line, _} = entry, acc -> - if indent_of(line) == step_indent and Regex.match?(~r/^\s*-(?:\s|$)/, line) and - acc != [] do - {:cont, Enum.reverse(acc), [entry]} - else - {:cont, [entry | acc]} - end - end, - fn - [] -> {:cont, []} - acc -> {:cont, Enum.reverse(acc), []} - end - ) - else - _ -> [] - end - end - - @doc """ - The flow-collection depth (`[`/`{` opened minus `]`/`}` closed) of `text` - before byte offset `position`; 0 outside any flow collection. - - Every bracket character counts, including one inside a quoted scalar or a - comment, so pass text with those already blanked (as the reader's own - YAML codes are); on raw YAML the depth can be wrong. - """ - def nesting_at(text, position) do - text - |> binary_part(0, position) - |> String.graphemes() - |> Enum.reduce(0, fn - c, depth when c in ["{", "["] -> depth + 1 - c, depth when c in ["}", "]"] -> depth - 1 - _, depth -> depth - end) - end - - @doc """ - True when the job (as `workflow_job_lines/1` gives it, header line first) - sets `field` as one of its own keys, at the job's field column, and not as - a deeper key of a step or a nested mapping. - """ - def direct_job_field?(job_lines, field) do - case job_field_indent(job_lines) do - nil -> - false - - field_indent -> - Enum.any?(job_lines, fn {line, _} -> - indent_of(line) == field_indent and key_line?(line, field) - end) - end - end - - @doc """ - A line that opens the mapping key `key`: YAML reads `env:`, `"env":` and - `'env':` as the same key, and so does GitHub. - """ - def key_line?(line, key) do - key = Regex.escape(key) - Regex.match?(~r/^\s*(?:#{key}|"#{key}"|'#{key}')\s*:(?:\s|$)/, line) - end - - @doc """ - The column of a job's own keys: the shallowest indent among its non-blank - lines after the header. nil for a job with no body. - """ - def job_field_indent([_header | job_lines]) do - job_lines - |> Enum.map(&elem(&1, 0)) - |> Enum.reject(&(String.trim(&1) == "")) - |> Enum.map(&indent_of/1) - |> Enum.min(fn -> nil end) - end - - def job_field_indent([]), do: nil - - @doc """ - The lines of the job's `steps:` value. Its items may sit deeper than the - key or, as an indentless sequence YAML also allows, at the key's own - column: `steps:` then `- run: …` directly beneath it. - """ - def lines_in_steps(job_lines, field_indent) do - case Enum.find_index(job_lines, fn {line, _} -> - indent_of(line) == field_indent and key_line?(line, "steps") - end) do - nil -> - [] - - index -> - job_lines - |> Enum.drop(index + 1) - |> Enum.take_while(fn {line, _} -> - String.trim(line) == "" or indent_of(line) > field_indent or - (indent_of(line) == field_indent and sequence_item?(line)) - end) - end - end - - @doc """ - The column of the step items: the shallowest `-` in the steps value, which - is the key's own column for an indentless sequence. - """ - def step_indent(step_lines, field_indent) do - step_lines - |> Enum.filter(fn {line, _} -> - indent_of(line) >= field_indent and sequence_item?(line) - end) - |> Enum.map(fn {line, _} -> indent_of(line) end) - |> Enum.min(fn -> nil end) - end - - defp sequence_item?(line), do: Regex.match?(~r/^\s*-(?:\s|$)/, line) - - @doc """ - A line without its YAML comment: a `#` at the start or after whitespace, - outside a quoted scalar. A quote opens a scalar only where one can start - (after whitespace or a flow indicator), so the apostrophe in `it's` does not. - With `blank_quoted`, each quoted scalar's text becomes spaces of the same - byte length, so what remains is structure at its original offsets. - """ - def strip_yaml_comment(line, blank_quoted \\ false) do - text = fn c -> if blank_quoted, do: String.duplicate(" ", byte_size(c)), else: c end - - line - |> String.graphemes() - |> Enum.reduce_while({[], nil, " ", false}, fn c, {acc, quote, prev, escaped} -> - cond do - quote == "\"" and escaped -> - {:cont, {[text.(c) | acc], quote, c, false}} - - quote == "\"" and c == "\\" -> - {:cont, {[text.(c) | acc], quote, c, true}} - - quote != nil and c == quote -> - {:cont, {[c | acc], nil, c, false}} - - quote != nil -> - {:cont, {[text.(c) | acc], quote, c, false}} - - c == "#" and String.trim(prev) == "" -> - {:halt, {acc, nil, c, false}} - - c in ["'", "\""] and (String.trim(prev) == "" or prev in ["{", "[", ","]) -> - {:cont, {[c | acc], c, c, false}} - - true -> - {:cont, {[c | acc], nil, c, false}} - end - end) - |> elem(0) - |> Enum.reverse() - |> Enum.join() - |> String.trim_trailing() - end - - # The top-level `jobs:` key, quoted or not, optionally carrying YAML node - # properties (an anchor `&all_jobs`, a tag `!!map`) before its block value. - @jobs_key ~r/^(?:jobs|"jobs"|'jobs')\s*:\s*(?:(?:&[A-Za-z0-9_-]+|![^\s#]*)\s*)*(?:#.*)?$/ - - @doc """ - Follow block-style jobs by indentation, retaining physical source lines. - As with the other research rules, this is a local static text analysis. - """ - def workflow_job_lines(lines) do - {_in_jobs, _indent, groups} = - Enum.reduce(lines, {false, nil, []}, fn {line, _} = entry, {in_jobs, indent, groups} -> - cond do - Regex.match?(@jobs_key, line) -> - {true, nil, groups} - - not in_jobs -> - {false, indent, groups} - - Regex.match?(~r/^\S/, line) -> - {false, nil, groups} - - true -> - header = Regex.run(~r/^(\s+)(?:[A-Za-z0-9_-]+|"[^"]+"|'[^']+'):/, line) - width = if header, do: String.length(Enum.at(header, 1)), else: nil - - cond do - width && (is_nil(indent) || width == indent) -> - {true, width, [[entry] | groups]} - - groups != [] -> - [current | rest] = groups - {true, indent, [[entry | current] | rest]} - - true -> - {true, indent, groups} - end - end - end) - - groups |> Enum.reverse() |> Enum.map(&Enum.reverse/1) - end - - @doc """ - Read a workflow for the line- and indent-based rules. A KYAML workflow - (YAML-POLICY Y-3: the estate's base form, `yq -o kyaml`) is rewritten to - the equivalent block layout one line for one line, so physical line - numbers in findings still point at the source. Block YAML is returned - byte-identical. - """ - def read_workflow(path), do: path |> File.read!() |> flow_to_block() - - @doc """ - Rewrite a KYAML document to block layout, preserving the line count: - the root `{` and every closer become blank lines, `key: {`/`key: [` - lose their opener, the first key of a `{` sequence item and each scalar - sequence item gain `- `, trailing commas go, and plain double-quoted - scalars are unquoted. Anything whose first significant line is not a - bare `{` is not KYAML and passes through untouched — block YAML's own - `on: [push]` and `permissions: {}` are never rewritten. - """ - def flow_to_block(content) when is_binary(content) do - lines = String.split(content, "\n") - - if kyaml_root?(lines) do - {out, _stack, _dash} = Enum.reduce(lines, {[], [], false}, &flow_line/2) - out |> Enum.reverse() |> Enum.join("\n") - else - content - end - end - - # True when the first line that is neither blank nor a comment is a bare `{`. - defp kyaml_root?(lines) do - lines - |> Enum.map(&String.trim/1) - |> Enum.find(&(&1 != "" and not String.starts_with?(&1, "#"))) - |> Kernel.==("{") - end - - # Rewrite one KYAML line. The stack holds the open collections (`:map` or - # `:seq`); `dash` marks that the next property opens a sequence item. - defp flow_line(line, {out, stack, dash}) do - trimmed = String.trim(line) - - if trimmed == "" or String.starts_with?(trimmed, "#") do - {[dedent(line) | out], stack, dash} - else - {head, comment} = - case Regex.run(~r/^(.*?),?(\s+#[^"]*)?$/, String.trim_trailing(line)) do - [_, head, comment] -> {head, comment} - [_, head] -> {head, ""} - end - - indent = String.duplicate(" ", indent_of(head)) - body = String.trim(head) - in_seq? = match?([:seq | _], stack) - - cond do - body in ["}", "]"] -> - {["" | out], Enum.drop(stack, 1), false} - - body == "{" -> - {["" | out], [:map | stack], in_seq?} - - String.ends_with?(body, ": {") or String.ends_with?(body, ": [") -> - kind = if String.ends_with?(body, "{"), do: :map, else: :seq - key = String.slice(body, 0..-3//1) - {[emit(indent, key, comment, dash) | out], [kind | stack], false} - - in_seq? -> - item = Regex.replace(~r/^\{\s*(.*?)\s*\}$/, body, "\\1") - {[emit(indent <> " ", unquote_scalar(item), comment, true) | out], stack, false} - - true -> - {[emit(indent, unquote_scalar(body), comment, dash) | out], stack, false} - end - end - end - - # Assemble one block-layout line, two columns left of the KYAML one, with a - # `- ` sequence marker in the two columns just above the key when `dash`. - defp emit(indent, body, comment, dash) do - prefix = if dash, do: String.slice(indent, 2..-1//1) <> "- ", else: indent - dedent(prefix <> body <> (comment || "")) - end - - # Drop the two columns of indentation the KYAML root mapping adds. - defp dedent(" " <> rest), do: rest - defp dedent(line), do: line - - # Unquote a `key: "value"` or bare `"value"` whose value holds no quote or - # escape; anything else keeps its quotes. - defp unquote_scalar(body) do - Regex.replace(~r/^((?:[^\s"':]+:\s+)?)"([^"\\]*)"$/, body, "\\1\\2") - end - - @doc "The number of leading whitespace characters of `line`." - def indent_of(line) do - case Regex.run(~r/^(\s*)/, line) do - [_, ws] -> String.length(ws) - _ -> 0 - end - end -end diff --git a/lib/rules/research_extensions.ex b/lib/rules/research_extensions.ex index 31c853ce..cd6cc00c 100644 --- a/lib/rules/research_extensions.ex +++ b/lib/rules/research_extensions.ex @@ -108,10 +108,6 @@ defmodule Hypatia.Rules.ResearchExtensions do alias Hypatia.Workflow alias Hypatia.Workflow.Step - # The rules after RE002 still read lines; they share this primitive with - # Hypatia.WorkflowYaml until they move to the model too (#971). - import Hypatia.WorkflowYaml, only: [indent_of: 1] - # ─── Workflow-file discovery ──────────────────────────────────────── @doc false @@ -1799,16 +1795,107 @@ defmodule Hypatia.Rules.ResearchExtensions do } end - # ─── KYAML (flow-style) workflows ─────────────────────────────────── + # ─── KYAML (flow-style) workflows, for the line-based rules ───────── + + # RE003 and RE005 still read workflow text line by line, and so does + # `WorkflowAudit.check_sha_pinned_toolchain_without_input/1`, whose findings + # feed a fixer that rewrites files by line. Until they read + # `Hypatia.Workflow` (#971), a KYAML workflow is shown to them as its block + # layout, one line for one line. The rules that read the model (RE001, RE002, + # WH013, WH014) do not use this. + defp read_workflow(path), do: path |> File.read!() |> flow_to_block() + + # Rewrite a KYAML document to block layout, preserving the line count: + # the root `{` and every closer become blank lines, `key: {`/`key: [` + # lose their opener, the first key of a `{` sequence item and each scalar + # sequence item gain `- `, trailing commas go, and plain double-quoted + # scalars are unquoted. Anything whose first significant line is not a + # bare `{` is not KYAML and passes through untouched — block YAML's own + # `on: [push]` and `permissions: {}` are never rewritten. + @doc false + def flow_to_block(content) when is_binary(content) do + lines = String.split(content, "\n") - # ─── KYAML (flow-style) workflows ──────────────────── + if kyaml_root?(lines) do + {out, _stack, _dash} = Enum.reduce(lines, {[], [], false}, &flow_line/2) + out |> Enum.reverse() |> Enum.join("\n") + else + content + end + end - # Kept here for existing callers; the reader is Hypatia.WorkflowYaml. - @doc false - defdelegate read_workflow(path), to: Hypatia.WorkflowYaml + # True when the first line that is neither blank nor a comment is a bare `{`. + defp kyaml_root?(lines) do + lines + |> Enum.map(&String.trim/1) + |> Enum.find(&(&1 != "" and not String.starts_with?(&1, "#"))) + |> Kernel.==("{") + end - @doc false - defdelegate flow_to_block(content), to: Hypatia.WorkflowYaml + # Rewrite one KYAML line. The stack holds the open collections (`:map` or + # `:seq`); `dash` marks that the next property opens a sequence item. + defp flow_line(line, {out, stack, dash}) do + trimmed = String.trim(line) + + if trimmed == "" or String.starts_with?(trimmed, "#") do + {[dedent(line) | out], stack, dash} + else + {head, comment} = + case Regex.run(~r/^(.*?),?(\s+#[^"]*)?$/, String.trim_trailing(line)) do + [_, head, comment] -> {head, comment} + [_, head] -> {head, ""} + end + + indent = String.duplicate(" ", indent_of(head)) + body = String.trim(head) + in_seq? = match?([:seq | _], stack) + + cond do + body in ["}", "]"] -> + {["" | out], Enum.drop(stack, 1), false} + + body == "{" -> + {["" | out], [:map | stack], in_seq?} + + String.ends_with?(body, ": {") or String.ends_with?(body, ": [") -> + kind = if String.ends_with?(body, "{"), do: :map, else: :seq + key = String.slice(body, 0..-3//1) + {[emit(indent, key, comment, dash) | out], [kind | stack], false} + + in_seq? -> + item = Regex.replace(~r/^\{\s*(.*?)\s*\}$/, body, "\\1") + {[emit(indent <> " ", unquote_scalar(item), comment, true) | out], stack, false} + + true -> + {[emit(indent, unquote_scalar(body), comment, dash) | out], stack, false} + end + end + end + + # Assemble one block-layout line, two columns left of the KYAML one, with a + # `- ` sequence marker in the two columns just above the key when `dash`. + defp emit(indent, body, comment, dash) do + prefix = if dash, do: String.slice(indent, 2..-1//1) <> "- ", else: indent + dedent(prefix <> body <> (comment || "")) + end + + # Drop the two columns of indentation the KYAML root mapping adds. + defp dedent(" " <> rest), do: rest + defp dedent(line), do: line + + # Unquote a `key: "value"` or bare `"value"` whose value holds no quote or + # escape; anything else keeps its quotes. + defp unquote_scalar(body) do + Regex.replace(~r/^((?:[^\s"':]+:\s+)?)"([^"\\]*)"$/, body, "\\1\\2") + end + + # The width of a line's leading whitespace, in characters. + defp indent_of(line) do + case Regex.run(~r/^(\s*)/, line) do + [_, ws] -> String.length(ws) + _ -> 0 + end + end # ─── Internals ────────────────────────────────────────────────────── diff --git a/lib/rules/workflow_hardening.ex b/lib/rules/workflow_hardening.ex index a34ba6de..0da439af 100644 --- a/lib/rules/workflow_hardening.ex +++ b/lib/rules/workflow_hardening.ex @@ -75,9 +75,11 @@ defmodule Hypatia.Rules.WorkflowHardening do ## Implementation note - WH014 reads job/step structure and scalar values through `Hypatia.Workflow`. - Its shell analysis remains a bounded literal-path analysis. The remaining - rules still use the legacy readers below; their migration is tracked in #971. + WH013 and WH014 read job/step structure and scalar values through + `Hypatia.Workflow`: they judge a job by its own keys and values, and a + workflow the reader refuses is reported, not guessed at. WH014's shell + analysis remains a bounded literal-path analysis. The remaining rules still + use the legacy readers below; their migration is tracked in #971. Those remaining rules use regex-on-YAML-text rather than full YAML parsing because: @@ -96,7 +98,8 @@ defmodule Hypatia.Rules.WorkflowHardening do require Logger - alias Hypatia.{Kyaml, Workflow, WorkflowYaml} + alias Hypatia.{Kyaml, Workflow} + alias Hypatia.Workflow.Step # ─── Workflow-file discovery ──────────────────────────────────────── @@ -1136,60 +1139,155 @@ defmodule Hypatia.Rules.WorkflowHardening do Masking stays a defect there: `git push … 2>/dev/null || echo` cannot turn the job red when the credential expires or loses scope. That shape is reported at `:warn` with its own fix recipe, because no grant is missing. + + ## Reading the workflow + + The workflow is read through `Hypatia.Workflow`: grants are `permissions:` + values, writes and masks are read from the jobs' string values, a + credential from their `token:`/`GH_TOKEN:`/`GITHUB_TOKEN:` keys, and a + step's own `continue-on-error: true`, in any layout. A workflow the reader + refuses is reported at `:warn`, at the error's line, when its text names a + write, and skipped otherwise. """ def wh013_permission_starved_write(repo_path) do repo_path |> workflow_files() |> Enum.flat_map(fn path -> - content = WorkflowYaml.read_workflow(path) + bytes = File.read!(path) rel = Path.relative_to(path, repo_path) - {credentialed, token_writes} = - content - |> writing_jobs() - |> Enum.split_with(& &1.credentialed?) + case Workflow.parse(bytes, rel) do + {:ok, nil} -> + [] - masked_credentialed = - if Enum.any?(credentialed, &masked_job?/1), - do: [wh013_masked_credentialed_finding(rel)], - else: [] + {:ok, workflow} -> + wh013_workflow(rel, workflow) - wh013_starved_findings(rel, content, token_writes) ++ masked_credentialed + # A workflow the reader refuses is reported when its text names a + # write, and skipped otherwise: its grants and jobs cannot be read, + # and a silent skip of a writing workflow would read as a pass. + {:error, error} -> + if performs_contents_write?(bytes), do: [wh013_unreadable(rel, error)], else: [] + end end) end - # Each job that performs a write, as `%{text, steps, credentialed?}`, read - # by the shared workflow reader: comment lines dropped, aliases expanded, - # steps only under the job's own `steps:` key. A job is credentialed by its - # own text or by a workflow-level `env:` it inherits. A file whose jobs the - # reader cannot split is judged as one body and is never credentialed: one - # job's PAT must not excuse another job's write. - defp writing_jobs(content) do - case content |> WorkflowYaml.active_lines() |> WorkflowYaml.workflow_job_lines() do - [] -> - if performs_contents_write?(content), - do: [%{text: content, steps: [], credentialed?: false}], - else: [] + defp wh013_workflow(rel, workflow) do + {credentialed, token_writes} = + workflow + |> writing_jobs() + |> Enum.split_with(& &1.credentialed?) - jobs -> - inherited? = own_credential_job?(workflow_preamble(content)) + masked_credentialed = + if Enum.any?(credentialed, &masked_job?/1), + do: [wh013_masked_credentialed_finding(rel)], + else: [] - for job <- jobs, text = lines_text(job), performs_contents_write?(text) do - %{ - text: text, - steps: WorkflowYaml.job_steps(job), - credentialed?: inherited? or own_credential_job?(text) - } - end + wh013_starved_findings(rel, workflow, token_writes) ++ masked_credentialed + end + + defp wh013_unreadable(rel, error) do + %{ + rule: "WH013", + file: rel, + severity: :warn, + line: error.line, + reason: + "#{rel} could not be read as KYAML (#{error.code} at line #{error.line}, column " <> + "#{error.column}: #{error.message}), so WH013 cannot check that its writes " <> + "(push/commit/release/PR) are granted `contents: write`", + action: :report, + detail: %{ + line: error.line, + fix: + "Rewrite the construct the reader names in the KYAML subset (kyq " <> + "docs/KYAML_SPEC.adoc; standards YAML-POLICY). GitHub Actions refuses some of " <> + "these too, such as the `<<` merge key." + } + } + end + + # Each job that performs a write, as `%{values, steps, credentialed?}`, read + # from the parsed model (`Hypatia.Workflow`): `values` are the job's string + # values (YAML values, so a comment is never one; in a block scalar, lines + # that open with `#` are dropped as shell comments), `steps` its own steps. + # A job is credentialed by its own keys or by a workflow-level key it + # inherits (`env:`). A workflow with no jobs is judged as one body and is + # never credentialed: one job's PAT must not excuse another job's write. + defp writing_jobs(%Workflow{jobs: [], doc: doc}) do + values = scalar_values(doc) + + if writes?(values), + do: [%{values: values, steps: [], credentialed?: false}], + else: [] + end + + defp writing_jobs(%Workflow{doc: doc, jobs: jobs}) do + inherited? = + doc + |> Workflow.entries() + |> Enum.reject(&(&1.key.value == "jobs")) + |> Enum.flat_map(&entry_key_lines/1) + |> Enum.any?(&own_credential_job?/1) + + for job <- jobs, values = scalar_values(job.node), writes?(values) do + %{ + values: values, + steps: job.steps, + credentialed?: inherited? or Enum.any?(key_lines(job.node), &own_credential_job?/1) + } + end + end + + defp writes?(values), do: Enum.any?(values, &performs_contents_write?/1) + + # The string values under a node in document order, aliases read through. + # The workflow was read within kyq's node budget, so the walk is bounded. + defp scalar_values(nil), do: [] + + defp scalar_values(node) do + case Kyaml.deref(node) do + %{kind: :map, entries: entries} -> Enum.flat_map(entries, &scalar_values(&1.value)) + %{kind: :seq, items: items} -> Enum.flat_map(items, &scalar_values/1) + %{kind: :string, style: :block, value: value} -> [drop_comment_lines(value)] + %{kind: :string, value: value} -> [value] + _ -> [] end end - defp lines_text(lines), do: Enum.map_join(lines, "\n", &elem(&1, 0)) + defp drop_comment_lines(text) do + text + |> String.split("\n") + |> Enum.reject(&String.starts_with?(String.trim_leading(&1), "#")) + |> Enum.join("\n") + end + + # The keys a job (or the workflow) supplies a credential in, as one line of + # text each (`token: ${{ secrets.FARM_PAT }}`) for `own_credential_job?/1`, + # aliases read through. Keys are YAML keys, quoted or not, at any depth + # (`with:`, `env:`); text inside a scalar (a `run:` script that writes a + # file) is not a key. + @credential_keys ~w(token GH_TOKEN GITHUB_TOKEN) - # Everything above the top-level `jobs:` key (name, on, permissions, env). - # The key may be quoted (`"jobs":`, `'jobs':`), as WorkflowYaml reads it. - defp workflow_preamble(content) do - content |> String.split(~r/^(?:jobs|"jobs"|'jobs')\s*:/m, parts: 2) |> hd() + defp key_lines(node) do + case node && Kyaml.deref(node) do + %{kind: :map, entries: entries} -> Enum.flat_map(entries, &entry_key_lines/1) + %{kind: :seq, items: items} -> Enum.flat_map(items, &key_lines/1) + _ -> [] + end + end + + defp entry_key_lines(%{key: key, value: value}) do + own = + case {key, Workflow.text(value)} do + {%{kind: :string, value: name}, text} when name in @credential_keys and is_binary(text) -> + ["#{name}: #{text}"] + + _ -> + [] + end + + own ++ key_lines(value) end # `token:` / `GH_TOKEN:` / `GITHUB_TOKEN:` set to a secret other than @@ -1233,17 +1331,18 @@ defmodule Hypatia.Rules.WorkflowHardening do } end - # The grant is judged file-wide (`content`); starvation and masking only + # The grant is judged workflow-wide; starvation and masking only # over the jobs whose writes ride on GITHUB_TOKEN (`token_writes`). - defp wh013_starved_findings(rel, content, token_writes) do + defp wh013_starved_findings(rel, workflow, token_writes) do cond do token_writes == [] -> [] # A grant exists at either level. This rule deliberately does not try - # to prove the grant is on the RIGHT job — that needs a real YAML - # parse, and a false "you are broken" is worse than a missed one. - grants_contents_write_anywhere?(content) -> + # to prove the grant is on the RIGHT job (a job-level block replaces + # the workflow-level one, and an absent block inherits it): a false + # "you are broken" is worse than a missed one. + grants_contents_write_anywhere?(workflow) -> [] # No grant anywhere, and the write is masked so it can never go red. @@ -1297,12 +1396,21 @@ defmodule Hypatia.Rules.WorkflowHardening do @doc """ Return true when `contents: write` (or `write-all`) is granted at the - workflow level or at any job level. + workflow level or at any job level: a `permissions:` value of `write-all`, + or a `permissions:` mapping whose `contents` is `write`, in any layout + (`permissions: {contents: write}` is the same grant as the block form). + Text that only looks like a grant, such as a `run:` script that writes a + workflow file, is not one. """ - def grants_contents_write_anywhere?(content) when is_binary(content) do - Regex.match?(~r/^[ \t]*contents:\s*write\b/m, content) or - Regex.match?(~r/^[ \t]*permissions:\s*write-all\b/m, content) or - Regex.match?(~r/^[ \t]*write-all:\s*true\b/m, content) + def grants_contents_write_anywhere?(%Workflow{permissions: permissions, jobs: jobs}) do + Enum.any?([permissions | Enum.map(jobs, & &1.permissions)], &grants_contents_write?/1) + end + + defp grants_contents_write?(nil), do: false + + defp grants_contents_write?(permissions) do + Workflow.text(permissions) == "write-all" or + Workflow.text(Kyaml.get(permissions, "contents")) == "write" end # A write whose failure is swallowed: stderr redirected away, or the @@ -1314,11 +1422,6 @@ defmodule Hypatia.Rules.WorkflowHardening do ~r/(?:git\s+push|git\s+commit|gh\s+pr\s+(?:create|merge)|gh\s+release\s+(?:create|upload|edit))[^\n]*\|\|\s*echo/ ] - # `continue-on-error: true` as a key of the step, quoted or not. The prefix - # (indent, plus `- ` on the step's opening line) is captured so the key's - # column can be compared with the step's own key column. - @step_continue_on_error ~r/^(\s*(?:-\s+)?)(?:continue-on-error|"continue-on-error"|'continue-on-error')\s*:\s*true\b/ - @doc """ Return true when a write operation's failure is masked, so a denied write cannot turn the job red: a shell mask on the write itself, or @@ -1328,40 +1431,33 @@ defmodule Hypatia.Rules.WorkflowHardening do (standards' github-backup-mirror-reusable.yml documents and surfaces it). `content` is a workflow (or a fragment of one). Its steps are read with - `Hypatia.WorkflowYaml`, so only the items of each job's own `steps:` count; - a fragment without `jobs:` is judged by its shell masks alone. + `Hypatia.Workflow`, so only the items of each job's own `steps:` count; a + fragment the reader cannot read as a workflow is judged by its shell masks + alone. """ def masked_write?(content) when is_binary(content) do shell_masked_write?(content) or - content - |> WorkflowYaml.active_lines() - |> WorkflowYaml.workflow_job_lines() - |> Enum.flat_map(&WorkflowYaml.job_steps/1) - |> Enum.any?(&masked_step?/1) - end + case Workflow.parse(content) do + {:ok, %Workflow{jobs: jobs}} -> + Enum.any?(jobs, fn job -> Enum.any?(job.steps, &masked_step?/1) end) - defp masked_job?(%{text: text, steps: steps}) do - shell_masked_write?(text) or Enum.any?(steps, &masked_step?/1) + _ -> + false + end end - defp shell_masked_write?(text), do: Enum.any?(@write_masks, &Regex.match?(&1, text)) - - defp masked_step?(step) do - step_continue_on_error?(step) and performs_contents_write?(lines_text(step)) + defp masked_job?(%{values: values, steps: steps}) do + Enum.any?(values, &shell_masked_write?/1) or Enum.any?(steps, &masked_step?/1) end - # `continue-on-error: true` at the step's own key column. Deeper text (a - # `run: |` script that writes a workflow file, a `with:` value) is not the - # step's flag. - defp step_continue_on_error?(step) do - key_column = WorkflowYaml.step_key_column(step) + defp shell_masked_write?(text), do: Enum.any?(@write_masks, &Regex.match?(&1, text)) - Enum.any?(step, fn {line, _} -> - case Regex.run(@step_continue_on_error, line) do - [_, prefix] -> String.length(prefix) == key_column - nil -> false - end - end) + # `continue-on-error: true` as a key of the step itself, on a step that + # writes. Text deeper in the step (a `run: |` script that writes a workflow + # file, a `with:` value) is not the step's flag. + defp masked_step?(%Step{node: node}) do + match?(%{kind: :bool, value: true}, Kyaml.deref(Kyaml.get(node, "continue-on-error"))) and + writes?(scalar_values(node)) end # ─── Internals ────────────────────────────────────────────────────── diff --git a/test/workflow_hardening_test.exs b/test/workflow_hardening_test.exs index 59ecbdbf..53d1ee98 100644 --- a/test/workflow_hardening_test.exs +++ b/test/workflow_hardening_test.exs @@ -1953,6 +1953,8 @@ defmodule Hypatia.Rules.WorkflowHardeningTest do end test "literal working directories distinguish same-named report files" do + # The jq program goes in a block scalar: as a plain scalar, its `: ` + # makes the line invalid YAML, which GitHub (and the reader) refuse. for {input, expected?} <- [{"findings.json", false}, {"../scanner/findings.json", true}] do repo = create_repo_with_workflow(""" @@ -1972,6 +1974,7 @@ defmodule Hypatia.Rules.WorkflowHardeningTest do on_exit(fn -> File.rm_rf!(repo) end) findings = WorkflowHardening.wh014_masked_scanner_upload(repo) + assert Enum.all?(findings, &(&1.severity == :high)) assert findings != [] == expected? end end @@ -2109,4 +2112,329 @@ defmodule Hypatia.Rules.WorkflowHardeningTest do end end end + + # ─── WH013 and WH014 read the parsed model (#971) ─────────────────── + # + # Each test below, except the refused workflow that names no write, fails + # on the line reader the two rules used before (deleted with this change; + # see CHANGELOG.adoc); each pins a place where it read the workflow + # differently from YAML. + + describe "WH013 reads the parsed workflow model (#971)" do + test "a flow-style grant is a grant" do + repo = + create_repo_with_workflow(""" + name: Release + permissions: + contents: read + jobs: + publish: + runs-on: ubuntu-latest + permissions: { contents: write } + steps: + - run: git push origin HEAD + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [] = WorkflowHardening.wh013_permission_starved_write(repo) + end + + test "text that only looks like a grant is not one" do + for grant <- ["contents: write", "write-all: true"] do + repo = + create_repo_with_workflow(""" + name: Release + permissions: + contents: read + jobs: + publish: + runs-on: ubuntu-latest + steps: + - run: | + cat > generated.yml <<'YML' + permissions: + #{grant} + YML + git commit -am generated && git push origin HEAD + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{severity: :high, fix_recipe: "grant-contents-write-to-writing-job"}] = + WorkflowHardening.wh013_permission_starved_write(repo), + grant + end + end + + test "a YAML comment that names a write is not a write" do + repo = + create_repo_with_workflow(""" + name: CI + permissions: + contents: read + jobs: + test: + runs-on: ubuntu-latest + steps: + - run: mix test # the release job used to git push origin main + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [] = WorkflowHardening.wh013_permission_starved_write(repo) + end + + test "a credential a script writes into a file is not the job's credential" do + repo = + create_repo_with_workflow(""" + name: Release + permissions: + contents: read + jobs: + publish: + runs-on: ubuntu-latest + steps: + - run: | + cat > deploy.yml <<'YML' + GH_TOKEN: ${{ secrets.DEPLOY_PAT }} + YML + git push origin HEAD + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{severity: :high, fix_recipe: "grant-contents-write-to-writing-job"}] = + WorkflowHardening.wh013_permission_starved_write(repo) + end + + test "flow-style jobs are judged per job, with their credentials and step flags" do + # One job releases with its own PAT; the other pushes with GITHUB_TOKEN + # from a step marked continue-on-error. Written in flow style, the line + # reader judged the file as one body: never credentialed, never masked. + repo = + create_repo_with_workflow(""" + name: Release + permissions: { contents: read } + jobs: { + notes: { runs-on: ubuntu-latest, steps: [ + { env: { GH_TOKEN: "${{ secrets.RELEASE_PAT }}" }, run: "gh release create v1" } ] }, + record: { runs-on: ubuntu-latest, steps: [ + { continue-on-error: true, run: "git push origin HEAD" } ] } + } + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{severity: :high, fix_recipe: "grant-contents-write-to-writing-job-and-unmask"}] = + WorkflowHardening.wh013_permission_starved_write(repo) + end + + test "a workflow-level env written after `jobs:` still covers every job" do + repo = + create_repo_with_workflow(""" + name: Release + permissions: + contents: read + jobs: + publish: + runs-on: ubuntu-latest + steps: + - run: gh release create v1 dist/* + env: + GH_TOKEN: ${{ secrets.RELEASE_PAT }} + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [] = WorkflowHardening.wh013_permission_starved_write(repo) + end + + test "a writing workflow the reader refuses is reported at the error, not guessed at" do + repo = + create_repo_with_workflow(""" + name: Release + x-defaults: &defaults + runs-on: ubuntu-latest + jobs: + publish: + <<: *defaults + permissions: + contents: write + steps: + - run: git push origin HEAD + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{rule: "WH013", severity: :warn, line: 6, reason: reason}] = + WorkflowHardening.wh013_permission_starved_write(repo) + + assert reason =~ "could not be read" + end + + test "a refused workflow that names no write is not reported" do + repo = + create_repo_with_workflow(""" + x-defaults: &defaults + runs-on: ubuntu-latest + jobs: + test: + <<: *defaults + steps: + - run: mix test + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [] = WorkflowHardening.wh013_permission_starved_write(repo) + end + end + + describe "WH014 reads the parsed workflow model (#971)" do + test "a KYAML workflow is read, with its flow-style `with:`" do + repo = + create_repo_with_workflow(""" + { + jobs: { + scan: { + "runs-on": "ubuntu-latest", + steps: [ + { + run: "hypatia scan . > findings.json || true", + }, + { + run: "jq '{runs: [{results: .}]}' findings.json > out.sarif", + }, + { + uses: "github/codeql-action/upload-sarif@v4", + with: {sarif_file: "out.sarif"}, + }, + ], + }, + }, + } + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{rule: "WH014", severity: :high, detail: %{jobs: ["scan"]}}] = + WorkflowHardening.wh014_masked_scanner_upload(repo) + end + + test "a folded or quoted run script is read as its YAML value" do + for run <- [ + ">-\n hypatia scan . > findings.json\n || true", + ~s("hypatia scan . > findings.json || true"), + "'hypatia scan . > findings.json || true'" + ] do + repo = + create_repo_with_workflow(""" + jobs: + scan: + runs-on: ubuntu-latest + steps: + - run: #{run} + - run: | + jq '{runs: [{results: .}]}' findings.json > out.sarif + - uses: github/codeql-action/upload-sarif@v4 + with: + sarif_file: out.sarif + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{rule: "WH014", severity: :high}] = + WorkflowHardening.wh014_masked_scanner_upload(repo), + run + end + end + + test "a job's defaults.run.working-directory counts wherever the key is written" do + for {sarif_file, expected?} <- [{"out.sarif", false}, {"scanner/out.sarif", true}] do + repo = + create_repo_with_workflow(""" + jobs: + scan: + runs-on: ubuntu-latest + steps: + - run: hypatia scan . > findings.json || true + - run: | + jq '{runs: [{results: .}]}' findings.json > out.sarif + - uses: github/codeql-action/upload-sarif@v4 + with: + sarif_file: #{sarif_file} + defaults: + run: + working-directory: scanner + """) + + on_exit(fn -> File.rm_rf!(repo) end) + findings = WorkflowHardening.wh014_masked_scanner_upload(repo) + assert findings != [] == expected?, sarif_file + end + end + + test "a reusable workflow's `working-directory` input is not a default directory" do + repo = + create_repo_with_workflow(""" + on: + workflow_call: + inputs: + working-directory: + type: string + default: . + jobs: + scan: + runs-on: ubuntu-latest + steps: + - run: hypatia scan . > findings.json || true + - run: | + jq '{runs: [{results: .}]}' findings.json > out.sarif + - uses: github/codeql-action/upload-sarif@v4 + with: + sarif_file: out.sarif + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [%{rule: "WH014"}] = WorkflowHardening.wh014_masked_scanner_upload(repo) + end + + test "a step whose working directory is an expression is not followed" do + repo = + create_repo_with_workflow(""" + jobs: + scan: + runs-on: ubuntu-latest + steps: + - working-directory: ${{ inputs.dir }} + run: hypatia scan . > findings.json || true + - run: | + jq '{runs: [{results: .}]}' findings.json > out.sarif + - uses: github/codeql-action/upload-sarif@v4 + with: + sarif_file: out.sarif + """) + + on_exit(fn -> File.rm_rf!(repo) end) + assert [] = WorkflowHardening.wh014_masked_scanner_upload(repo) + end + + test "an uploading workflow the reader refuses is reported at the error" do + repo = + create_repo_with_workflow(""" + x-defaults: &defaults + runs-on: ubuntu-latest + jobs: + scan: + <<: *defaults + steps: + - run: hypatia scan . > findings.json || true + - uses: github/codeql-action/upload-sarif@v4 + with: + sarif_file: findings.json + """) + + on_exit(fn -> File.rm_rf!(repo) end) + + assert [%{rule: "WH014", severity: :warn, line: 5}] = + WorkflowHardening.wh014_masked_scanner_upload(repo) + end + end end From 5735428d03392aa854dd5a290385a825509ea747 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 16 Nov 2026 19:58:37 +0000 Subject: [PATCH 2/3] Keep the two public helpers' contracts through the WH013 migration Codex on #1192: `ResearchExtensions.read_workflow/1` was a public `@doc false` delegate, and `WorkflowHardening.grants_contents_write_anywhere?/1` took workflow text. The migration made the first private and the second accept only the parsed model. No caller in the estate depends on either, but both are public, so they keep their shape: `read_workflow/1` stays a public `@doc false` def, and the grant check takes workflow text again, parsing it and delegating to the model clause. Text the reader refuses grants nothing. Two tests; both fail on fc60ab1. Under Elixir 1.19.4 / OTP 27 the format check passes, the changed modules compile with no warnings, and the workflow-rule suites pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01J4QvDithtn5Xk5MbSPm4Px --- lib/rules/research_extensions.ex | 3 ++- lib/rules/workflow_hardening.ex | 10 ++++++++++ test/workflow_hardening_test.exs | 24 ++++++++++++++++++++++++ 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/lib/rules/research_extensions.ex b/lib/rules/research_extensions.ex index cd6cc00c..06793915 100644 --- a/lib/rules/research_extensions.ex +++ b/lib/rules/research_extensions.ex @@ -1803,7 +1803,8 @@ defmodule Hypatia.Rules.ResearchExtensions do # `Hypatia.Workflow` (#971), a KYAML workflow is shown to them as its block # layout, one line for one line. The rules that read the model (RE001, RE002, # WH013, WH014) do not use this. - defp read_workflow(path), do: path |> File.read!() |> flow_to_block() + @doc false + def read_workflow(path), do: path |> File.read!() |> flow_to_block() # Rewrite a KYAML document to block layout, preserving the line count: # the root `{` and every closer become blank lines, `key: {`/`key: [` diff --git a/lib/rules/workflow_hardening.ex b/lib/rules/workflow_hardening.ex index 0da439af..6a1be055 100644 --- a/lib/rules/workflow_hardening.ex +++ b/lib/rules/workflow_hardening.ex @@ -1401,7 +1401,17 @@ defmodule Hypatia.Rules.WorkflowHardening do (`permissions: {contents: write}` is the same grant as the block form). Text that only looks like a grant, such as a `run:` script that writes a workflow file, is not one. + + It takes the parsed model or workflow text. Text the reader refuses grants + nothing. """ + def grants_contents_write_anywhere?(content) when is_binary(content) do + case Workflow.parse(content) do + {:ok, %Workflow{} = workflow} -> grants_contents_write_anywhere?(workflow) + _ -> false + end + end + def grants_contents_write_anywhere?(%Workflow{permissions: permissions, jobs: jobs}) do Enum.any?([permissions | Enum.map(jobs, & &1.permissions)], &grants_contents_write?/1) end diff --git a/test/workflow_hardening_test.exs b/test/workflow_hardening_test.exs index 53d1ee98..de7cde14 100644 --- a/test/workflow_hardening_test.exs +++ b/test/workflow_hardening_test.exs @@ -2121,6 +2121,30 @@ defmodule Hypatia.Rules.WorkflowHardeningTest do # differently from YAML. describe "WH013 reads the parsed workflow model (#971)" do + test "the grant check still takes workflow text, and text the reader refuses grants nothing" do + assert WorkflowHardening.grants_contents_write_anywhere?( + "on: push\njobs:\n a:\n permissions: {contents: write}\n runs-on: x\n" + ) + + refute WorkflowHardening.grants_contents_write_anywhere?( + "on: push\njobs:\n a:\n runs-on: x\n steps:\n - run: echo 'contents: write' > p.yml\n" + ) + + refute WorkflowHardening.grants_contents_write_anywhere?("jobs: [unclosed\n") + end + + test "ResearchExtensions.read_workflow/1 stays callable from outside the module" do + path = + Path.join( + System.tmp_dir!(), + "wh013-read-workflow-#{System.unique_integer([:positive])}.yml" + ) + + File.write!(path, "on: push\n") + assert Hypatia.Rules.ResearchExtensions.read_workflow(path) == "on: push\n" + File.rm!(path) + end + test "a flow-style grant is a grant" do repo = create_repo_with_workflow(""" From b7b4cf3f9c98406ce760347c0999a93eeb67db78 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 16 Nov 2026 20:14:09 +0000 Subject: [PATCH 3/3] README: lib/ holds 171 Elixir source files after WorkflowYaml's removal Validate Documentation's readme-claims check counts the tree and failed on 5735428: "lib/ is 171 in the tree; README.adoc should say 171". Deleting lib/hypatia/workflow_yaml.ex took the count from 172 to 171; CLAUDE.md was updated in c3b3c14, README.adoc was not. scripts/check-readme-claims.sh now reports "counts and local links match the tree". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01J4QvDithtn5Xk5MbSPm4Px --- README.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.adoc b/README.adoc index 31854362..e42bb8f2 100644 --- a/README.adoc +++ b/README.adoc @@ -76,7 +76,7 @@ PatternRegistry ──> TriangleRouter ──> FleetDispatcher ──> DispatchM [cols="1,3"] |=== -| `lib/` | 172 Elixir source files — pipeline, rules, VCL, neural, safety, web, merge orchestration +| `lib/` | 171 Elixir source files — pipeline, rules, VCL, neural, safety, web, merge orchestration | `lib/rules/` | 44 source files: rule modules and their helpers; see link:docs/rules/catalogue.adoc[the catalogue] | `lib/neural/` | 8 networks + blackboard, coordinator, persistence, rebalancer | `lib/vcl/` | VCL query layer over verisim-data: parser, file executor, cache, federation