bin/flash-gui.sh & initrd/bin/flash.sh: Show SHA256SUM for manual verification prior of flashing

[tlaurion]

Add ROM and ROM hash when non-npf file is selected for flashing as per #1514 (comment)

---

Old

• Refactoring of arguments parsing, so they are not positional.
• Internal usage of -i argument from flash-gui.sh to flash.sh to bypass hash validation prompt in case of npf
• Rewording of flash-gui.sh output in case of non-npf file
  
• Adds this validation prompt so the user can validate hash output from external source prior of accepting flashing
  

Notes:

• Talos II still only user of tgz archive which is similar to npf and doesn't show prompt either.

[tlaurion]

Cleaner code at 6f253cd: -i doesn't need any argument, just as -c and -r.

And all other flash.sh caller verified to see if they needed to have -i passed (no manual integrity validation needed since rom backup/cbfs injected or npf file provided).

From command line, one can test manually with (done personally)

• flash.sh -r /tmp/test.rom
• flash.sh -c /tmp/test.rom
• flash.sh /tmp/test.rom
• flash.sh /tmp/test.rom -i
• flash.sh -c /tmp/test.rom -i

Also, config manager, oem-factory-reset and other callers who backup and inject files in cbfs were verified (-i passed).
npf files bypasses this, as well as tgz firmware package (talos).

@JonathonHall-Purism please review!

[JonathonHall-Purism]

(thanks for patience while I was away 😁 )

@tlaurion I support integrity checks on the ROMs in general, but the workflow changes here put me in a tough place to distribute future updates - there's no single file I can distribute that would work well on all recent releases, since NPF support was just merged recently.

Adding another incompatible update format also complicates life a bit - we support external and manual flashing, so plain ROMs are still needed for that.

I will still have to ship plain ROMs until the NPF has been out for long enough that we can break backward compatibility, so I don't think a long, scary, wordy prompt is appropriate. Purism's update script verifies the integrity of the decompressed ROM, so it does eliminate many (but admittedly not all) sources of corruption. FWIW I have not had any reports of corrupt flashes being a problem.

Could we do this in a backward compatible way? Possibly, add a cbfs region to the ROM designed for the SHA-256 digest. It could be initially all zero, calculate the digest of that ROM, then replace it with the actual digest. Verification would zero it again to check the digest. This way the ROM could still be flashed as-is or by an older release unaware of the digest.

---

Also, IMO, three confirmation prompts for flashing a plain ROM is excessive:

Could we combine this into one prompt? Such as:

This will replace your current ROM with:

$(basename of ROM)

{only for a plain ROM -->}  Heads could not verify the integrity of this file.  Please confirm the SHA-256 hash:
SHA256: aaaaabbbbbbcccccdddddd....

Do you want to proceed?

[JonathonHall-Purism]

I also noted a few problems in the current state of NPF support. (I don't think I had a chance to review it while it was in a PR)

• Trying to flash an NPF a second time doesn't work (say, if the first one was bad) - needs to rm -rf /tmp/verified_rom
• Should show basename rather than /tmp/verified_rom/ in the prompt, /tmp/verified_rom is an implementation detail / clutter
• create-npf.sh doesn't use git describe in the same way as Makefile, so it doesn't always get the correct ROM file name
• I don't love the fact that /tmp/verified_rom has to appear in the sha256sum.txt, that's tying the file format to current implementation details. Heads should be able to handle a relative path here IMO

I worked on create-npf.sh a bit to fix the git describe and add a way to generate a bad NPF:

#!/bin/bash
set -exuo pipefail

ARG_BAD=

ARGS_DONE=

usage() {
	cat <<USAGE_END
usage:
	$1 [--bad] <board>
	$1 --help

arguments:
	--bad: Intentionally alter the SHA256 checksum to test a corrupted ROM
	--help: Show usage
	<board>: Name of board, the board's ROM must already be built
USAGE_END
}

while [ -z "$ARGS_DONE" ]; do
	case "$1" in
		--help)
			usage
			exit 0
			;;
		--bad)
			ARG_BAD=y
			shift
			;;
		--)
			ARGS_DONE=y
			shift
			;;
		--*)
			echo "unknown argument: $1" >&2
			usage
			exit 1
			;;
		*)
			ARGS_DONE=y
			;;
	esac
done

HEADS_GIT_VERSION=$(git describe --abbrev=7 --tags --dirty)
BOARD="$1"
cd ./build/x86/${BOARD}/
sha256sum heads-${BOARD}-${HEADS_GIT_VERSION}.rom > sha256sum.txt
sed -ie 's@  @  /tmp/verified_rom/@g' sha256sum.txt
if [ -n "$ARG_BAD" ]; then
	sed -ie 's@^........@00000000@g' sha256sum.txt
fi
zip "heads-${BOARD}-${HEADS_GIT_VERSION}${ARG_BAD:+-bad}.npf" "heads-${BOARD}-${HEADS_GIT_VERSION}.rom" sha256sum.txt

[JonathonHall-Purism]

Nitpick, the comments here shouldn't be worded with "refactor" since this refactoring has been done at this point

[tlaurion]

@JonathonHall-Purism

To be honest, I missed that prompt in my approval of nv41/ns50 merge which included npf support (nitrokey format).
So as of today, manual builders, Heads CircleCI and purism provides unverified rom files, where Talos II distribute parts of roms from CI or a tgz format that is validated upon firmware upgrade.

Basically, Heads now supports npf if provided, but most people will get to second prompt and third prompt with current state of PR since most do not use npf files but Nitrokey.

I think we could easily modify codebase to merge second and third prompts above together, showing FLASH ROM? with sha256sum calculated.

As for CBFS hash file of rom, Heads has this hidden feature as of now: reflashing the same externally provided rom as currently flashed will result in preservation of cbfs user files being added in ROM, resulting in temp rom matching what is currently flashed. As a result, going forward and flashing same git commit hash will result in no flashrom write. We could add a prompt when the firmware flashed is not resulting in a real write of the SPI flash where flashrom reports no change. This hidden feature could bring awareness to the users on the fact that upon reboot, TOTP/HOTP will be the same. Reflashing current ROM is actually a proper way of making sure that the firmware on SPI is the same as on external flashing media and is recommended to be reflashed in case of doubts. @JonathonHall-Purism thoughts? Based on that, I would not add a CBFS file, but I am ready to be challenged on that. The whole filename and git hash can be used to track the sha256sum of rom and therefore the original hash of the ROM that was flashed for verification and reflashing.

TLDR regarding this PR further changes:

• We could combine second ROM flash prompt with third so that SHA256SUM hash is showed for manual verification by the user.
• @JonathonHall-Purism I suggest you propose fixes to npf-create.sh in a seperate PR so that @daringer can review from bin/flash-gui.sh & initrd/bin/flash.sh: Show SHA256SUM for manual verification prior of flashing #1514 (comment)

Will look in merging the two last prompts

[JonathonHall-Purism]

@JonathonHall-Purism

To be honest, I missed that prompt in my approval of nv41/ns50 merge which included npf support (nitrokey format). So as of today, manual builders, Heads CircleCI and purism provides unverified rom files, where Talos II distribute parts of roms from CI or a tgz format that is validated upon firmware upgrade.

Basically, Heads now supports npf if provided, but most people will get to second prompt and third prompt with current state of PR since most do not use npf files but Nitrokey.

I think we could easily modify codebase to merge second and third prompts above together, showing FLASH ROM? with sha256sum calculated.

I still think we should combine all three prompts, not just the last two. There's no reason to ask the same question twice. The NPF change still doesn't offer a solution for me to actually switch to distributing NPF cleanly due to supporting backward compatibility and external flashing, so I'm going to have to continue distributing ROMs as-is for the foreseeable future.

As for CBFS hash file of rom, Heads has this hidden feature as of now: reflashing the same externally provided rom as currently flashed will result in preservation of cbfs user files being added in ROM, resulting in temp rom matching what is currently flashed. As a result, going forward and flashing same git commit hash will result in no flashrom write. We could add a prompt when the firmware flashed is not resulting in a real write of the SPI flash where flashrom reports no change. This hidden feature could bring awareness to the users on the fact that upon reboot, TOTP/HOTP will be the same. Reflashing current ROM is actually a proper way of making sure that the firmware on SPI is the same as on external flashing media and is recommended to be reflashed in case of doubts. @JonathonHall-Purism thoughts? Based on that, I would not add a CBFS file, but I am ready to be challenged on that. The whole filename and git hash can be used to track the sha256sum of rom and therefore the original hash of the ROM that was flashed for verification and reflashing.

I don't see how any of this relates to "verifying integrity of an update" to avoid a corrupted flash, which it seems the NPF change is trying to solve. (Which is a good thing to solve, but I would prefer to solve it in a way that is easier to support.)

The "No flashrom write" scenario doesn't help avoid corrupted firmware images because you've already flashed it at that point. It also doesn't offer any assurance that your firmware wasn't tampered, because the tampered firmware could lie.

If I mock up a strategy with a backward-compatible built-in digest, would you consider moving to that instead of NPF?

[JonathonHall-Purism]

@tlaurion I mocked up an alternative in #1518 that is backward compatible. Let me know what you think 😁 If we can go this route, then I won't mind noise as much on the "unverified ROM" path because I can start distributing verifiable ROMs without breaking backward compatibility.

[tlaurion]

722121b replaces multiple prompts with the following

(Also fixes whiptail fixed size window, text previously neglecting *.npf promotion)

@JonathonHall-Purism @daringer

[tlaurion]

@daringer: I could fix npf-builder to build zip files, and have zip files support (where npf is a zip rebranding I'm not sure how to handle under Heads as of now).

Also fixing things to remove /tmp/verified_rom.

@JonathonHall-Purism @daringer ? This needs a little bit of prioritizing, current state of master with npf promotion without Heads building npf is just weird.

[JonathonHall-Purism]

This is unreachable now and it is the only place flash.sh is invoked without -i. Can we please delete the integrity-verified variables, the -i parameter to flash.sh, and confirm_rom_sha256sum_hash?

[tlaurion]

right. any other change is irrelevant as of current scope. Calling flash.sh manually implies one knows what he is doing.

[tlaurion]

@daringer @JonathonHall-Purism: c0cf446 compared to master just changes the main rom prompt (to promote npf which was missing) and non-npf rom selection so show ROM hash (with automatic length whiptail prompt), and stating that verification cannot be automatically done on rom file type, removing all other changes in codebase.

[JonathonHall-Purism]

@tlaurion Thanks for working on this. Given the current state of master, I'm good with merging this. I just tested it on Mini v1 to confirm the behavior when flashing a plain ROM.

Re: the bigger issues around NPF and the built-in alternative in #1518, I have asked our support team about migration strategies to start distributing a ZIP-based format which is not backward compatible, which will help determine where we stand on that. I'll follow up on #1518 and/or other PRs if we open them, I have the create_npf.sh fixes from above that I will put in a branch if we go that route.