[sjoblomj]

List the encryption keys of files whose names aren't known

tl;dr:

When listing files and requesting encryption-key or encryption-key-raw for archives with files of unknown name, the result is just 0. If we read a single byte from these files rather than just list them, we trigger StormLib to work out the key. That's what this PR does.

Longer explanation

mpqcli list -p encryption-key -p encryption-key-raw printed 00000000 for encrypted files whose names aren't known, which StormLib lists as FileXXXXXXXX.<ext>.

StormLib works out the key of an encrypted file from its name when the file is opened. Without the name, it can still find the key, but only on the first read. For a compressed file it uses the sector offset table, and for an uncompressed one it looks for a known file header (RIFF, MZ, XML). list never read the files, so their key stayed at 0.

For encryption-key-raw, the previous values are just file-size − byte-offset (mod 2³²). They aren't real keys. They're what you get when the key is 0 and you undo the MPQ_FILE_KEY_V2 (FIX_KEY) adjustment.

Changes

• When encryption-key or encryption-key-raw is requested, list reads one byte of each encrypted file whose key is still 0, so StormLib finds the key before it is printed. Files opened by name already have their key and unencrypted files have none, so neither is read. Listings that don't print a key are unchanged.
• New test test_list_encryption_keys_of_files_listed_without_names. It lists an MPQ that has no internal listfile twice, first without names and then with an external listfile. It checks that every encrypted file gets the same non-zero keys both times.

If StormLib can't find the key, for example for an uncompressed file whose content it doesn't recognise, the key is still shown as 00000000.

[sjoblomj]

Without this PR:

$ mpqcli list StarDat.mpq -da -p file-index -p name-hash1 -p name-hash2 -p encryption-key -p encryption-key-raw | sort -n
...
 2915 8496ee2c 201bfdf5 00000000 fc3b3cd7  File00002915.xxx
 2916 3acefa3d d13b9d4f 00000000 fc3b316d  File00002916.xxx
 2917 31aae4cf 47684a19 00000000 fc3b0f58  File00002917.xxx
 2918 3e50e476 ac83186c 00000000 fc3b4b89  File00002918.xxx
 2919 344541ec bc5f9961 00000000 00000000  File00002919.smk
 2920 0d16da7f 9eed9b53 00000000 fc17b8d5  File00002920.exe
 2921 a8a0c767 013aaf95 00000000 fc164aa9  File00002921.exe
 2922 dd3ef02a 73239a42 00000000 fc1206e1  File00002922.exe
 2923 a60678ce aaf79a9e 00000000 fc0fd1cc  File00002923.xxx

With this PR:

$ mpqcli list StarDat.mpq -da -p file-index -p name-hash1 -p name-hash2 -p encryption-key -p encryption-key-raw | sort -n
...
 2915 8496ee2c 201bfdf5 1c5099d8 188bb61f  File00002915.xxx
 2916 3acefa3d d13b9d4f f5b86bfc f1f307d9  File00002916.xxx
 2917 31aae4cf 47684a19 c916dcb6 c5515c0e  File00002917.xxx
 2918 3e50e476 ac83186c c0599476 bc93b7ff  File00002918.xxx
 2919 344541ec bc5f9961 00000000 00000000  File00002919.smk
 2920 0d16da7f 9eed9b53 51e451d6 4dfc0aab  File00002920.exe
 2921 a8a0c767 013aaf95 62cc392d 5ee2337e  File00002921.exe
 2922 dd3ef02a 73239a42 6852ae64 64646885  File00002922.exe
 2923 a60678ce aaf79a9e 7ef38430 7b0355fc  File00002923.xxx

As you can see, the encryption-key and encryption-key-raw are now different. The smk file (2919) is not encrypted, so it remains with values 00000000.