diff --git a/README-es.md b/README-es.md index 1f157390..5d446d57 100644 --- a/README-es.md +++ b/README-es.md @@ -297,7 +297,7 @@ Cuando no se especifica `type`, DLoad automáticamente usa todos los manejadores > Usar `type="archive"` extraerá incluso archivos `.phar`. > [!NOTE] -> Cuando hay un `` configurado, con `type="binary"` o sin `type`, los paquetes del sistema operativo (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) nunca se seleccionan: DLoad no puede desempaquetarlos. Un release que solo ofrece un paquete para el host se omite en favor de uno más antiguo, entre los 10 releases más recientes que cumplen la restricción de versión; fija `version` para ir más atrás. +> Cuando hay un `` configurado, con `type="binary"` o sin `type`, los paquetes del sistema operativo (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) nunca se seleccionan: DLoad no puede desempaquetarlos. Tampoco los archivos comprimidos en un formato que DLoad no puede desempaquetar: los formatos de archivo comprimido conocidos distintos de `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz` y `.phar`, como `.tar.xz`, `.tar.bz2`, `.zst`, `.7z` o `.rar`; un archivo sin una extensión de archivo comprimido conocida, como un binario simple o un `.AppImage`, sí se selecciona. Un release que solo ofrece estos assets para el host se omite en favor de uno más antiguo, entre los 10 releases más recientes que cumplen la restricción de versión; fija `version` para ir más atrás. #### Extracción de Archivos (Preservando la Estructura) diff --git a/README-ru.md b/README-ru.md index c3c7839b..e501b8a8 100644 --- a/README-ru.md +++ b/README-ru.md @@ -298,7 +298,7 @@ DLoad поддерживает три типа загрузки, которые > Использование `type="archive"` распакует даже `.phar` архивы. > [!NOTE] -> Если задан `` — с `type="binary"` или без `type`, — пакеты ОС (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) никогда не выбираются: DLoad не умеет их распаковывать. Релиз, в котором для хоста есть только пакет, пропускается ради более старого — среди 10 новейших релизов, удовлетворяющих ограничению версии; чтобы уйти дальше в прошлое, закрепите `version`. +> Если задан `` — с `type="binary"` или без `type`, — пакеты ОС (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) никогда не выбираются: DLoad не умеет их распаковывать. Не выбираются и архивы форматов, которые DLoad не распаковывает, — известные форматы архивов, кроме `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz` и `.phar`, например `.tar.xz`, `.tar.bz2`, `.zst`, `.7z` или `.rar`; файл без известного расширения архива, такой как обычный бинарник или `.AppImage`, по-прежнему выбирается. Релиз, в котором для хоста есть только такие ассеты, пропускается ради более старого — среди 10 новейших релизов, удовлетворяющих ограничению версии; чтобы уйти дальше в прошлое, закрепите `version`. #### Распаковка архива с сохранением структуры diff --git a/README-zh.md b/README-zh.md index 55313381..5f10681c 100644 --- a/README-zh.md +++ b/README-zh.md @@ -297,7 +297,7 @@ DLoad 支持三种下载类型,它们决定了资源的处理方式: > 使用 `type="archive"` 会解包甚至 `.phar` 压缩包。 > [!NOTE] -> 配置了 `` 时(无论是 `type="binary"` 还是未指定 `type`),永远不会选择操作系统安装包(`.deb`、`.rpm`、`.apk`、`.snap`、`.flatpak`、`.msi`、`.msix`、`.msixbundle`、`.appx`、`.appxbundle`、`.nupkg`、`.dmg`、`.pkg`):DLoad 无法解包它们。如果某个发布版本只为当前主机提供安装包,则会跳过它,改用更早的版本,范围限于满足版本约束的最新 10 个发布版本;要回溯更早的版本,请固定 `version`。 +> 配置了 `` 时(无论是 `type="binary"` 还是未指定 `type`),永远不会选择操作系统安装包(`.deb`、`.rpm`、`.apk`、`.snap`、`.flatpak`、`.msi`、`.msix`、`.msixbundle`、`.appx`、`.appxbundle`、`.nupkg`、`.dmg`、`.pkg`):DLoad 无法解包它们。DLoad 无法解包的格式的压缩包同样不会被选择——即 `.zip`、`.tar.gz`、`.tgz`、`.tar`、`.gz` 和 `.phar` 以外的已知压缩格式,例如 `.tar.xz`、`.tar.bz2`、`.zst`、`.7z` 或 `.rar`;没有已知压缩包扩展名的文件(例如普通二进制文件或 `.AppImage`)仍会被选择。如果某个发布版本只为当前主机提供这类资源,则会跳过它,改用更早的版本,范围限于满足版本约束的最新 10 个发布版本;要回溯更早的版本,请固定 `version`。 #### 解包压缩包(保留结构) diff --git a/README.md b/README.md index c09ca8e6..6700098d 100644 --- a/README.md +++ b/README.md @@ -298,7 +298,7 @@ When `type` is not specified, DLoad automatically uses all available handlers: > Using `type="archive"` will unpack even `.phar` archives. > [!NOTE] -> When a `` is configured, with `type="binary"` or without a `type`, OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) are never selected: DLoad cannot unpack them. A release that offers only a package for the host is skipped for an older one, among the 10 newest releases that satisfy the version constraint; pin `version` to go further back. +> When a `` is configured, with `type="binary"` or without a `type`, OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) are never selected: DLoad cannot unpack them. Neither are archives of a format DLoad cannot unpack — known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz` and `.phar`, such as `.tar.xz`, `.tar.bz2`, `.zst`, `.7z` or `.rar`; a file without a known archive extension, like a plain binary or an `.AppImage`, is still selected. A release that offers only such assets for the host is skipped for an older one, among the 10 newest releases that satisfy the version constraint; pin `version` to go further back. #### Archive Extraction (Preserving Structure) diff --git a/skills/dload-fetch-tool/references/registry-entry.md b/skills/dload-fetch-tool/references/registry-entry.md index 51d8d3cb..d6912755 100644 --- a/skills/dload-fetch-tool/references/registry-entry.md +++ b/skills/dload-fetch-tool/references/registry-entry.md @@ -22,7 +22,7 @@ Filters the release's asset list. dload then runs OS/arch detection on every mat - match all OS/arch variants of the tool; - leave out sibling tools, checksums, signatures, source archives. -Build variants stay in the pattern too. Among the assets for the host, dload prefers the host libc (`musl` builds on Alpine and Android, the others elsewhere), then the name with the fewest extra tokens — so `bun-linux-x64.zip` wins over `-baseline`, `-profile` and `-debug` twins — then archives. Android builds are dropped on Linux whenever the entry has a `binary` (without one, every other platform is only ranked lower), and so are OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) everywhere: dload cannot unpack them, so a release that has only a package for the host gives way to an older one among the 10 newest releases that satisfy `version` (pin `version` to go further back). On ARM macOS and Windows, x86-64 builds are a fallback: Rosetta 2 and the Windows emulation run them. +Build variants stay in the pattern too. Among the assets for the host, dload prefers the host libc (`musl` builds on Alpine and Android, the others elsewhere), then the name with the fewest extra tokens — so `bun-linux-x64.zip` wins over `-baseline`, `-profile` and `-debug` twins — then archives. Android builds are dropped on Linux whenever the entry has a `binary` (without one, every other platform is only ranked lower), and so are OS packages (`.deb`, `.rpm`, `.apk`, `.snap`, `.flatpak`, `.msi`, `.msix`, `.msixbundle`, `.appx`, `.appxbundle`, `.nupkg`, `.dmg`, `.pkg`) and archives dload has no extractor for (known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz`, `.phar` — such as `.tar.xz`, `.tar.bz2`, `.zst`, `.pkg.tar.zst`, `.7z`, `.rar`) everywhere: dload cannot unpack them, so a release that has only such assets for the host gives way to an older one among the 10 newest releases that satisfy `version` (pin `version` to go further back). On ARM macOS and Windows, x86-64 builds are a fallback: Rosetta 2 and the Windows emulation run them. Tokens the OS/arch matchers recognise (case-insensitive, bounded by `_` or a word boundary): diff --git a/skills/dload-fetch-tool/references/troubleshooting.md b/skills/dload-fetch-tool/references/troubleshooting.md index 4d107874..23111894 100644 --- a/skills/dload-fetch-tool/references/troubleshooting.md +++ b/skills/dload-fetch-tool/references/troubleshooting.md @@ -47,7 +47,7 @@ A candidate that matches neither is discarded. Common offenders: Compare with the host: `php -r "echo PHP_OS_FAMILY, ' / ', php_uname('m'), PHP_EOL;"`. -With a `binary`, OS packages (`.deb`, `.msi`… — the list is in [`registry-entry.md`](registry-entry.md#asset-pattern)) are discarded too, since dload cannot unpack them, and a release offering only packages is skipped for an older one among the 10 newest releases that satisfy `version`, with a warning naming both versions. When the package names carry the host OS and arch, the report says "only OS packages match" and lists them under "OS packages are skipped because a binary is expected"; a package without an OS token (`tool.dmg`) is discarded by the OS check above instead. When no older release fits, pin a `version` that ships an archive or a raw binary for the host — pinning also reaches past the 10 newest. +With a `binary`, OS packages (`.deb`, `.msi`… — the list is in [`registry-entry.md`](registry-entry.md#asset-pattern)) and archives of a format dload cannot unpack (known archive formats other than `.zip`, `.tar.gz`, `.tgz`, `.tar`, `.gz`, `.phar`, such as `.tar.xz`, `.7z`, `.zst`) are discarded too, and a release offering only such assets is skipped for an older one among the 10 newest releases that satisfy `version`, with a warning naming both versions. When their names carry the host OS and arch, the report says "only OS packages match" or "only archives of an unsupported format match" and lists them under "OS packages are skipped because a binary is expected" or "Archives of an unsupported format are skipped because a binary is expected"; one without an OS token (`tool.dmg`) is discarded by the OS check above instead. When no older release fits, pin a `version` that ships a supported archive or a raw binary for the host — pinning also reaches past the 10 newest. Several candidates left: dload tries them best first — host libc, then the fewest extra name tokens, then archives, then the release order. Run with `-vvv` to see the rank of every asset (`Asset ... ranked: os=0 arch=0 libc=0 extras=1 archive=0`); a wrong pick usually means the wanted build carries an extra token the others lack. diff --git a/src/Module/Archive/ArchiveFactory.php b/src/Module/Archive/ArchiveFactory.php index def5bede..78bf0bd7 100644 --- a/src/Module/Archive/ArchiveFactory.php +++ b/src/Module/Archive/ArchiveFactory.php @@ -122,11 +122,22 @@ private function bootDefaultMatchers(): void static fn(\SplFileInfo $info): Archive => new GzArchive($info), ), ['gz']); + // Registered after `gz`: the matchers added last are tried first. $this->extend($this->matcher( 'tar.gz', static fn(\SplFileInfo $info): Archive => new TarPharArchive($info), ), ['tar.gz']); + $this->extend($this->matcher( + 'tgz', + static fn(\SplFileInfo $info): Archive => new TarPharArchive($info), + ), ['tgz']); + + $this->extend($this->matcher( + 'tar', + static fn(\SplFileInfo $info): Archive => new TarPharArchive($info), + ), ['tar']); + $this->extend($this->matcher( 'phar', static fn(\SplFileInfo $info): Archive => new PharArchive($info), diff --git a/src/Module/Archive/Internal/TarPharArchive.php b/src/Module/Archive/Internal/TarPharArchive.php index fbc37839..5029b42b 100644 --- a/src/Module/Archive/Internal/TarPharArchive.php +++ b/src/Module/Archive/Internal/TarPharArchive.php @@ -5,7 +5,7 @@ namespace Internal\DLoad\Module\Archive\Internal; /** - * Archive handler for TAR.GZ archives + * Archive handler for TAR archives, plain or gzipped (`.tar`, `.tar.gz`, `.tgz`) * * ```php * // Extracting from TAR.GZ archive @@ -21,7 +21,7 @@ final class TarPharArchive extends PharAwareArchive { /** - * Opens TAR.GZ archive for reading + * Opens TAR archive for reading * * @param \SplFileInfo $file Archive file */ diff --git a/src/Module/Downloader/Downloader.php b/src/Module/Downloader/Downloader.php index 0e89961d..be037c53 100644 --- a/src/Module/Downloader/Downloader.php +++ b/src/Module/Downloader/Downloader.php @@ -22,6 +22,7 @@ use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Candidate; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\ArchitectureRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\PackageRule; +use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\UnsupportedArchiveRule; use Internal\DLoad\Module\Downloader\Internal\DownloadContext; use Internal\DLoad\Module\Downloader\Task\DownloadResult; use Internal\DLoad\Module\Downloader\Task\DownloadTask; @@ -179,6 +180,15 @@ public function download( ); } + /** + * @param list $candidates + * @return list + */ + private static function assetNames(array $candidates): array + { + return \array_map(static fn(Candidate $candidate): string => $candidate->asset->getName(), $candidates); + } + /** * Creates the repository of the current config. * @@ -214,7 +224,8 @@ private function createRepository(DownloadContext $context): ?Repository private function processRepository(Repository $repository, DownloadContext $context, bool $mayRetry = true): \Closure { return function () use ($repository, $context, $mayRetry): ReleaseInterface { - $context->packagesOnlyRelease = null; + $context->skippedRelease = null; + $context->skippedReleaseAssets = ''; // Set when a release turned out to be deleted: the release list is outdated then $forgotten = false; @@ -273,11 +284,12 @@ private function processRepository(Repository $repository, DownloadContext $cont await(coroutine($this->processRelease($context))); /** @var ReleaseInterface|null $skipped */ - $skipped = $context->packagesOnlyRelease; + $skipped = $context->skippedRelease; $skipped === null or $this->logger->warning( - '`%s` %s has only OS packages for `%s`/`%s`, downloaded %s instead.', + '`%s` %s has only %s for `%s`/`%s`, downloaded %s instead.', $context->software->getId(), $skipped->getVersion()->string, + $context->skippedReleaseAssets, $this->operatingSystem->value, $this->architecture->value, $context->release->getVersion()->string, @@ -349,18 +361,25 @@ private function processRelease(DownloadContext $context): \Closure ); $this->logger->debug('%d matching assets found.', \count($selection->candidates)); - $skippedPackages = \array_map( - static fn(Candidate $candidate): string => $candidate->asset->getName(), - $selection->removed[PackageRule::KEY] ?? [], - ); + $skippedPackages = self::assetNames($selection->removed[PackageRule::KEY] ?? []); + $skippedArchives = self::assetNames($selection->removed[UnsupportedArchiveRule::KEY] ?? []); $context->releaseAttempt->registerSkippedPackages($skippedPackages); + $context->releaseAttempt->registerSkippedArchives($skippedArchives); + + $skipped = \implode(' and ', [ + ...($skippedPackages === [] ? [] : ['OS packages']), + ...($skippedArchives === [] ? [] : ['archives of an unsupported format']), + ]); + if ($selection->isEmpty() && $skipped !== '' && $context->skippedRelease === null) { + $context->skippedRelease = $context->release; + $context->skippedReleaseAssets = $skipped; + } - $selection->isEmpty() && $skippedPackages !== [] and $context->packagesOnlyRelease ??= $context->release; $selection->isEmpty() and throw new NotFound( $strict ? \sprintf( '%s OS `%s`, architecture `%s`, name pattern `%s`%s', - $skippedPackages === [] ? 'no asset matches' : 'only OS packages match', + $skipped === '' ? 'no asset matches' : "only {$skipped} match", $this->operatingSystem->value, $this->architecture->value, $context->repoConfig->assetPattern, diff --git a/src/Module/Downloader/Internal/AssetSelection/AssetSelector.php b/src/Module/Downloader/Internal/AssetSelection/AssetSelector.php index 4acbaac6..fe6ab9bd 100644 --- a/src/Module/Downloader/Internal/AssetSelection/AssetSelector.php +++ b/src/Module/Downloader/Internal/AssetSelection/AssetSelector.php @@ -18,6 +18,7 @@ use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\NamePatternRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\OperatingSystemRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\PackageRule; +use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\UnsupportedArchiveRule; use Internal\DLoad\Module\Repository\AssetInterface; use Psr\Container\ContainerInterface; @@ -55,9 +56,10 @@ public function __construct( new CompanionRule(), new OperatingSystemRule($operatingSystem), new ArchitectureRule($architecture, $operatingSystem), - // After the platform rules, so the report names only packages built for the host. A package whose - // name carries no OS token, like `tool.dmg`, is already gone then and stays out of the report. + // After the platform rules, so the report names only packages and archives built for the host. One whose + // name carries no OS token, like `tool.dmg` or `tool.7z`, is gone by then and stays out of the report. new PackageRule(), + new UnsupportedArchiveRule($archiveFactory), new LibcRule($container), new ExtrasRule(), new ArchiveRule($archiveFactory), diff --git a/src/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRule.php b/src/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRule.php new file mode 100644 index 00000000..124af181 --- /dev/null +++ b/src/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRule.php @@ -0,0 +1,55 @@ +archiveFactory->getSupportedExtensions(); + + $selection->strict and $selection = $selection->remove( + static fn(Candidate $candidate): bool => $candidate->hasExtension(self::EXTENSIONS) + && !$candidate->hasExtension($supported), + self::KEY, + ); + + return $next($selection); + } +} diff --git a/src/Module/Downloader/Internal/Diagnostics/ReleaseAttempt.php b/src/Module/Downloader/Internal/Diagnostics/ReleaseAttempt.php index 8ea5225b..09a1bec0 100644 --- a/src/Module/Downloader/Internal/Diagnostics/ReleaseAttempt.php +++ b/src/Module/Downloader/Internal/Diagnostics/ReleaseAttempt.php @@ -26,6 +26,9 @@ final class ReleaseAttempt /** @var list Names of the OS packages for the host, skipped because a binary is expected */ private array $skippedPackages = []; + /** @var list Names of the archives for the host dload cannot unpack, skipped because a binary is expected */ + private array $skippedArchives = []; + /** @var list Errors occurred while downloading matched assets */ private array $failures = []; @@ -53,6 +56,15 @@ public function registerSkippedPackages(array $names): void $this->skippedPackages = $names; } + /** + * @param list $names Names of the archives for the host dload cannot unpack, + * skipped because a binary is expected + */ + public function registerSkippedArchives(array $names): void + { + $this->skippedArchives = $names; + } + /** * Registers a failure of a matched asset, e.g. a broken download. * @@ -95,6 +107,11 @@ public function describe(): array \implode(', ', $this->skippedPackages), ); + $this->skippedArchives === [] or $lines[] = \sprintf( + ' Archives of an unsupported format are skipped because a binary is expected: %s', + \implode(', ', $this->skippedArchives), + ); + foreach ($this->failures as $failure) { $lines[] = ' Failed asset ' . $failure; } diff --git a/src/Module/Downloader/Internal/DownloadContext.php b/src/Module/Downloader/Internal/DownloadContext.php index e10e23a5..a7d8bdc2 100644 --- a/src/Module/Downloader/Internal/DownloadContext.php +++ b/src/Module/Downloader/Internal/DownloadContext.php @@ -49,10 +49,13 @@ final class DownloadContext /** * @var ReleaseInterface|null Newest release of the current repository pass skipped because it has only - * OS packages for the host. Another repository or a refetched release list starts over, so a version - * found elsewhere is not reported against it. + * OS packages or archives of an unsupported format for the host. Another repository or a refetched + * release list starts over, so a version found elsewhere is not reported against it. */ - public ?ReleaseInterface $packagesOnlyRelease = null; + public ?ReleaseInterface $skippedRelease = null; + + /** @var string What {@see self::$skippedRelease} has for the host instead, like `OS packages` */ + public string $skippedReleaseAssets = ''; /** * Creates a new download context. diff --git a/tests/Integration/Module/Archive/ArchiveIntegrationTest.php b/tests/Integration/Module/Archive/ArchiveIntegrationTest.php index 2853ca8a..0097bdf5 100644 --- a/tests/Integration/Module/Archive/ArchiveIntegrationTest.php +++ b/tests/Integration/Module/Archive/ArchiveIntegrationTest.php @@ -44,6 +44,8 @@ public static function provideArchiveTypes(): \Generator { yield 'zip' => ['zip', ZipPharArchive::class]; yield 'tar.gz' => ['tar.gz', TarPharArchive::class]; + yield 'tgz' => ['tgz', TarPharArchive::class]; + yield 'tar' => ['tar', TarPharArchive::class]; yield 'phar' => ['phar', PharArchive::class]; yield 'exe' => ['exe', NullArchive::class]; } @@ -52,6 +54,8 @@ public static function provideNestedArchiveTypes(): \Generator { yield 'zip' => ['zip']; yield 'tar.gz' => ['tar.gz']; + yield 'tgz' => ['tgz']; + yield 'tar' => ['tar']; } #[DataProvider('provideArchiveTypes')] @@ -180,7 +184,7 @@ protected function cleanup(): void * and reopened within the same process reads its entries back as empty on Linux, so building * the fixture on the fly is unreliable across platforms. * - * @param non-empty-string $type Either `zip` or `tar.gz` + * @param non-empty-string $type One of `zip`, `tar.gz`, `tgz` or `tar` * @return non-empty-string Path to the fixture archive */ private function nestedArchiveFixture(string $type): string diff --git a/tests/Integration/Module/Archive/Fixture/nested.tar b/tests/Integration/Module/Archive/Fixture/nested.tar new file mode 100644 index 00000000..719b31c9 Binary files /dev/null and b/tests/Integration/Module/Archive/Fixture/nested.tar differ diff --git a/tests/Integration/Module/Archive/Fixture/nested.tgz b/tests/Integration/Module/Archive/Fixture/nested.tgz new file mode 100644 index 00000000..e0a680d2 Binary files /dev/null and b/tests/Integration/Module/Archive/Fixture/nested.tgz differ diff --git a/tests/Unit/Module/Archive/API/ArchiveFactoryTest.php b/tests/Unit/Module/Archive/API/ArchiveFactoryTest.php index 7ad2e383..11c3ea49 100644 --- a/tests/Unit/Module/Archive/API/ArchiveFactoryTest.php +++ b/tests/Unit/Module/Archive/API/ArchiveFactoryTest.php @@ -6,6 +6,7 @@ use Internal\DLoad\Module\Archive\Archive; use Internal\DLoad\Module\Archive\ArchiveFactory; +use Internal\DLoad\Module\Archive\Internal\GzArchive; use Internal\DLoad\Module\Archive\Internal\NullArchive; use Internal\DLoad\Module\Archive\Internal\PharArchive; use Internal\DLoad\Module\Archive\Internal\TarPharArchive; @@ -33,6 +34,8 @@ public static function provideDefaultSupportedExtensions(): \Generator { yield 'zip extension' => ['zip']; yield 'tar.gz extension' => ['tar.gz']; + yield 'tgz extension' => ['tgz']; + yield 'tar extension' => ['tar']; yield 'phar extension' => ['phar']; } @@ -43,6 +46,17 @@ public static function provideArchiveFiles(): \Generator yield 'phar file' => ['archive.phar', PharArchive::class]; } + public static function provideNamesWithOverlappingExtensions(): \Generator + { + yield 'tar.gz' => ['tool.tar.gz', TarPharArchive::class]; + yield 'tgz' => ['tool.tgz', TarPharArchive::class]; + yield 'uppercase tgz' => ['tool.TGZ', TarPharArchive::class]; + yield 'tar' => ['tool.tar', TarPharArchive::class]; + yield 'gz' => ['tool.gz', GzArchive::class]; + yield 'tar earlier in the name' => ['tool.tar.bin.gz', GzArchive::class]; + yield 'tar under another compression' => ['tool.tar.xz', NullArchive::class]; + } + #[BeforeClass] public static function prepareClass(): void { @@ -98,6 +112,15 @@ public function createReturnsCorrectArchiveTypeForExtension( Assert::instanceOf($archive, $expectedClass); } + #[DataProvider('provideNamesWithOverlappingExtensions')] + #[Test] + public function createPicksTheHandlerByTheWholeExtension(string $filename, string $expectedClass): void + { + $archive = $this->factory->create($this->createFileInfoMock($filename)); + + Assert::instanceOf($archive, $expectedClass); + } + #[Test] public function extendAddsCustomMatcher(): void { diff --git a/tests/Unit/Module/Archive/Internal/TarPharArchiveTest.php b/tests/Unit/Module/Archive/Internal/TarPharArchiveTest.php index 3df78cbf..24d927cf 100644 --- a/tests/Unit/Module/Archive/Internal/TarPharArchiveTest.php +++ b/tests/Unit/Module/Archive/Internal/TarPharArchiveTest.php @@ -5,13 +5,30 @@ namespace Internal\DLoad\Tests\Unit\Module\Archive\Internal; use Internal\DLoad\Module\Archive\Internal\TarPharArchive; +use Testo\Assert; use Testo\Codecov\Covers; +use Testo\Data\DataProvider; use Testo\Expect; +use Testo\Lifecycle\AfterTest; +use Testo\Lifecycle\BeforeTest; use Testo\Test; #[Covers(TarPharArchive::class)] final class TarPharArchiveTest { + /** A tar.gz with `pkg-1.0/bin/app` (`binary\n`), `pkg-1.0/lib/app/libphp.so` and `pkg-1.0/share/app/VERSION.txt`. */ + private const NESTED_TAR_GZ = __DIR__ . '/../../../../Integration/Module/Archive/Fixture/nested.tar.gz'; + + private string $workDir; + + public static function provideArchiveNames(): \Generator + { + yield 'tar.gz' => ['tool-linux-amd64.tar.gz']; + yield 'tgz' => ['tool-linux-amd64.tgz']; + yield 'uppercase tgz' => ['tool-linux-amd64.TGZ']; + yield 'tar' => ['tool-linux-amd64.tar']; + } + #[Test] public function constructorValidatesFile(): void { @@ -24,4 +41,63 @@ public function constructorValidatesFile(): void new TarPharArchive($file); } + + #[DataProvider('provideArchiveNames')] + #[Test] + public function entriesListsTheArchiveRelativePaths(string $archiveName): void + { + $entries = $this->tarArchive($archiveName)->entries(); + \sort($entries); + + Assert::same($entries, ['pkg-1.0/bin/app', 'pkg-1.0/lib/app/libphp.so', 'pkg-1.0/share/app/VERSION.txt']); + } + + #[DataProvider('provideArchiveNames')] + #[Test] + public function extractCopiesTheBinaryToTheDestinationSentBack(string $archiveName): void + { + $destination = $this->workDir . \DIRECTORY_SEPARATOR . 'app'; + + $extractor = $this->tarArchive($archiveName)->extract(); + while ($extractor->valid() && $extractor->key() !== 'pkg-1.0/bin/app') { + $extractor->next(); + } + $extractor->send(new \SplFileInfo($destination)); + + Assert::same(\file_get_contents($destination), "binary\n"); + } + + #[BeforeTest] + protected function prepare(): void + { + $this->workDir = \sys_get_temp_dir() . \DIRECTORY_SEPARATOR . 'dload-tar-' . \uniqid(); + \mkdir($this->workDir, 0777, true); + } + + #[AfterTest] + protected function cleanup(): void + { + foreach (\array_diff((array) \scandir($this->workDir), ['.', '..']) as $entry) { + \unlink($this->workDir . \DIRECTORY_SEPARATOR . $entry); + } + + \rmdir($this->workDir); + } + + /** + * Writes the fixture under the given name, decompressed for a plain tar. + * + * The bytes are copied rather than built with PharData: a tarball created and reopened within + * the same process reads its entries back as empty on Linux. + */ + private function tarArchive(string $archiveName): TarPharArchive + { + $content = (string) \file_get_contents(self::NESTED_TAR_GZ); + \str_ends_with(\strtolower($archiveName), '.tar') and $content = (string) \gzdecode($content); + + $archivePath = $this->workDir . \DIRECTORY_SEPARATOR . $archiveName; + \file_put_contents($archivePath, $content); + + return new TarPharArchive(new \SplFileInfo($archivePath)); + } } diff --git a/tests/Unit/Module/Downloader/DownloaderTest.php b/tests/Unit/Module/Downloader/DownloaderTest.php index 631562a6..66cd69fe 100644 --- a/tests/Unit/Module/Downloader/DownloaderTest.php +++ b/tests/Unit/Module/Downloader/DownloaderTest.php @@ -81,6 +81,15 @@ public static function providePackagesOnlyWarnings(): \Generator 'v3.2.0', '`rr` v3.5.0 has only OS packages for `linux`/`arm64`, downloaded v3.2.0 instead.', ]; + yield 'the newest skipped release is named, whatever it was skipped for' => [ + [self::repoWithReleases([ + 'v3.5.0' => ['roadrunner-3.5.0-linux-arm64.deb'], + 'v3.4.0' => ['roadrunner-3.4.0-linux-arm64.tar.xz'], + 'v3.2.0' => ['roadrunner-3.2.0-linux-arm64.tar.gz'], + ])], + 'v3.2.0', + '`rr` v3.5.0 has only OS packages for `linux`/`arm64`, downloaded v3.2.0 instead.', + ]; yield 'a failed binary is not a packages-only release' => [ [self::repoWithReleases( [ @@ -108,6 +117,14 @@ public static function providePackagesOnlyWarnings(): \Generator 'v3.4.0', null, ]; + yield 'an unsupported-archive-only release of another repository is forgotten' => [ + [ + self::repoWithReleases(['v3.4.0' => ['roadrunner-3.4.0-linux-arm64.tar.xz']]), + self::repoWithReleases(['v3.4.0' => ['roadrunner-3.4.0-linux-arm64.tar.gz']]), + ], + 'v3.4.0', + null, + ]; yield 'a packages-only release of an outdated list is forgotten' => [ new SequenceRepositoryFactoryStub([ self::repoWithReleases( @@ -342,10 +359,59 @@ public function packagesOnlyWarningNamesOnlyAReleaseSkippedForItsPackages( Assert::same($result->version->string, $version); $warning === null - ? Assert::string($output->fetch())->notContains('has only OS packages') + ? Assert::string($output->fetch())->notContains(' has only ') : Assert::string($output->fetch())->contains($warning); } + #[Test] + public function aReleaseWithOnlyAnUnsupportedArchiveForTheHostGivesWayToAnOlderOne(): void + { + $repository = self::repoWithReleases([ + 'v3.4.0' => ['roadrunner-3.4.0-linux-amd64.tar.xz', 'roadrunner-3.4.0-darwin-amd64.tar.gz'], + 'v3.2.0' => ['roadrunner-3.2.0-linux-amd64.tar.gz'], + ]); + $output = new BufferedOutput(); + + $result = $this->run([$repository], self::software(binary: true), logger: new Logger($output)); + + Assert::same($result->version->string, 'v3.2.0'); + Assert::same($result->file->getFilename(), 'roadrunner-3.2.0-linux-amd64.tar.gz'); + Assert::string($output->fetch()) + ->contains('`rr` v3.4.0 has only archives of an unsupported format for `linux`/`amd64`, downloaded v3.2.0 instead.'); + } + + #[Test] + public function skippedArchivesAreNamedInTheReport(): void + { + $repository = self::repoWithReleases([ + 'v3.4.0' => ['roadrunner-3.4.0-linux-amd64.tar.xz', 'roadrunner-3.4.0-linux-amd64.deb', 'roadrunner-3.4.0-darwin-amd64.7z'], + ]); + + try { + $this->run([$repository], self::software(binary: true)); + Assert::fail('DownloadFailed is expected when the release has only unsupported archives for the host.'); + } catch (DownloadFailed $e) { + Assert::string($e->report) + ->contains('v3.4.0: 3 asset(s), only OS packages and archives of an unsupported format match OS `linux`, architecture `amd64`') + ->contains('OS packages are skipped because a binary is expected: roadrunner-3.4.0-linux-amd64.deb') + ->contains('Archives of an unsupported format are skipped because a binary is expected: roadrunner-3.4.0-linux-amd64.tar.xz'); + } + } + + #[Test] + public function withoutABinaryTheUnsupportedArchiveIsStillChosen(): void + { + $repository = self::repoWithReleases([ + 'v3.4.0' => ['roadrunner-3.4.0-linux-amd64.tar.xz'], + 'v3.2.0' => ['roadrunner-3.2.0-linux-amd64.tar.gz'], + ]); + + $result = $this->run([$repository], self::software()); + + Assert::same($result->version->string, 'v3.4.0'); + Assert::same($result->file->getFilename(), 'roadrunner-3.4.0-linux-amd64.tar.xz'); + } + #[Test] public function withoutABinaryThePackageIsStillChosen(): void { diff --git a/tests/Unit/Module/Downloader/Internal/AssetSelection/AssetSelectorTest.php b/tests/Unit/Module/Downloader/Internal/AssetSelection/AssetSelectorTest.php index beac7e58..d8341128 100644 --- a/tests/Unit/Module/Downloader/Internal/AssetSelection/AssetSelectorTest.php +++ b/tests/Unit/Module/Downloader/Internal/AssetSelection/AssetSelectorTest.php @@ -11,6 +11,7 @@ use Internal\DLoad\Module\Common\OperatingSystem; use Internal\DLoad\Module\Config\Schema\Action\Type; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\AssetSelector; +use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Candidate; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\ArchitectureRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\ArchiveRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\CompanionRule; @@ -20,9 +21,11 @@ use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\NamePatternRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\OperatingSystemRule; use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\PackageRule; +use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Rule\UnsupportedArchiveRule; +use Internal\DLoad\Module\Downloader\Internal\AssetSelection\Selection; use Internal\DLoad\Module\Repository\AssetInterface; -use Internal\DLoad\Tests\Unit\Module\Downloader\Internal\AssetSelection\Stub\NamedAssets; use Internal\DLoad\Tests\Unit\Module\Downloader\Internal\AssetSelection\Stub\LibcContainer; +use Internal\DLoad\Tests\Unit\Module\Downloader\Internal\AssetSelection\Stub\NamedAssets; use Testo\Assert; use Testo\Codecov\Covers; use Testo\Test; @@ -31,6 +34,7 @@ #[Covers(NamePatternRule::class)] #[Covers(FormatRule::class)] #[Covers(PackageRule::class)] +#[Covers(UnsupportedArchiveRule::class)] #[Covers(CompanionRule::class)] #[Covers(OperatingSystemRule::class)] #[Covers(ArchitectureRule::class)] @@ -221,6 +225,50 @@ public function archivesAndPlainBinariesOutliveOsPackages(): void Assert::same($names, ['tool-linux-amd64.tar.gz', 'tool-linux-amd64']); } + #[Test] + public function anArchiveDloadCannotUnpackGivesWayToAPlainBinary(): void + { + $names = self::select(['tool-linux-amd64.tar.xz', 'tool-linux-amd64.AppImage', 'tool-linux-amd64']); + + Assert::same($names, ['tool-linux-amd64.AppImage', 'tool-linux-amd64']); + } + + #[Test] + public function onlyAssetsForTheHostAreReportedAsSkipped(): void + { + $selection = (new AssetSelector(OperatingSystem::Linux, Architecture::X86_64, new LibcContainer(Libc::Gnu), new ArchiveFactory())) + ->select(NamedAssets::create('tool-darwin-amd64.tar.xz', 'tool-linux-amd64.tar.xz', 'tool-linux-amd64.deb'), '/.*/', null, true); + + Assert::true($selection->isEmpty()); + Assert::same(self::removed($selection, UnsupportedArchiveRule::KEY), ['tool-linux-amd64.tar.xz']); + Assert::same(self::removed($selection, PackageRule::KEY), ['tool-linux-amd64.deb']); + } + + #[Test] + public function withoutABinaryAnUnsupportedArchiveIsOnlyRankedLower(): void + { + $names = self::select(['tool-linux-amd64.tar.xz', 'tool-linux-amd64.zip'], strict: false); + + Assert::same($names, ['tool-linux-amd64.zip', 'tool-linux-amd64.tar.xz']); + } + + #[Test] + public function aPharActionIsNotAffectedByUnsupportedArchives(): void + { + $names = self::select(['tool.tar.xz', 'tool.phar'], type: Type::Phar, strict: false); + + Assert::same($names, ['tool.phar']); + } + + /** + * @param non-empty-string $reason + * @return list + */ + private static function removed(Selection $selection, string $reason): array + { + return \array_map(static fn(Candidate $candidate): string => $candidate->asset->getName(), $selection->removed[$reason] ?? []); + } + /** * @param list $assets * @param non-empty-string $pattern diff --git a/tests/Unit/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRuleTest.php b/tests/Unit/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRuleTest.php new file mode 100644 index 00000000..8c6f4b56 --- /dev/null +++ b/tests/Unit/Module/Downloader/Internal/AssetSelection/Rule/UnsupportedArchiveRuleTest.php @@ -0,0 +1,94 @@ +removed), [UnsupportedArchiveRule::KEY]); + Assert::same( + \array_map( + static fn(Candidate $candidate): string => $candidate->asset->getName(), + $selection->removed[UnsupportedArchiveRule::KEY], + ), + self::UNSUPPORTED, + ); + } + + #[Test] + public function archivesAreKeptWhenNoBinaryIsExpected(): void + { + $selection = RuleRunner::run(self::rule(), [...self::OTHERS, ...self::UNSUPPORTED], strict: false); + + Assert::same(RuleRunner::names($selection), [...self::OTHERS, ...self::UNSUPPORTED]); + Assert::same($selection->removed, []); + } + + #[Test] + public function anArchiveActionWithABinaryLosesThemToo(): void + { + $selection = RuleRunner::run(self::rule(), ['tool-linux-amd64.tar.gz', 'tool-linux-amd64.tar.xz'], type: Type::Archive); + + Assert::same(RuleRunner::names($selection), ['tool-linux-amd64.tar.gz']); + } + + #[Test] + public function aFormatGainingAnExtractorIsKept(): void + { + $factory = new ArchiveFactory(); + $factory->extend(static fn(): null => null, ['tar.xz']); + + $selection = RuleRunner::run(new UnsupportedArchiveRule($factory), ['tool-linux-amd64.tar.xz', 'tool-linux-amd64.7z']); + + Assert::same(RuleRunner::names($selection), ['tool-linux-amd64.tar.xz']); + } + + private static function rule(): UnsupportedArchiveRule + { + return new UnsupportedArchiveRule(new ArchiveFactory()); + } +} diff --git a/tests/Unit/Module/Downloader/Internal/Diagnostics/DownloadDiagnosticsTest.php b/tests/Unit/Module/Downloader/Internal/Diagnostics/DownloadDiagnosticsTest.php index ab0adc1b..4c682b54 100644 --- a/tests/Unit/Module/Downloader/Internal/Diagnostics/DownloadDiagnosticsTest.php +++ b/tests/Unit/Module/Downloader/Internal/Diagnostics/DownloadDiagnosticsTest.php @@ -114,7 +114,27 @@ public function reportOmitsTheSkippedPackagesLineWhenNoneAreSkipped(): void $repository->matchedReleases = 1; $repository->addRelease('v1.2.0')->registerAssets(['app-1.2.0-darwin-arm64.tar.gz']); - Assert::string($diagnostics->render())->notContains('OS packages are skipped'); + Assert::string($diagnostics->render()) + ->notContains('OS packages are skipped') + ->notContains('Archives of an unsupported format are skipped'); + } + + #[Test] + public function reportNamesTheSkippedArchivesAfterThePackages(): void + { + $diagnostics = self::diagnostics(); + $repository = $diagnostics->addRepository('github', 'owner/repo', '/^app-.*/'); + $repository->matchedReleases = 1; + + $release = $repository->addRelease('v1.2.0'); + $release->registerAssets(['app-1.2.0-linux-amd64.deb', 'app-1.2.0-linux-amd64.tar.xz', 'app-1.2.0-linux-amd64.7z']); + $release->registerSkippedPackages(['app-1.2.0-linux-amd64.deb']); + $release->registerSkippedArchives(['app-1.2.0-linux-amd64.tar.xz', 'app-1.2.0-linux-amd64.7z']); + + Assert::same(\array_slice($release->describe(), 2), [ + ' OS packages are skipped because a binary is expected: app-1.2.0-linux-amd64.deb', + ' Archives of an unsupported format are skipped because a binary is expected: app-1.2.0-linux-amd64.tar.xz, app-1.2.0-linux-amd64.7z', + ]); } #[Test] diff --git a/tests/Unit/Module/Installer/InstallerTest.php b/tests/Unit/Module/Installer/InstallerTest.php index 9867999c..d126edff 100644 --- a/tests/Unit/Module/Installer/InstallerTest.php +++ b/tests/Unit/Module/Installer/InstallerTest.php @@ -26,6 +26,7 @@ use Symfony\Component\Console\Output\OutputInterface; use Testo\Assert; use Testo\Codecov\Covers; +use Testo\Data\DataProvider; use Testo\Expect; use Testo\Lifecycle\AfterTest; use Testo\Lifecycle\BeforeTest; @@ -45,6 +46,14 @@ final class InstallerTest private Path $destination; private BufferedOutput $log; + public static function provideNestedArchives(): \Generator + { + yield 'zip' => [self::NESTED_ZIP]; + yield 'tar.gz' => [__DIR__ . '/../../../Integration/Module/Archive/Fixture/nested.tar.gz']; + yield 'tgz' => [__DIR__ . '/../../../Integration/Module/Archive/Fixture/nested.tgz']; + yield 'tar' => [__DIR__ . '/../../../Integration/Module/Archive/Fixture/nested.tar']; + } + #[Test] public function pharIsMovedIntoTheDestinationAsIs(): void { @@ -190,10 +199,11 @@ public function pharWithoutExtractionRulesIsStillInstalledAsAPhar(): void Assert::string($this->log->fetch())->contains('as a PHAR archive'); } + #[DataProvider('provideNestedArchives')] #[Test] - public function binaryIsExtractedUnderItsNameAndLocated(): void + public function binaryIsExtractedUnderItsNameAndLocated(string $fixture): void { - $download = $this->downloadOf(self::NESTED_ZIP); + $download = $this->downloadOf($fixture); $software = Software::fromArray(['name' => 'App', 'binary' => ['name' => 'app']]); $binary = \Mockery::mock(Binary::class); @@ -208,6 +218,7 @@ public function binaryIsExtractedUnderItsNameAndLocated(): void $result = $this->installer($binaryProvider)->install($download, $software, null, $this->destination, removeDownload: true); Assert::same(self::paths($result->files), [(string) $this->destination->join('app')]); + Assert::same(\file_get_contents((string) $this->destination->join('app')), "binary\n"); Assert::same($result->binary, $binary); }