Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion internal/dist/cmd/errdetect/diagnose.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ func Diagnose(err error) *diagnostic.HelpfulError {
return help(err, "registry-packages-proxy: forbidden (403)",
"the identity may not download the CLI",
"bind the ClusterRole 'd8:registry-packages-proxy:cli-download' to the user/group",
"authorization is cached ~5 min - after binding, retry with a fresh token")
"a denied check is cached ~30s - after binding the role, wait half a minute and retry with the same token")
case errors.Is(err, rpp.ErrNotFound):
return help(err, "registry-packages-proxy: version not found (404)",
"this deckhouse-cli version is not published",
Expand Down
19 changes: 16 additions & 3 deletions internal/mirror/README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,8 @@ Registry layout (at the **bare root**, outside the edition segment - like the in
| `<root>/deckhouse-cli/plugins` | Plugin catalog; its tags are plugin names |
| `<root>/deckhouse-cli/plugins/<name>:<vX.Y.Z>` | One plugin version - a multi-platform OCI index (linux/darwin/windows) |

`<root>` is the registry path **without** the edition: `registry.deckhouse.io/deckhouse/deckhouse-cli/plugins`, never `registry.deckhouse.io/deckhouse/ee/deckhouse-cli/plugins`. Pull derives it by cutting a trailing edition segment (`ce`, `be`, `se`, `se-plus`, `ee`, `fe`) off `--source`, and the in-cluster registry-packages-proxy derives it the same way from the cluster's `imagesRepo`. Push applies the same rule to its target, so the two sides agree: `d8 mirror push ./bundle REGISTRY/dkp/ee` writes the platform, modules, packages and security databases under `REGISTRY/dkp/ee` and the plugins to `REGISTRY/dkp/deckhouse-cli/plugins`. A target without an edition segment (`registry.company.com/deckhouse`, `dev-registry.deckhouse.io/sys/deckhouse-oss`) is used as is. The push log says where plugins go before the first plugin upload.

**Bundle output:** one `plugin-<name>.tar` per plugin. Multi-platform indexes are stored whole, so every platform binary and the contract annotation reach the target registry exactly as published.

What is checked at mirror time vs install time:
Expand All @@ -419,7 +421,9 @@ What is checked at mirror time vs install time:
- `kubernetes` and `noneOf` requirements are cluster-side: `d8 plugins install` enforces them on the target cluster as usual.
- Conditional requirements never gate mirroring; conflicts show up as warnings.

After `d8 mirror push`, `d8 plugins install <name>` works in the air-gapped cluster through the registry-packages-proxy. Note the proxy serves plugins **by exact name** - listing the plugin catalog through it is not supported (a registry-packages-proxy limitation, not a bundle one; `crane ls <target>/deckhouse-cli/plugins` shows the names).
After `d8 mirror push`, `d8 plugins install <name>` works in the air-gapped cluster through the registry-packages-proxy. Note the proxy serves plugins **by exact name** - listing the plugin catalog through it is not supported (a registry-packages-proxy limitation, not a bundle one; `crane ls <target-without-edition>/deckhouse-cli/plugins` shows the names, e.g. `crane ls REGISTRY/dkp/deckhouse-cli/plugins` after a push to `REGISTRY/dkp/ee`).

Because plugins are written one level above the target, the push credentials must be allowed to write there too. When the registry answers HTTP 401/403 for `<root>/deckhouse-cli/...` while the target itself is writable, the push fails at the first plugin write - the pre-push access check covers the target only, so the components under it are uploaded first - with a diagnostic that names the refused repository, the target it sits above and the edition that was cut off; grant the account push access to `<root>/deckhouse-cli` (or create that repository, on registries that need repositories created up front), or push the bundle without `plugin-<name>.tar` archives via `--file`.

If the source registry has no `deckhouse-cli/plugins` catalog, or denies access to it (HTTP 401/403 - token-auth registries answer this way for any path outside the license's scope, published or not), the automatic selection is skipped quietly. Explicit `--include-plugin` entries still resolve against their own repositories, and a denied explicit include fails the pull.

Expand Down Expand Up @@ -491,7 +495,9 @@ When `--modules-path-suffix` moves modules off the default `modules/` and module

Uploads a previously downloaded Deckhouse Kubernetes Platform distribution bundle to a third-party container registry. This is typically used to populate an air-gapped registry with Deckhouse images.

Before uploading, push runs a write-access check against the target repository (15-second timeout, or `D8_MIRROR_TIMEOUT`); set `MIRROR_BYPASS_ACCESS_CHECKS=1` to proceed even if that check fails. Chunked packages (`<name>.tar.NNNN.chunk`) are reassembled transparently, and each image upload is retried up to 4 times. After the layouts are pushed, small discovery-index tags are created at `<repo>/modules:<module>` and `<repo>/packages:<package>` so the mirrored modules and packages can be enumerated by tag listing.
Before uploading, push runs a write-access check against the target repository (15-second timeout, or `D8_MIRROR_TIMEOUT`); set `MIRROR_BYPASS_ACCESS_CHECKS=1` to proceed even if that check fails. Chunked packages (`<name>.tar.NNNN.chunk`) are reassembled transparently, and each image upload is retried up to 4 times. After the layouts are pushed, small discovery-index tags are created at `<repo>/modules:<module>`, `<repo>/packages:<package>` and `<root>/deckhouse-cli/plugins:<plugin>` so the mirrored modules, packages and plugins can be enumerated by tag listing.

Everything in the bundle is written under the target as is, with one exception: d8 CLI plugins go to the registry root **above** the target's edition segment, `REGISTRY/dkp/ee` -> `REGISTRY/dkp/deckhouse-cli/plugins`, because that is where pull reads them from and where the in-cluster registry-packages-proxy looks for them (it cuts the edition off the cluster's `imagesRepo` the same way). A target without an edition segment is used as is. See [Plugin Mirroring](#plugin-mirroring) for the rule and the access rights it needs.

### Synopsis

Expand Down Expand Up @@ -553,11 +559,18 @@ d8 mirror push /tmp/d8-bundle registry.company.com/deckhouse \
--registry-login admin \
--registry-password secretpassword \
--tmp-dir /mnt/large-disk/tmp

# Push to an edition-suffixed target (the DKP docs layout): the platform,
# modules, packages and security databases go under registry.company.com/dkp/ee,
# the CLI plugins to registry.company.com/dkp/deckhouse-cli/plugins
d8 mirror push /tmp/d8-bundle registry.company.com/dkp/ee \
--registry-login admin \
--registry-password secretpassword
```

### Summary

After a push, `d8 mirror` prints a framed summary of what was written to the target registry (platform, installer, security databases, module and package counts).
After a push, `d8 mirror` prints a framed summary of what was written to the target registry (platform, installer, security databases, module, package and plugin counts).

When `--modules-path-suffix` moves modules off the default `modules/` and modules were actually pushed, the summary warns about the non-default modules path, highlighting it with a hint of the standard path. A default path, or a moved path that no module went through, produces no warning.

Expand Down
53 changes: 53 additions & 0 deletions internal/mirror/cmd/push/errdetect/diagnose.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,13 @@ import (
"net"
"net/http"
"os"
"path"
"syscall"

"github.com/google/go-containerregistry/pkg/v1/remote/transport"

"github.com/deckhouse/deckhouse-cli/internal"
"github.com/deckhouse/deckhouse-cli/internal/mirror"
"github.com/deckhouse/deckhouse-cli/internal/mirror/errmatch"
"github.com/deckhouse/deckhouse-cli/pkg/diagnostic"
)
Expand All @@ -41,6 +44,7 @@ const (
categoryAuth = "Authentication failed"
categoryAuth401 = "Authentication failed (HTTP 401 Unauthorized)"
categoryAuth403 = "Access denied (HTTP 403 Forbidden)"
categoryPluginsRoot = "Access denied to the CLI plugins path above the target"
categoryRateLimit = "Rate limited by registry (HTTP 429 Too Many Requests)"
categoryServerError = "Registry server error"
categoryDNS = "DNS resolution failed"
Expand Down Expand Up @@ -123,6 +127,11 @@ func Diagnose(err error) *diagnostic.HelpfulError {
},
}

// A denied write to the plugins root names a path the user never typed:
// spell that path out instead of giving the generic credentials advice.
case isPluginsRootAuthError(err):
return diagnosePluginsRootDenied(err)

case isAuthenticationError(err):
category := categoryAuth
if code := authStatusCode(err); code == http.StatusUnauthorized {
Expand Down Expand Up @@ -346,8 +355,52 @@ func Diagnose(err error) *diagnostic.HelpfulError {
return nil
}

// diagnosePluginsRootDenied explains a 401/403 on the CLI plugins root: where
// the write went, why that path is above the target, and how to unblock it.
func diagnosePluginsRootDenied(err error) *diagnostic.HelpfulError {
var rootErr *mirror.PluginsRootError
errors.As(err, &rootErr)

report := rootErr.Report

category := categoryPluginsRoot
if code := authStatusCode(err); code != 0 {
category = fmt.Sprintf("%s (HTTP %d)", categoryPluginsRoot, code)
}

pluginsRepo := path.Join(report.Root, internal.D8CLISegment)

return &diagnostic.HelpfulError{
Category: category,
OriginalErr: err,
Suggestions: []diagnostic.Suggestion{
{
Cause: fmt.Sprintf(
"d8 CLI plugins are written to %s, the registry root above the %q edition of the target %s "+
"(registry-packages-proxy in the cluster looks for them there). "+
"The registry refused the write to %s: the credentials cover only the target path, or the registry has no repository at that level",
report.Path, report.Edition, report.Target, rootErr.Repo,
),
Solutions: []string{
fmt.Sprintf("Grant the account push access to %s and its sub-repositories", pluginsRepo),
fmt.Sprintf("If the registry needs repositories (projects) created up front, create %s next to %s", pluginsRepo, report.Target),
"To push the bundle without plugins, pass the archives with --file and leave out plugin-<name>.tar",
},
},
},
}
}

// --- detection functions ---

// isPluginsRootAuthError reports a 401/403 on a write to the CLI plugins
// root above the target (see mirror.PluginsRootError).
func isPluginsRootAuthError(err error) bool {
var rootErr *mirror.PluginsRootError

return errors.As(err, &rootErr) && isAuthenticationError(err)
}

func isEOF(err error) bool {
return errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF)
}
Expand Down
59 changes: 59 additions & 0 deletions internal/mirror/cmd/push/errdetect/diagnose_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"github.com/deckhouse/deckhouse-cli/internal/mirror"
"github.com/deckhouse/deckhouse-cli/pkg/diagnostic"
)

Expand Down Expand Up @@ -86,6 +87,64 @@ func TestDiagnose_PushSpecificAuth(t *testing.T) {
assert.NotContains(t, solutions, "--source-login")
}

// pluginsRootErr wraps a registry status the way push does for a failed write
// above the edition: the status sits inside the typed error's chain.
func pluginsRootErr(status int) error {
return fmt.Errorf("push to registry: %w", &mirror.PluginsRootError{
Repo: "registry.example.com/dkp/deckhouse-cli/plugins/postgresql-mgr",
Report: mirror.PluginsPathReport{
Edition: "ee",
Target: "registry.example.com/dkp/ee",
Root: "registry.example.com/dkp",
Path: "registry.example.com/dkp/deckhouse-cli/plugins",
},
Err: fmt.Errorf("push layout: %w", &transport.Error{StatusCode: status}),
})
}

// TestDiagnose_PluginsRootDenied checks that a 401/403 on the plugins root
// above the edition is explained in terms of the paths: where the write went,
// which target it sits above, and what to grant. The generic auth advice
// (check --registry-login) is wrong here: the credentials do work for the
// target.
func TestDiagnose_PluginsRootDenied(t *testing.T) {
for _, status := range []int{http.StatusUnauthorized, http.StatusForbidden} {
t.Run(http.StatusText(status), func(t *testing.T) {
diag := Diagnose(pluginsRootErr(status))
require.NotNil(t, diag)

assert.Contains(t, diag.Category, categoryPluginsRoot)
assert.Contains(t, diag.Category, fmt.Sprintf("HTTP %d", status))
assert.NotContains(t, diag.Category, categoryAuth)

require.NotEmpty(t, diag.Suggestions)
cause := diag.Suggestions[0].Cause
assert.Contains(t, cause, "registry.example.com/dkp/deckhouse-cli/plugins")
assert.Contains(t, cause, `"ee" edition`)
assert.Contains(t, cause, "registry.example.com/dkp/ee")
assert.Contains(t, cause, "registry.example.com/dkp/deckhouse-cli/plugins/postgresql-mgr")

solutions := allSolutions(diag)
assert.Contains(t, solutions, "registry.example.com/dkp/deckhouse-cli")
assert.Contains(t, solutions, "--file")
assert.NotContains(t, solutions, "--registry-login")

var transportErr *transport.Error
assert.True(t, errors.As(diag, &transportErr), "the registry status must stay reachable")
})
}
}

// TestDiagnose_PluginsRootNonAuthFallsThrough checks that the plugins-root
// wrapper changes nothing for errors that are not about access: a 500 on the
// plugins path is still a server error.
func TestDiagnose_PluginsRootNonAuthFallsThrough(t *testing.T) {
diag := Diagnose(pluginsRootErr(http.StatusInternalServerError))
require.NotNil(t, diag)
assert.Contains(t, diag.Category, categoryServerError)
assert.NotContains(t, diag.Category, categoryPluginsRoot)
}

func TestDiagnose_DiskFull(t *testing.T) {
diag := Diagnose(fmt.Errorf("write temp: %w", syscall.ENOSPC))
require.NotNil(t, diag)
Expand Down
16 changes: 9 additions & 7 deletions internal/mirror/cmd/push/push.go
Original file line number Diff line number Diff line change
Expand Up @@ -244,8 +244,11 @@ func (p *Pusher) warnNonDefaultModulesPath() {

// executeNewPush runs the push using the push service.
// This service expects the bundle to have the exact same structure as the registry:
// - Each OCI layout's relative path becomes its registry segment
// - Works with unified bundles where pull saved the structure as-is
// - Each OCI layout's relative path becomes its registry segment
// - Works with unified bundles where pull saved the structure as-is
// - CLI plugins (deckhouse-cli/) are the exception: they go to the registry
// root above the target's edition segment, e.g. REGISTRY/dkp/ee pushes them
// to REGISTRY/dkp/deckhouse-cli/plugins
func (p *Pusher) executeNewPush() error {
// Set up graceful cancellation on Ctrl+C
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
Expand All @@ -268,16 +271,15 @@ func (p *Pusher) executeNewPush() error {
clientOpts = append(clientOpts, regclient.WithTimeout(MirrorTimeout))
}

// The client is scoped to the host only. The service scopes the target
// path itself: it needs the path to place CLI plugins at the registry
// root above the target's edition segment.
client := pkgclient.NewFromOptions(p.pushParams.RegistryHost, clientOpts...)

// Scope to the registry path
if p.pushParams.RegistryPath != "" {
client = client.WithSegment(p.pushParams.RegistryPath)
}

svc := mirror.NewPushService(
client,
&mirror.PushServiceOptions{
TargetPath: p.pushParams.RegistryPath,
Packages: Packages,
WorkingDir: p.pushParams.WorkingDir,
ModulesPathSuffix: p.pushParams.ModulesPathSuffix,
Expand Down
121 changes: 121 additions & 0 deletions internal/mirror/plugins_path.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
/*
Copyright 2026 Flant JSC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package mirror

import (
"fmt"
"path"

"github.com/deckhouse/deckhouse-cli/internal"
"github.com/deckhouse/deckhouse-cli/pkg"
pkgclient "github.com/deckhouse/deckhouse-cli/pkg/registry/client"
registryservice "github.com/deckhouse/deckhouse-cli/pkg/registry/service"
)

// PluginsPathReport describes where d8 CLI plugins are written by push.
//
// CLI artifacts live at the registry root above the edition segment:
// registry.example.com/deckhouse/deckhouse-cli/plugins, never
// registry.example.com/deckhouse/ee/deckhouse-cli/plugins. Pull reads them
// from there, and the in-cluster registry-packages-proxy looks them up there
// by cutting the edition off the cluster's imagesRepo. Push applies the same
// rule to its target, so the other components (platform, modules, packages,
// installer, security) stay under the target while plugins go one level up.
type PluginsPathReport struct {
// Edition is the edition segment cut from the end of the push target,
// e.g. "ee" for a target ".../deckhouse/ee". Empty when the target has
// no edition segment, in which case Root equals Target.
Edition string
// Target is the push target as given by the user, e.g.
// "registry.example.com/deckhouse/ee".
Target string
// Root is the registry root plugins are written under: Target without
// Edition, e.g. "registry.example.com/deckhouse".
Root string
// Path is the plugins catalog, "<Root>/deckhouse-cli/plugins". Plugin
// images go to "<Path>/<name>", discovery tags to Path itself.
Path string
}

// pluginsRootPath returns the target path CLI artifacts are written under:
// targetPath without its trailing edition segment, plus that segment.
//
// The root always keeps at least one path segment. A target like
// "registry.example.com/ee" names a project that happens to be called "ee",
// not an edition of a Deckhouse repository, so it is left alone. The
// registry-packages-proxy applies the same rule when it looks the artifacts
// up, and both sides must agree on the result.
func pluginsRootPath(targetPath string) (string, string) {
root, edition := registryservice.GetEditionFromRegistryPath(targetPath)
if edition == pkg.NoEdition {
return targetPath, ""
}

if len(pkgclient.PathToSegments(root)) == 0 {
return targetPath, ""
}

return root, edition.String()
}

func newPluginsPathReport(target, root, edition string) PluginsPathReport {
return PluginsPathReport{
Edition: edition,
Target: target,
Root: root,
Path: path.Join(root, internal.D8CLISegment, internal.D8PluginsSegment),
}
}

// Moved reports whether plugins go above the push target, i.e. an edition
// segment was cut off it.
func (p PluginsPathReport) Moved() bool {
return p.Edition != ""
}

// Notice is the single-line form for the push log, printed before the first
// plugin write so the user sees where plugins go and why it is not the target.
func (p PluginsPathReport) Notice() string {
if !p.Moved() {
return fmt.Sprintf("CLI plugins go to %s", p.Path)
}

return fmt.Sprintf("CLI plugins go to %s (registry root above the %q edition of %s; registry-packages-proxy looks for them there)",
p.Path, p.Edition, p.Target)
}

// PluginsRootError is a failed write to the CLI plugins root when that root
// sits above the push target (see PluginsPathReport). It carries the paths so
// the diagnostic can say where the write went and why, instead of leaving the
// user with a bare registry error for a path they never typed.
type PluginsRootError struct {
// Repo is the repository the write went to, e.g.
// "registry.example.com/deckhouse/deckhouse-cli/plugins/postgresql-mgr".
Repo string
// Report describes the plugins root relative to the push target.
Report PluginsPathReport
// Err is the underlying push error.
Err error
}

func (e *PluginsRootError) Error() string {
return e.Err.Error()
}

func (e *PluginsRootError) Unwrap() error {
return e.Err
}
Loading
Loading