Add docs

docs/antora.yml

@@ -0,0 +1,6 @@
+---
+# Cockpit is a module of the 'home' component, like the operators.
+# The release scripts set the version to YY.M and prerelease to false on release branches.
+name: home
+version: 'nightly'
+prerelease: true

docs/modules/cockpit/pages/getting-started.adoc

@@ -0,0 +1,59 @@
+= Getting started
+:description: Install Stackable Cockpit on Kubernetes with Helm, connect it to an OIDC provider, PostgreSQL and Trino, and run your first SQL query.
+
+This guide installs Cockpit with Helm and runs a first Trino query.
+You need:
+
+* a PostgreSQL database,
+* an OIDC client for Cockpit with the redirect URI `https://cockpit.example.com/api/auth/callback/oidc`, see the xref:installation/authentication.adoc[authentication] page for more details,
+* a Trino cluster,
+* the xref:secret-operator:index.adoc[Stackable Secret Operator],
+* an Ingress or reverse proxy that serves `https://cockpit.example.com`.
+
+Create a Secret with the database password and the OIDC client secret:
+
+[source,console]
+----
+$ kubectl create secret generic cockpit \
+  --from-literal=database-password='<database password>' \
+  --from-literal=oidc-client-secret='<OIDC client secret>'
+----
+
+Create a values file `cockpit-values.yaml`:
+
+[source,yaml]
+----
+origin: https://cockpit.example.com # <1>
+auth:
+  baseUrl: https://cockpit.example.com
+  oidc:
+    discoveryUrl: https://keycloak.example.com/realms/sdp/.well-known/openid-configuration
+    clientId: cockpit
+    clientSecret:
+      secretKeyRef:
+        name: cockpit
+database:
+  host: postgresql
+  password:
+    secretKeyRef:
+      name: cockpit
+  tls:
+    secretClass: tls # <2>
+trino:
+  url: https://trino-coordinator-default-headless:8443 # <3>
+  tls:
+    secretClass: tls # <4>
+----
+<1> The URL users open in the browser. It must use HTTPS, see xref:installation/helm.adoc#_ingress_and_reverse_proxy[Ingress and reverse proxy].
+<2> The SecretClass that provides the CA certificate of PostgreSQL. Cockpit verifies the certificate by default, see xref:installation/database.adoc[].
+<3> Optional. Without `trino.url`, each user enters a Trino connection in Cockpit, see xref:modules/trino.adoc#_connection[Connection].
+<4> The SecretClass that provides the CA certificate of Trino.
+
+Install the chart:
+
+[source,console,subs="attributes"]
+----
+$ helm install cockpit oci://oci.stackable.tech/sdp-charts/cockpit --version {stackable-operator-version} --values cockpit-values.yaml
+----
+
+Open the URL in a browser, log in, select *Trino* in the sidebar and run a query such as `SELECT * FROM system.runtime.nodes`.

docs/modules/cockpit/pages/index.adoc

@@ -0,0 +1,23 @@
+= Stackable Cockpit
+:description: Stackable Cockpit is a web UI for the Stackable Data Platform (SDP) to work with services such as Trino and S3 storage from a single place, behind one OIDC login.
+:github: https://github.com/stackabletech/cockpit/
+
+[.link-bar]
+* {github}[GitHub {external-link-icon}^]
+
+Stackable Cockpit is a web UI for the Stackable Data Platform (SDP).
+Users log in once through OpenID Connect (OIDC) and work with the platform's services from a single place.
+
+image::dashboard.png[Cockpit dashboard with bookmarks to Trino, Superset, Airflow and other tools]
+
+== Getting started
+
+Follow the xref:getting-started.adoc[] guide to install Cockpit with Helm and run your first Trino query.
+
+== Modules
+
+Cockpit consists of modules:
+
+* xref:modules/trino.adoc[Trino]: an SQL editor with code completion and a schema browser.
+* xref:modules/storage.adoc[Storage]: a file browser for S3-compatible object storage.
+* xref:modules/bookmarks.adoc[Bookmarks]: links to frequently used tools on the dashboard.

docs/modules/cockpit/pages/installation/authentication.adoc

@@ -0,0 +1,31 @@
+= Authentication
+:description: Configure login to Stackable Cockpit with any OpenID Connect provider, such as Keycloak, including callback URL, scopes and session secret.
+
+Users log in to Cockpit through an OpenID Connect (OIDC) provider, for example Keycloak.
+Register a confidential client for Cockpit with the redirect URI `<base URL>/api/auth/callback/oidc`.
+
+[source,yaml]
+----
+auth:
+  baseUrl: https://cockpit.example.com # <1>
+  oidc:
+    discoveryUrl: https://keycloak.example.com/realms/sdp/.well-known/openid-configuration
+    clientId: cockpit
+    clientSecret:
+      secretKeyRef:
+        name: cockpit
+        key: oidc-client-secret
+    extraScopes: [] # <2>
+sessionSecret:
+  secretKeyRef:
+    name: "" # <3>
+----
+<1> The URL users open in the browser.
+<2> Scopes requested in addition to `openid`, `profile` and `email`.
+<3> The secret that signs session cookies. If empty, the chart generates a Secret (`cockpit-session` for the release `cockpit`) and keeps it across `helm upgrade`.
+Set it explicitly when deploying with `helm template` or Argo CD. They cannot read the existing Secret, so every deployment would generate a new one and log out all users.
+
+The OIDC claim that becomes the Trino user is configured in the xref:modules/trino.adoc#_user_identity[Trino module].
+
+WARNING: With `auth.oidc.enabled: false`, Cockpit has no login and all visitors share one anonymous user.
+Use this only for local testing.

docs/modules/cockpit/pages/installation/database.adoc

@@ -0,0 +1,23 @@
+= Database
+:description: Connect Stackable Cockpit to its PostgreSQL database and configure TLS for the connection.
+
+Cockpit stores its data in a PostgreSQL database.
+It creates and migrates its tables on startup.
+
+[source,yaml]
+----
+database:
+  host: postgresql
+  port: 5432
+  name: cockpit
+  user: cockpit
+  password:
+    secretKeyRef:
+      name: cockpit
+      key: database-password
+  tls:
+    mode: verify-full # <1>
+    secretClass: tls # <2>
+----
+<1> `verify-full` (default), `require` (encrypt without verification) or `disable`.
+<2> Provides the CA certificate of PostgreSQL. Alternatively set `caCert` to a mounted file. `secretClass` and `caCert` both require `verify-full`.

docs/modules/cockpit/pages/installation/helm.adoc

@@ -0,0 +1,109 @@
+= Helm chart
+:description: Install Stackable Cockpit with its Helm chart and configure ingress, reverse proxy headers, CA certificates and the scratch volume.
+
+The Helm chart deploys Cockpit.
+It does not deploy PostgreSQL, the OIDC provider or Trino.
+The chart is published to `oci.stackable.tech/sdp-charts/cockpit`.
+The https://github.com/stackabletech/cockpit/blob/{stackable-operator-branch}/deploy/helm/cockpit/README.md[chart README{external-link-icon}^] lists all values.
+
+[source,console,subs="attributes"]
+----
+$ helm install cockpit oci://oci.stackable.tech/sdp-charts/cockpit --version {stackable-operator-version} --values cockpit-values.yaml
+----
+
+== Ingress and reverse proxy
+
+IMPORTANT: Cockpit must be served over HTTPS, because session cookies and credentials such as S3 keys pass through the browser connection.
+Cockpit itself serves plain HTTP, so terminate TLS at the Ingress or reverse proxy and use `https://` URLs for `origin` and `auth.baseUrl`.
+
+The chart creates a ClusterIP Service on port 3000 and can create an Ingress with the `ingress` values.
+
+Cockpit must be served at the root path `/` of its host, for example `https://cockpit.example.com/`.
+Other base paths such as `https://example.com/cockpit/` are not supported, so do not route a path prefix to Cockpit or rewrite paths in the Ingress or reverse proxy.
+
+Set `origin` to the URL users open in the browser.
+Behind a reverse proxy, also set the forwarded headers Cockpit trusts:
+
+[source,yaml]
+----
+origin: https://cockpit.example.com
+proxy:
+  protocolHeader: x-forwarded-proto
+  hostHeader: x-forwarded-host
+----
+
+=== Response header size
+
+Pages send a `Link` header that preloads their JavaScript and CSS.
+It grows with the number of files a page loads and exceeds 4 KB on some pages.
+Proxies with a smaller header buffer answer these pages with `502 Bad Gateway`, for example ingress-nginx and nginx with their default `4k` buffer.
+Raise the buffer to 16 KB:
+
+[source,yaml]
+----
+ingress:
+  enabled: true
+  className: nginx
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-buffer-size: "16k"
+----
+
+For plain nginx, set `proxy_buffer_size 16k;` and `proxy_busy_buffers_size 16k;`.
+
+// TODO: Remove once Cockpit uses SvelteKit 3, which preloads with <link> elements instead of a header, see https://github.com/sveltejs/kit/pull/15939.
+
+== CA certificates
+
+For backends with a private CA, set either a SecretClass or the path of a mounted CA file:
+
+[source,yaml]
+----
+trino:
+  tls:
+    secretClass: tls # <1>
+database:
+  tls:
+    caCert: /stackable/database-ca/ca.crt # <2>
+extraVolumes:
+  - name: database-ca
+    secret:
+      secretName: database-ca
+extraVolumeMounts:
+  - name: database-ca
+    mountPath: /stackable/database-ca
+    readOnly: true
+----
+<1> The Secret Operator mounts the CA certificate of the SecretClass.
+<2> A CA file mounted from a Secret.
+
+The same options exist under `storage.tls` for S3 endpoints.
+
+== Additional environment variables
+
+To set xref:reference/environment-variables.adoc[environment variables] directly, use `extraEnv` or `extraEnvFrom`:
+
+[source,yaml]
+----
+extraEnv:
+  - name: LOG_LEVEL
+    value: debug
+----
+
+== Scratch volume
+
+The chart mounts an emptyDir at `/tmp` for the archive preview cache.
+Keep `tmp.sizeLimit` above `storage.archiveCacheMaxMb`, otherwise the kubelet evicts the pod when the cache fills up:
+
+[source,yaml]
+----
+tmp:
+  sizeLimit: 1Gi
+storage:
+  archiveCacheMaxMb: 512
+----
+
+== Replicas
+
+Cockpit keeps running queries in memory, so keep `replicaCount` at `1`.
+
+// TODO: Remove once query state is shared between replicas, see https://github.com/stackabletech/cockpit/issues/310.

docs/modules/cockpit/pages/installation/monitoring.adoc

@@ -0,0 +1,10 @@
+= Monitoring
+:description: Stackable Cockpit exposes Prometheus metrics at /metrics and a health endpoint at /healthz, and writes structured JSON logs.
+
+Cockpit exposes Prometheus metrics at `/metrics` on port 3000, without authentication.
+They include HTTP request durations, Trino query counts and storage operation metrics.
+
+The chart's liveness, readiness and startup probes use `/healthz`.
+
+Cockpit writes JSON logs to stdout.
+Set the log level with `logging.level`; the default is `info`.

docs/modules/cockpit/pages/modules/bookmarks.adoc

@@ -0,0 +1,29 @@
+= Bookmarks
+:description: The dashboard of Stackable Cockpit shows bookmarks to the web UIs of the platform's services, set up by each user or globally by the administrator.
+
+The dashboard and the sidebar show bookmarks to frequently used tools, such as the web UIs of the platform's services.
+Each user can add up to 25 bookmarks of their own.
+Pinned bookmarks appear under _Favourites_ at the top of the sidebar.
+
+== Global bookmarks
+
+Global bookmarks are shown to every user, before their own bookmarks.
+Users cannot change them.
+
+[source,yaml]
+----
+bookmarks:
+  global:
+    - productId: superset # <1>
+      name: Dashboards # <2>
+      url: https://superset.example.com # <3>
+      environment: prod # <4>
+      pinned: true # <5>
+----
+<1> One of `trino`, `superset`, `airflow`, `nifi`, `druid`, `opensearch`, `hdfs`, `hbase`, `spark` or `custom`. It selects the icon; unknown values are shown as `custom`.
+<2> Up to 40 characters.
+<3> An HTTP or HTTPS URL without a user name or password. Every user can see it in the page source, so it must not contain a token or other secret either.
+<4> Optional. A label shown on the bookmark, up to 100 characters.
+<5> Optional. Shows the bookmark under _Favourites_. Defaults to `false`.
+
+`bookmarks.global` must be a YAML list.

docs/modules/cockpit/pages/modules/storage.adoc

@@ -0,0 +1,63 @@
+= Storage
+:description: The storage module of Stackable Cockpit is a file browser for S3-compatible object storage with previews, uploads, downloads and file editing.
+
+The storage module is a file browser for S3-compatible object storage.
+Users browse buckets, upload and download files, preview common formats such as CSV, Parquet, images, PDFs and archives, and edit text files.
+
+The module is disabled by default.
+To enable it, add an encryption key to the `cockpit` Secret:
+
+[source,console]
+----
+$ kubectl patch secret cockpit --type merge \
+  -p "{\"stringData\":{\"storage-encryption-key\":\"$(openssl rand -hex 32)\"}}"
+----
+
+Then set:
+
+[source,yaml]
+----
+features:
+  storage:
+    enabled: true
+storage:
+  encryptionKey:
+    secretKeyRef:
+      name: cockpit
+      key: storage-encryption-key # <1>
+  tls:
+    secretClass: tls # <2>
+----
+<1> A 64-character hex string. Keep a backup, see below.
+<2> Optional. Provides the CA certificate of the S3 endpoints. Alternatively set `caCert` to a mounted file.
+
+Each user adds their own S3 storage connections in the UI.
+Cockpit stores them in the database, with the credentials encrypted by the encryption key.
+If the key changes or is lost, the stored credentials can no longer be read.
+
+The CA certificate in `storage.tls` replaces the system trust store, so endpoints with publicly trusted certificates fail while it is set.
+
+== File operations
+
+Cut, copy, paste, rename and drag-and-drop move are disabled by default, because they can cause many S3 requests:
+
+[source,yaml]
+----
+features:
+  storage:
+    transfer: true
+----
+
+== Limits
+
+Preview sizes, upload and transfer concurrency, the archive cache and the download history retention are set under `storage`, see the xref:reference/environment-variables.adoc#_storage[storage environment variables reference]:
+
+[source,yaml]
+----
+storage:
+  filePreviewRows: 500
+  maxEditableFileSize: 10485760 # <1>
+config:
+  bodySizeLimit: 12M
+----
+<1> `config.bodySizeLimit` must be at least this large, otherwise saving large files fails.