diff --git a/src/content/docs/enterprise.mdx b/src/content/docs/enterprise.mdx index ac5a874f71..63a5131c74 100644 --- a/src/content/docs/enterprise.mdx +++ b/src/content/docs/enterprise.mdx @@ -12,6 +12,9 @@ Enterprise, from requirements to installation, integrations, maintenance, and tr [Manual installation](/enterprise/manual-installation-legacy) if you create the GitHub App yourself instead of using the installer. +- Read [Trusting a Private CA](/enterprise/custom-ca) if any of your infrastructure presents + certificates that are not publicly trusted. + - Explore [Advanced features](/enterprise/advanced-features) such as Datadog telemetry, CI Insights, and Slack integrations. diff --git a/src/content/docs/enterprise/advanced-features.mdx b/src/content/docs/enterprise/advanced-features.mdx index dca067f7c2..c2798d2dce 100644 --- a/src/content/docs/enterprise/advanced-features.mdx +++ b/src/content/docs/enterprise/advanced-features.mdx @@ -118,6 +118,9 @@ MERGIFYENGINE_CI_TRACES_DONE_BUCKET="-mergify-ci-traces-done" MERGIFYENGINE_AWS_ENDPOINT_URL_S3=https://my-s3-domain.example.com:1234/ ``` +A self-hosted endpoint whose certificate is not publicly trusted needs its certificate authority +too: see [Trusting a Private CA](/enterprise/custom-ca). + #### Option A: IAM role discovery (recommended) Leave `MERGIFYENGINE_AWS_ACCESS_KEY_ID` and `MERGIFYENGINE_AWS_SECRET_ACCESS_KEY` unset and Mergify diff --git a/src/content/docs/enterprise/custom-ca.mdx b/src/content/docs/enterprise/custom-ca.mdx new file mode 100644 index 0000000000..67a62c6938 --- /dev/null +++ b/src/content/docs/enterprise/custom-ca.mdx @@ -0,0 +1,148 @@ +--- +title: 'Trusting a Private CA' +description: 'Make an on-premise deployment trust certificates issued by your own certificate authority.' +--- + +Mergify checks the TLS certificates it is presented against a set of public certificate authorities +(CAs). When a server presents a certificate that traces back to none of them, verification fails and +the connection is refused. + +Point the `MERGIFYENGINE_EXTRA_CA_BUNDLE` environment variable at a PEM file and Mergify trusts what +it holds on top of the public certificate authorities it already ships. It is the trust store for +every outbound connection the engine opens. You need it when: + +- Your GitHub Enterprise Server presents a self-signed certificate, or one issued by your internal + PKI. This covers the API calls as well as the `git` clones and pushes the merge queue makes. + +- Your Redis, PostgreSQL, or object storage endpoints do the same. Object storage means both Amazon + S3 and Google Cloud Storage. Managed providers usually publish their certificate authority as a + downloadable PEM file, and that file is what goes here. + +- An egress proxy intercepts outbound TLS and re-signs it with a corporate root. + +Your certificates are added to the public ones rather than replacing them, so calls to Mergify's own +[subscription endpoints](/enterprise/requirements/#external-network-access) keep working. + +Two dependencies need one more setting before they check anything against the bundle: +[PostgreSQL](#postgresql-needs-sslmode-too), which verifies no certificate at all by default, and +[Redis](#redis-and-certificate-verification). + +## Prepare the Bundle + +The file is a PEM bundle: one or more `-----BEGIN CERTIFICATE-----` blocks in a single file. What +belongs in it depends on how the server certificate was issued: + +- **Issued by a certificate authority**, whether your internal PKI or a managed provider's. Use the + root certificate and any intermediates, obtained from whoever runs that authority rather than + harvested from the connection you are trying to verify. + +- **Self-signed.** The certificate is its own issuer, so there is no separate authority to ask for. + Put the server certificate itself in the bundle. + +If what you were given is DER-encoded, convert it first: + +```sh +openssl x509 -inform der -in mycompany-ca.crt -out mycompany-ca.pem +``` + +## Configure the Engine + +Mount the file into the container and name it in the environment. Add these two options to your +[normal-mode command](/enterprise/installation/#start-mergify-in-normal-mode): + +```sh +-v /etc/pki/mycompany-ca.pem:/etc/mergify/ca.pem:ro \ +-e MERGIFYENGINE_EXTRA_CA_BUNDLE=/etc/mergify/ca.pem \ +``` + +The container runs as an unprivileged user, so the mounted file has to be readable by it: + +```sh +chmod 644 /etc/pki/mycompany-ca.pem +``` + +Mergify loads the bundle while it starts and refuses to boot when the file is missing, unreadable, +or not valid PEM, so a wrong path is a startup error naming the setting rather than a handshake +failure hours later. + +If you split the deployment across several containers, mount the file into each one and set the +variable on all of them. Every process reads the bundle for itself. + +## PostgreSQL Needs `sslmode` Too + +The bundle gives PostgreSQL the certificates to check against, but it does not decide whether they +get checked. That is `sslmode` in the DSN, and its default, `prefer`, encrypts the connection +without verifying the certificate at all. Ask for verification explicitly: + +```ini +MERGIFYENGINE_DATABASE_URL=postgresql://postgres:password@db.mycompany.com:5432/postgres?sslmode=verify-full +``` + +`verify-full` also requires the certificate to name the host you connect by. When it does not, a +container alias for instance, use `sslmode=verify-ca`, which checks the certificate against the +bundle without pinning the hostname. Either way the engine opens the database while it starts, so a +mismatch is a boot failure rather than something you find later. + +A `?sslrootcert=` already in the DSN takes precedence, and PostgreSQL then verifies against that +file alone. + +## Redis and Certificate Verification + +Connect with `rediss://` and Mergify verifies the Redis certificate against the bundle like any +other. A `?ssl_ca_certs=` in `MERGIFYENGINE_REDIS_URL` points Redis at that file instead. + +:::caution + `MERGIFYENGINE_REDIS_SSL_VERIFY_MODE_CERT_NONE=1` turns Redis certificate verification off + entirely and overrides the bundle. The connection stays encrypted, but Mergify can no longer tell + your Redis apart from anything else answering at that address. Keep it only while you cannot + obtain the certificate, and unset it once you can; Mergify logs a line when both are configured. +::: + +## Verify It Took Effect + +Finish the two sections above first. A connectivity check against a PostgreSQL or Redis that is not +verifying anything reports `ok` without a certificate having been checked. + +Restart the container and look for the bundle in the startup logs: + +```sh +docker logs mergify-engine | grep "Extra CA bundle" +``` + +Each service logs one line naming the bundle it loaded. If the grep finds nothing, read the logs +unfiltered before suspecting your `-e` flag: a bundle the engine cannot read or parse stops the +container before that line, with a different error naming the setting. + +Then check that Mergify reaches each dependency: + +```sh +docker exec -u root -it mergify-engine /bin/bash +mergify-admin connectivity-check +``` + +Every configured dependency should report `ok`; `skipped` means it is not configured at all. A check +that fails on certificate verification means that server's issuer is still missing from the bundle. +See [Troubleshooting](/enterprise/troubleshooting/#checking-third-party-connectivity) for the full +output format. + +## Rotating the Certificate Authority + +The bundle is read once, when the process starts. Replacing the file on disk changes nothing until +the containers restart. + +To rotate without downtime, add the incoming certificate to the bundle alongside the outgoing one, +restart Mergify, switch your servers over, and only then drop the old certificate and restart again. + +## Other Trust Store Variables + +`SSL_CERT_FILE` and `SSL_CERT_DIR` replace the public certificate authorities Mergify would +otherwise use, and `MERGIFYENGINE_EXTRA_CA_BUNDLE` is added on top of whichever you set. + +Use `SSL_CERT_FILE` with a PEM bundle. `SSL_CERT_DIR` reaches the GitHub and Redis connections, but +not the ones that take a single bundle file: PostgreSQL, object storage, Sentry and `git` fall back +to the public authorities Mergify ships, plus your bundle. Your replacement trust store is then only +partly in effect, and Mergify logs a line saying so at startup. + +`AWS_CA_BUNDLE`, if you already have one set, keeps object storage on that file and the bundle is +not applied there. `REQUESTS_CA_BUNDLE` and `CURL_CA_BUNDLE` do nothing at all: Mergify clears them +before it builds any client. diff --git a/src/content/docs/enterprise/installation.mdx b/src/content/docs/enterprise/installation.mdx index ff33321ff4..1b9367b421 100644 --- a/src/content/docs/enterprise/installation.mdx +++ b/src/content/docs/enterprise/installation.mdx @@ -163,21 +163,32 @@ Redis server provides TLS termination. To connect to such an instance you need t #### Optional: use Redis with self-signed TLS certificate -Some Redis server providers set it up with self-signed certificates. In that case, to connect to -such an instance you need to: +Some Redis server providers set it up with self-signed certificates, or with certificates from their +own certificate authority. In that case, to connect to such an instance you need to: - Use `rediss://` instead of `redis://` in the `MERGIFYENGINE_REDIS_URL` environment variable. -- Set `MERGIFYENGINE_REDIS_SSL_VERIFY_MODE_CERT_NONE=1` in your environment variables. Mergify will - connect to Redis using encryption but will not verify the server certificate. +- Add the certificate that signed it, or the server certificate itself when it is self-signed, to + the bundle described in [Trusting a Private CA](/enterprise/custom-ca). Managed providers usually + publish theirs as a downloadable PEM file. Mergify then verifies the Redis certificate like any + other. + +:::caution + `MERGIFYENGINE_REDIS_SSL_VERIFY_MODE_CERT_NONE=1` is the fallback for when you cannot obtain the + certificate at all. Mergify keeps encrypting the connection but stops verifying the server + certificate, so it can no longer tell your Redis apart from anything else answering at that + address. It also overrides the bundle, so unset it once you hold the certificate. +::: ### PostgreSQL Provision a PostgreSQL instance, create a database/user for Mergify, and ensure the container can reach it over the network. Capture the DSN you will pass through `MERGIFYENGINE_DATABASE_URL` (for example `postgresql://postgres:password@postgres:5432/postgres`). -If your provider enforces TLS, configure the connection options accordingly. The first start also -creates a few extensions in that database, so check the +If your provider enforces TLS, add the matching `sslmode` to the DSN; a certificate issued by your +own certificate authority needs `sslmode=verify-full` and the bundle described in +[Trusting a Private CA](/enterprise/custom-ca). The first start also creates a few extensions in +that database, so check the [PostgreSQL requirements](/enterprise/requirements/#postgresql-requirements) before you lock down its privileges. @@ -206,7 +217,9 @@ each authentication mode. ::: The example below includes the CI Insights and Test Insights environment variables. Omit them if you -skipped the previous step. +skipped the previous step. If any of the services Mergify connects to present certificates that are +not publicly trusted, see [Trusting a Private CA](/enterprise/custom-ca) for the two options to add +to this command. ```sh docker run \ diff --git a/src/content/docs/enterprise/manual-installation-legacy.mdx b/src/content/docs/enterprise/manual-installation-legacy.mdx index c98fdb89e6..bd4ff7e89d 100644 --- a/src/content/docs/enterprise/manual-installation-legacy.mdx +++ b/src/content/docs/enterprise/manual-installation-legacy.mdx @@ -128,7 +128,9 @@ docker pull registry.mergify.com/enterprise:@@ENTERPRISE_VERSION@@ > A subscription token is required. Contact Mergify if you do not have one. > -> Ensure Redis (with persistence/TLS if needed) and Postgres are ready. +> Ensure Redis (with persistence/TLS if needed) and Postgres are ready. If they, or your GitHub +> Enterprise Server, present certificates that are not publicly trusted, see +> [Trusting a Private CA](/enterprise/custom-ca). ```sh docker run \ diff --git a/src/content/docs/enterprise/requirements.mdx b/src/content/docs/enterprise/requirements.mdx index c688f79ac3..0283932831 100644 --- a/src/content/docs/enterprise/requirements.mdx +++ b/src/content/docs/enterprise/requirements.mdx @@ -56,6 +56,9 @@ docker run --name some-redis-with-tls -d \ bitnami/redis ``` +That example makes Redis present a certificate signed by a CA you created. Give that CA to Mergify +as described in [Trusting a Private CA](/enterprise/custom-ca), so it can verify the connection. + ## Mergify Requirements The Mergify container needs at least: @@ -90,4 +93,6 @@ Two outbound HTTPS calls must be allowed so Mergify can validate and report lice - `POST https://subscription.mergify.com/on-premise/report` Both calls carry your subscription token and are made by the Mergify container itself, so no other -component needs egress to that host. +component needs egress to that host. If a proxy intercepts outbound TLS and re-signs it with your +own certificate authority, Mergify has to trust that authority for these calls to succeed: see +[Trusting a Private CA](/enterprise/custom-ca). diff --git a/src/content/docs/enterprise/troubleshooting.mdx b/src/content/docs/enterprise/troubleshooting.mdx index 75639ad177..b126a3b0f5 100644 --- a/src/content/docs/enterprise/troubleshooting.mdx +++ b/src/content/docs/enterprise/troubleshooting.mdx @@ -37,6 +37,9 @@ object_storage: skipped github_server: ok ``` +A check that fails on certificate verification means the server presents a certificate Mergify does +not trust. See [Trusting a Private CA](/enterprise/custom-ca). + ### API healthcheck Add a shared token: diff --git a/src/content/enterpriseNavItems.ts b/src/content/enterpriseNavItems.ts index 5f12a316ee..4ff52376b8 100644 --- a/src/content/enterpriseNavItems.ts +++ b/src/content/enterpriseNavItems.ts @@ -5,6 +5,7 @@ const enterpriseNavItems: NavItem[] = [ { title: 'Architecture', path: '/enterprise/architecture', icon: 'lucide:layout-grid' }, { title: 'Requirements', path: '/enterprise/requirements', icon: 'lucide:clipboard-list' }, { title: 'Installation', path: '/enterprise/installation', icon: 'lucide:wrench' }, + { title: 'Private CA', path: '/enterprise/custom-ca', icon: 'lucide:shield-check' }, { title: 'Advanced Features', path: '/enterprise/advanced-features',