feat: add resource manager stack and forwarder function

.github/workflows/release.yml

@@ -0,0 +1,18 @@
+name: Release
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  stack:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+      - run: forwarder/build.sh
+      - run: zip betterstack-oci.zip main.tf variables.tf schema.yaml forwarder.zip
+      - run: gh release create "$GITHUB_REF_NAME" betterstack-oci.zip --generate-notes
+        env:
+          GH_TOKEN: ${{ github.token }}

.github/workflows/test.yml

@@ -0,0 +1,27 @@
+name: Test
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  forwarder:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+      - run: python3 -m unittest test_func
+        working-directory: forwarder
+
+  stack:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+      - uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd # v3
+      - run: forwarder/build.sh
+      - run: terraform fmt -check
+      - run: terraform init -backend=false
+      - run: terraform validate

.gitignore

@@ -0,0 +1,5 @@
+__pycache__/
+.terraform/
+.terraform.lock.hcl
+*.tfstate*
+*.zip

README.md

@@ -1,28 +1,42 @@
 # Better Stack Oracle Cloud integration
 
-Terraform stack that connects an Oracle Cloud Infrastructure tenancy to [Better Stack](https://betterstack.com) for metric and log forwarding.
+This Terraform stack sends metrics and logs from an Oracle Cloud Infrastructure tenancy to [Better Stack](https://betterstack.com).
 
 ## How it works
 
-The stack runs in OCI Resource Manager inside your tenancy. Connector Hub reads **metrics** from OCI Monitoring and **logs** from OCI Logging, then passes them to a forwarder Function. The Function batches them and sends them to Better Stack over HTTPS.
+The stack runs in OCI Resource Manager inside your tenancy. Connector Hub reads **metrics** from OCI Monitoring and **logs** from OCI Logging. It sends them in batches to a forwarder Function. The Function compresses each batch and sends it to Better Stack over HTTPS.
 
-Better Stack never connects to your tenancy. Data only leaves through the Function, and the only secret stored in your tenancy is the Better Stack source token.
+Better Stack never connects to your tenancy. Data leaves your tenancy only through the Function. The only secret that the stack keeps in your tenancy is the Better Stack source token, in the configuration of the Function.
 
 ## Quick start
 
-[Create a new Oracle Cloud source in Better Stack](https://telemetry.betterstack.com/team/t0/sources/new?platform=oci). The source-creation flow hands you a **Deploy to Oracle Cloud** link with every stack variable pre-filled. Click it, review the variables, and click **Create**.
+Before you start, sign in to the **home region** of your tenancy. Use a user who can create compartments and policies in the root compartment.
 
-Sign in to your tenancy's home region as a user who can create dynamic groups and policies in the root compartment.
+1. [Create an Oracle Cloud source in Better Stack](https://telemetry.betterstack.com/team/0/sources/new?platform=oracle_cloud).
+2. On the **Ingest** tab of the source, click **Deploy to Oracle Cloud**.
+3. Accept the Oracle Terms of Use and click **Next**.
+4. Paste the **Source token** of your source and click **Next**. The **Ingesting host** is already filled in.
+5. Make sure that **Run apply** is selected, then click **Create**.
 
-## What gets deployed
+The stack creates a compartment with the name `betterstack`. If your tenancy already has a compartment with this name, remove or rename it first.
 
-- A `betterstack` compartment that holds every resource below.
-- In each region you pick, a Function application running the forwarder, a VCN with a private subnet and a NAT gateway so the Function can reach Better Stack, and Connector Hubs for metrics and logs.
-- Dynamic groups and policies that let Connector Hub read metrics and logs, and invoke the Function.
+The stack sends metrics and logs from the home region only.
+
+## What the stack creates
+
+- A `betterstack` compartment for all the resources below.
+- A Function application for the forwarder. It runs in a VCN with a private subnet. A NAT gateway lets the Function reach Better Stack.
+- One Connector Hub for logs, and one or more Connector Hubs for metrics. Each Connector Hub takes up to 5 compartments and 50 metric namespaces. A large tenancy can need a higher Connector Hub service limit.
+- A policy that lets the Connector Hubs read metrics and logs, and call the Function.
+
+When the stack runs, it finds the compartments that have metrics in the selected namespaces. After you add compartments or use new services, apply the stack again.
 
 ## Uninstall
 
-Go to **Developer Services → Resource Manager → Stacks**, select the Better Stack stack, and click **Destroy**.
+1. Go to **Developer Services → Resource Manager → Stacks** and select the Better Stack stack.
+2. Click **Destroy**.
+
+This takes about 15 minutes. Most of this time goes to removing the `betterstack` compartment.
 
 ## License
 

forwarder/build.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+# Packs the forwarder for a code-only Function: the managed Python runtime
+# expects function/func.py at the archive root and ships the Fn SDK itself.
+set -eu
+cd "$(dirname "$0")/.."
+dir=$(mktemp -d)
+mkdir "$dir/function"
+cp forwarder/func.py "$dir/function/"
+rm -f forwarder.zip
+(cd "$dir" && zip -qX "$OLDPWD/forwarder.zip" function/func.py)
+rm -rf "$dir"

forwarder/func.py

@@ -0,0 +1,33 @@
+import gzip
+import io
+import os
+import urllib.request
+
+from fdk import response
+
+
+class _NoRedirects(urllib.request.HTTPRedirectHandler):
+    # Following a redirect turns the POST into a bodyless GET and sends the
+    # token to another host, so a redirect fails the batch instead.
+    def redirect_request(self, *args, **kwargs):
+        return None
+
+
+_opener = urllib.request.build_opener(_NoRedirects)
+
+
+def handler(ctx, data: io.BytesIO = None):
+    # Connector Hub retries a batch when the function fails, so a non-2xx
+    # response from Better Stack is left to raise.
+    request = urllib.request.Request(
+        f"https://{os.environ['BETTERSTACK_INGESTING_HOST']}/oci",
+        data=gzip.compress(data.getvalue()),
+        headers={
+            "Authorization": f"Bearer {os.environ['BETTERSTACK_SOURCE_TOKEN']}",
+            "Content-Type": "application/json",
+            "Content-Encoding": "gzip",
+        },
+        method="POST",
+    )
+    _opener.open(request, timeout=60)
+    return response.Response(ctx, response_data="")

forwarder/test_func.py

@@ -0,0 +1,83 @@
+import gzip
+import io
+import os
+import sys
+import threading
+import types
+import unittest
+import urllib.error
+from http.server import BaseHTTPRequestHandler, HTTPServer
+from unittest import mock
+
+sys.modules["fdk"] = types.SimpleNamespace(response=types.SimpleNamespace(Response=lambda ctx, response_data: response_data))
+
+import func  # noqa: E402
+
+
+class Ingesting(BaseHTTPRequestHandler):
+    status = 202
+    received = []
+
+    def do_POST(self):
+        body = self.rfile.read(int(self.headers["Content-Length"]))
+        Ingesting.received.append((self.path, dict(self.headers), body))
+        self.send_response(Ingesting.status)
+        if 300 <= Ingesting.status < 400:
+            self.send_header("Location", "/elsewhere")
+        self.end_headers()
+
+    def do_GET(self):
+        Ingesting.received.append((self.path, dict(self.headers), b""))
+        self.send_response(200)
+        self.end_headers()
+
+    def log_message(self, *args):
+        pass
+
+
+class HandlerTest(unittest.TestCase):
+    def setUp(self):
+        Ingesting.received.clear()
+        Ingesting.status = 202
+        self.server = HTTPServer(("127.0.0.1", 0), Ingesting)
+        threading.Thread(target=self.server.serve_forever, daemon=True).start()
+        host = f"127.0.0.1:{self.server.server_port}"
+        # func.py always speaks HTTPS; the test server is plain HTTP.
+        real_request = urllib.request.Request
+        self.patches = [
+            mock.patch.dict(os.environ, {"BETTERSTACK_INGESTING_HOST": host, "BETTERSTACK_SOURCE_TOKEN": "token"}),
+            mock.patch("urllib.request.Request", lambda url, **kw: real_request(url.replace("https://", "http://"), **kw)),
+        ]
+        for patch in self.patches:
+            patch.start()
+
+    def tearDown(self):
+        for patch in self.patches:
+            patch.stop()
+        self.server.shutdown()
+
+    def test_posts_the_batch_gzipped_with_the_source_token(self):
+        func.handler(None, io.BytesIO(b'[{"namespace":"oci_faas"}]'))
+
+        path, headers, body = Ingesting.received[0]
+        self.assertEqual("/oci", path)
+        self.assertEqual("Bearer token", headers["Authorization"])
+        self.assertEqual("gzip", headers["Content-Encoding"])
+        self.assertEqual(b'[{"namespace":"oci_faas"}]', gzip.decompress(body))
+
+    def test_raises_on_a_redirect_instead_of_following_it(self):
+        Ingesting.status = 301
+
+        with self.assertRaises(urllib.error.HTTPError):
+            func.handler(None, io.BytesIO(b"[]"))
+        self.assertEqual(1, len(Ingesting.received))
+
+    def test_raises_on_a_rejected_batch_so_connector_hub_retries(self):
+        Ingesting.status = 500
+
+        with self.assertRaises(urllib.error.HTTPError):
+            func.handler(None, io.BytesIO(b"[]"))
+
+
+if __name__ == "__main__":
+    unittest.main()