From 7109a63d7b042700c7bf3ce23e3e149e20f15b99 Mon Sep 17 00:00:00 2001 From: "Arthur (smallgun01)" Date: Wed, 1 Jul 2026 21:41:55 +0800 Subject: [PATCH 1/6] docs: add refarch for running Telegram/LINE bots on AWS Add reference architecture document covering three AWS-based ingress options for OpenAB Telegram and LINE bots: - API Gateway HTTP API + VPC Link + Cloud Map (recommended, ~-10/mo) - ALB + ECS Fargate (~8-21/mo) - Cloudflare Tunnel sidecar (~-10/mo) Includes cost breakdowns, security group setup, webhook URL lifecycle, multi-platform guidance, and real-world pitfalls from production deployments. Co-authored-by: OpenAB Bot --- docs/refarch/running-telegram-line-on-aws.md | 541 +++++++++++++++++++ 1 file changed, 541 insertions(+) create mode 100644 docs/refarch/running-telegram-line-on-aws.md diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md new file mode 100644 index 000000000..4108ae2f0 --- /dev/null +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -0,0 +1,541 @@ +# Running Telegram / LINE Bots on AWS + +Three production-grade reference architectures for deploying OpenAB bots with Telegram +and/or LINE inbound webhooks on AWS ECS Fargate — without requiring Kubernetes. + +Each option includes cost estimates, security group configuration, webhook URL setup, +and pitfalls the authors actually hit in production. + +--- + +## Prerequisite Concept: Outbound vs Inbound + +OpenAB started as a Discord bot framework. Discord uses **outbound WebSocket** — the bot +connects *to* Discord. No ingress needed. Your ECS task only needs egress to the internet. + +Telegram and LINE are different. They send **inbound HTTP POST** (webhooks) *to your bot*. +Your ECS task must accept HTTPS traffic from the public internet on `:8080`. + +``` +Discord (outbound): Telegram / LINE (inbound): + + Bot ──WS──▶ Discord GW Platform ──HTTPS POST──▶ Your Bot + (no ingress) (needs public endpoint) +``` + +This is why the architectures below all solve one problem: **getting external HTTPS +traffic to your Fargate task's `:8080` without paying for an ALB.** + +--- + +## Architecture Overview + +| # | Option | Monthly Cost | Best For | +|---|--------|-------------|----------| +| 1 | **API Gateway HTTP API + VPC Link + Cloud Map** | ~$5–10 | Single/multi bot, no ALB, lowest AWS-native cost | +| 2 | **ALB + ECS Fargate** | ~$20+ | Health checks, auto-scaling, enterprise | +| 3 | **Cloudflare Tunnel sidecar** | ~$5–10 | Already have Cloudflare, simplest setup | + +> **Version note (callout)** +> OpenAB beta.5+ supports **unified webhook mode** — set `TELEGRAM_BOT_TOKEN` or +> `LINE_CHANNEL_SECRET` + `LINE_CHANNEL_ACCESS_TOKEN` as env vars directly on your bot +> container, no separate gateway service needed. The infra paths below work identically +> from beta.2 through beta.6. The only difference: beta.2 required a standalone gateway +> service + `[gateway]` config block; beta.5+ does not. + +--- + +## Option 1: API Gateway HTTP API + VPC Link + Cloud Map (Recommended) + +This is the cheapest AWS-native path. It replaces a $16+/month ALB with a ~$1/month +API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcoded IPs. + +### Architecture Diagram + +``` + Internet + │ + ┌───────────┴───────────┐ + │ Telegram / LINE │ + │ Platform │ + └───────────┬───────────┘ + │ HTTPS POST + ▼ + ┌───────────────────────┐ + │ API Gateway │ ~$1.00/million requests + │ (HTTP API) │ + │ routes: │ + │ /webhook/telegram │ + │ /webhook/line │ + └───────────┬───────────┘ + │ VPC Link ($0 — free for HTTP API) + ▼ + ┌───────────────────────┐ + │ AWS Cloud Map │ ~$0 (private DNS) + │ namespace: oab │ + │ service: your-bot │ + │ → DNS A record │ + │ resolves to task IP │ + └───────────┬───────────┘ + │ + ▼ + ┌───────────────────────┐ + │ ECS Fargate Task │ + │ OpenAB :8080 │ + │ SG: allow inbound │ + │ from VPC Link prefix │ + └───────────────────────┘ +``` + +### Prerequisites + +- ECS Fargate cluster with an OpenAB task running (unified binary) +- VPC with subnets for the ECS tasks (private or public both work; the VPC Link uses its own private subnet ENIs internally) +- The task's security group +- AWS CLI or Console access + +> **DNS record type: use A, not SRV** +> API Gateway HTTP API integration resolves DNS A records — SRV records do **not** work +> for VPC Link targets. Always use `{Type="A"}` in your Cloud Map service configuration. +> The port (`:8080`) is specified in the integration URI, not the DNS record. + +### Step-by-Step + +#### 1. Create a Cloud Map Namespace & Service + +```bash +# Private DNS namespace (one-time per VPC) +aws servicediscovery create-private-dns-namespace \ + --name oab \ + --vpc vpc-abc123 \ + --region us-east-1 + +# Service with DNS A record (auto-registers task IP) +aws servicediscovery create-service \ + --name your-bot \ + --namespace-id ns-xxx \ + --dns-config 'NamespaceId="ns-xxx",DnsRecords=[{Type="A",TTL="60"}]' \ + --region us-east-1 +``` + +#### 2. Enable Service Discovery on Your ECS Service + +ECS service discovery can only be set at **service creation time**, not via update. +If your service already exists, delete and recreate it: + +```bash +# Delete existing service (safe when desiredCount=0) +aws ecs delete-service --cluster oab --service your-bot --region us-east-1 + +# Recreate with service discovery +aws ecs create-service \ + --cluster oab \ + --service-name your-bot \ + --task-definition your-bot:latest \ + --desired-count 0 \ + --launch-type FARGATE \ + --network-configuration \ + "awsvpcConfiguration={subnets=[subnet-xxx,subnet-yyy],securityGroups=[sg-zzz],assignPublicIp=DISABLED}" \ + --service-registries "registryArn=arn:aws:servicediscovery:us-east-1:123456:service/srv-xxx" \ + --region us-east-1 + +# Start the bot (service was created with desired count 0) +aws ecs update-service --cluster oab --service your-bot --desired-count 1 --region us-east-1 +``` + +When the task starts, ECS auto-registers its private IP as the DNS A record for +`your-bot.oab`. Other services in the same VPC resolve this name to reach the task on +`:8080`. + +#### 3. Create a VPC Link + +```bash +aws apigatewayv2 create-vpc-link \ + --name oab-vpc-link \ + --subnet-ids subnet-xxx subnet-yyy \ + --security-group-ids sg-zzz \ + --region us-east-1 +``` + +> **Important**: The VPC Link needs subnets in the same VPC as your ECS tasks. +> It does NOT need public IPs — it uses private connectivity through AWS Hyperplane. + +#### 4. Create an API Gateway HTTP API + +```bash +# Create the API +API_ID=$(aws apigatewayv2 create-api \ + --name oab-webhook \ + --protocol-type HTTP \ + --region us-east-1 \ + --query 'ApiId' --output text) + +# Create the integration FIRST: VPC Link → Cloud Map service endpoint +# Endpoint = your-bot.oab:8080 (the Cloud Map DNS name) +INTEGRATION_ID=$(aws apigatewayv2 create-integration \ + --api-id $API_ID \ + --integration-type HTTP_PROXY \ + --integration-method POST \ + --integration-uri "http://your-bot.oab:8080" \ + --connection-type VPC_LINK \ + --connection-id $VPC_LINK_ID \ + --payload-format-version "1.0" \ + --region us-east-1 \ + --query 'IntegrationId' --output text) + +# Add Telegram route (uses the integration created above) +aws apigatewayv2 create-route \ + --api-id $API_ID \ + --route-key "POST /webhook/telegram" \ + --target "integrations/$INTEGRATION_ID" \ + --region us-east-1 + +# Add LINE route (same VPC Link, different path) +aws apigatewayv2 create-route \ + --api-id $API_ID \ + --route-key "POST /webhook/line" \ + --target "integrations/$INTEGRATION_ID" \ + --region us-east-1 + +# Deploy +aws apigatewayv2 create-stage \ + --api-id $API_ID \ + --stage-name prod \ + --auto-deploy \ + --region us-east-1 +``` + +Your webhook URL is now: +``` +https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/telegram +https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/line +``` + +> **Path passthrough is automatic**: API Gateway HTTP API appends the route path to the +> integration URI. The route `POST /webhook/telegram` combined with integration URI +> `http://your-bot.oab:8080` results in requests hitting +> `http://your-bot.oab:8080/webhook/telegram` on the container. No path rewriting +> needed. + +This URL **never changes** — even when tasks restart and get new private IPs, Cloud Map +auto-updates the DNS record and the VPC Link resolves the new IP transparently. + +#### 5. Security Group: Inbound Rules + +The biggest pitfall when moving from Discord-only to Telegram/LINE. + +**Before (Discord-only — outbound only):** + +| Type | Protocol | Port | Source | Purpose | +|------|----------|------|--------|---------| +| — | — | — | — | No inbound rules needed | + +**After (Telegram/LINE ready — inbound webhook):** + +| Type | Protocol | Port | Source | Purpose | +|------|----------|------|--------|---------| +| HTTP | TCP | 8080 | Self (sg-xxx) | Inter-service WS (bot ↔ gateway) | +| HTTP | TCP | 8080 | VPC Link prefix list | API Gateway → task traffic | + +> **Pitfall**: If you use self-referencing SG rules (same SG as source), the VPC Link +> traffic arrives from the VPC Link's ENI — which is in your SG. A self-referencing +> inbound rule on `:8080` covers both inter-service and VPC Link traffic in most cases. +> If not, add the VPC Link's prefix list explicitly. + +#### 6. Set Webhook URLs + +**Telegram (BotFather / setWebhook):** +``` +https://api.telegram.org/bot/setWebhook?url=https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/telegram +``` + +**LINE (LINE Developers Console):** +``` +Webhook URL: https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/line +``` + +#### 7. OpenAB Environment Variables + +```bash +# Telegram +TELEGRAM_BOT_TOKEN=123:abc + +# LINE +LINE_CHANNEL_SECRET=xxx +LINE_CHANNEL_ACCESS_TOKEN=yyy + +# Discord (unchanged) +DISCORD_BOT_TOKEN=zzz +``` + +With unified webhook mode (beta.5+), the OpenAB binary auto-detects which env vars are +set and starts the corresponding platform adapters. No `[gateway]` config block needed +for Telegram/LINE. + +### Cost Breakdown + +| Resource | Monthly Cost | +|----------|-------------| +| API Gateway HTTP API | ~$1.00/million requests (effectively $0 for small bots) | +| VPC Link | $0 (free for HTTP API; hourly charge only applies to REST API) | +| Cloud Map | $0 (private DNS, minimal queries) | +| ECS Fargate (256/512, 12h/day) | ~$5–10/month | +| **Total** | **~$5–10/month** | + +> The HTTP API VPC Link is free — unlike REST API VPC Links which charge ~$0.01/hour. +> This makes API Gateway the cheapest AWS-native ingress option for low-traffic bots. + +### Pitfalls We Hit + +1. **SG inbound for VPC Link traffic** + VPC Link traffic originates from the VPC Link's ENI in your subnet. A + self-referencing SG rule on `:8080` usually covers it. If you see 503 from API + Gateway, check your SG inbound rules first. + +2. **VPC Link subnets must be private** + VPC Link attaches to private subnets only. If your ECS task uses public subnets + with `assignPublicIp=ENABLED`, that's fine — the VPC Link still uses private + subnets for its ENIs. + +3. **ECS service discovery is create-only** + You cannot add `--service-registries` to an existing ECS service via + `update-service`. If your service already exists, delete it (safe when + `desiredCount=0`) and recreate with the registry ARN. + +4. **LINE webhook verification** + LINE sends an initial verification request when you set the webhook URL. OpenAB + handles signature verification automatically — just make sure the container has the + correct `LINE_CHANNEL_SECRET` env var. + +### Pros & Cons + +| Pros | Cons | +|------|------| +| Cheapest AWS-native path (~$5–10/mo total) | VPC Link adds an extra network hop | +| Webhook URL never changes (API Gateway endpoint is static) | One extra service to manage (API Gateway) | +| Same infra for Telegram + LINE + Discord | Initial setup has more moving parts | +| No ALB, no Kubernetes, no Cloudflare dependency | Debugging requires checking VPC Link logs | + +--- + +## Option 2: ALB + ECS Fargate + +> **Disclaimer**: The authors evaluated this option but did not deploy it in production. +> The comparison below is based on AWS public pricing and architecture best practices. + +### High-Level Architecture + +``` +Telegram / LINE ──HTTPS──▶ ALB (public) ──HTTP──▶ ECS Fargate :8080 + │ + Target Group + health check: /health +``` + +### Cost + +| Resource | Monthly | +|----------|---------| +| ALB (fixed) | ~$16.20 | +| ALB LCU (low traffic) | ~$2–5 | +| **Total** | **~$18–21/month** | + +### When to Choose This + +- You need native health checks and auto-scaling +- You run multiple bots behind a single ALB with path-based routing +- Enterprise compliance requires AWS WAF or Shield integration +- You already have an ALB in the account + +### Why We Didn't Choose It + +For a 1–4 bot personal deployment, the ALB's fixed cost (~$16/month) alone is **triple** +the total cost of the API Gateway + VPC Link + Cloud Map path. The feature gap (health +checks, auto-scaling) doesn't justify the cost increase at this scale. + +--- + +## Option 3: Cloudflare Tunnel Sidecar + +We use this in production for our LINE bots. If you already use +Cloudflare, this is the simplest path. + +> **Cross-reference**: For a Kubernetes-based Cloudflare Tunnel setup, see the existing +> [`docs/refarch/telegram-cloudflare-tunnel.md`][cf-k3s]. This section covers the +> **ECS Fargate specific** approach. + +[cf-k3s]: ../telegram-cloudflare-tunnel.md + +### Cost Breakdown + +| Resource | Monthly Cost | +|----------|-------------| +| Cloudflare Tunnel (cloudflared sidecar) | $0 (free tunnel, no ingress charges) | +| ECS Fargate (256/512, 12h/day) | ~$5–10/month | +| **Total** | **~$5–10/month** | + +> Same total as Option 1 — the difference is Cloudflare vs AWS for the ingress layer, +> not the Fargate cost. Choose based on which ecosystem you already use. + +### Architecture (ECS Fargate) + +``` +Telegram / LINE ──HTTPS──▶ Cloudflare Edge + │ + Tunnel (cloudflared) + │ + ┌────────┴────────┐ + │ ECS Fargate │ + │ sidecar: │ + │ cloudflared │ + │ → localhost: │ + │ 8080 │ + │ │ + │ main: OpenAB │ + │ :8080 │ + └─────────────────┘ +``` + +### Key Differences from K3s Version + +| Aspect | K3s (existing doc) | ECS Fargate (this guide) | +|--------|-------------------|-------------------------| +| Tunnel sidecar | Separate Pod or external | Sidecar container in same task-def | +| Network | Pod-to-Pod (localhost) | Container-to-container (localhost) | +| Tunnel config | ConfigMap / env var | ECS Secrets Manager or env var | +| QUIC vs HTTP2 | QUIC (default) | **Use `--protocol http2`** (QUIC unstable on 256 CPU) | + +### Step-by-Step (ECS Fargate) + +#### 1. Create a Cloudflare Tunnel + +In Cloudflare Zero Trust dashboard: create a tunnel, note the tunnel token. + +#### 2. Store Token in AWS Secrets Manager + +```bash +aws secretsmanager create-secret \ + --name openab/cloudflare-tunnel-token \ + --secret-string '{"token":"your-tunnel-token"}' \ + --region us-east-1 +``` + +#### 3. Add cloudflared Sidecar to Task Definition + +```json +{ + "containerDefinitions": [ + { + "name": "cloudflared", + "image": "cloudflare/cloudflared:latest", + "command": ["tunnel", "--no-autoupdate", "run", "--protocol", "http2"], + "secrets": [ + { + "name": "TUNNEL_TOKEN", + "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/cloudflare-tunnel-token:token::" + } + ], + "essential": true + }, + { + "name": "openab", + "image": "ghcr.io/openabdev/openab:0.9.0-beta.6-opencode", + "portMappings": [{"containerPort": 8080}], + "environment": [ + {"name": "TELEGRAM_BOT_TOKEN", "value": "123:abc"}, + {"name": "LINE_CHANNEL_SECRET", "value": "xxx"}, + {"name": "LINE_CHANNEL_ACCESS_TOKEN", "value": "yyy"} + ], + "essential": true + } + ] +} +``` + +#### 4. Set Public Hostname in Cloudflare Dashboard + +Point your tunnel's public hostname (e.g., `bot.example.com`) to `localhost:8080`. + +#### 5. Set Webhook URLs + +``` +Telegram: https://bot.example.com/webhook/telegram +LINE: https://bot.example.com/webhook/line +``` + +> **Network requirement**: The cloudflared sidecar must reach the internet (it connects +> **outbound** to Cloudflare's edge). Use `assignPublicIp=ENABLED` (public subnet) or a +> NAT Gateway (private subnet). Without internet egress, the tunnel will not establish. + +### Pitfalls (ECS-Specific) + +1. **QUIC on low CPU is unstable** + On 256 CPU Fargate, cloudflared's default QUIC protocol causes intermittent tunnel + disconnections. Always add `--protocol http2`. + +2. **Tunnel token rotation** + If you regenerate the tunnel token in Cloudflare, update the Secrets Manager secret. + You must force a new ECS deployment (stop existing task) for the sidecar to pick up + the new token. + +3. **Sidecar essential = true** + If cloudflared crashes, the entire task stops. This is intentional — a bot with no + ingress is useless. Set `essential: true` on both containers. + +### Pros & Cons + +| Pros | Cons | +|------|------| +| Simplest setup (just add a sidecar) | Requires Cloudflare account | +| Free tunnel (no AWS ingress costs) | QUIC instability on low CPU | +| Cloudflare DDoS protection included | Tunnel token must be in Secrets Manager | +| Same pattern works for any platform | Extra ~50MB RAM for cloudflared | + +--- + +## Multi-Platform: Telegram + LINE + Discord on One Task + +OpenAB's unified binary can handle all three platforms simultaneously from a single +ECS task: + +```bash +# Environment variables — set all three +TELEGRAM_BOT_TOKEN=123:abc +LINE_CHANNEL_SECRET=xxx +LINE_CHANNEL_ACCESS_TOKEN=yyy +DISCORD_BOT_TOKEN=zzz +``` + +The binary auto-detects which env vars are present and starts each platform adapter: + +| Env Var Present | Adapter | Webhook Path | +|----------------|---------|-------------| +| `DISCORD_BOT_TOKEN` | Discord | (outbound WS, no path) | +| `TELEGRAM_BOT_TOKEN` | Telegram | `/webhook/telegram` | +| `LINE_CHANNEL_SECRET` + `LINE_CHANNEL_ACCESS_TOKEN` | LINE | `/webhook/line` | + +**API Gateway routing**: Add one route per platform, all pointing to the same VPC Link: + +``` +POST /webhook/telegram ─┐ +POST /webhook/line ─┼── same VPC Link → same Cloud Map → same task :8080 +``` + +**Cost efficiency**: One task replaces three. The API Gateway + VPC Link is shared +across all platforms. Total: **~$5–10/month for Telegram + LINE + Discord on a single +Fargate task.** + +--- + +## Summary + +| Factor | Option 1 (API GW + CM) | Option 2 (ALB) | Option 3 (CF Tunnel) | +|--------|----------------------|----------------|---------------------| +| Monthly cost | ~$5–10 | ~$18–21 | ~$5–10 | +| Setup complexity | Medium | Low | Low | +| AWS-native | ✅ Yes | ✅ Yes | ❌ No (CF dependency) | +| Health checks | Manual | ✅ Built-in | Manual | +| Multi-platform | ✅ Single VPC Link | ✅ Single ALB | ✅ Single tunnel | +| Best for | **Budget + AWS-native** | Enterprise | **Cloudflare users** | + +--- From 12653abd68adfb5e3b5bf3bc1e441f7caf04b15f Mon Sep 17 00:00:00 2001 From: "Arthur (smallgun01)" Date: Wed, 1 Jul 2026 22:10:20 +0800 Subject: [PATCH 2/6] docs: fix review findings F1-F3 F1: use secrets block instead of plaintext env vars in Option 3 F2: fix your-bot:latest to your-bot (ECS family name) F3: soften VPC Link subnet claim to recommended not required --- docs/refarch/running-telegram-line-on-aws.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md index 4108ae2f0..7be83d29a 100644 --- a/docs/refarch/running-telegram-line-on-aws.md +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -147,6 +147,10 @@ When the task starts, ECS auto-registers its private IP as the DNS A record for `your-bot.oab`. Other services in the same VPC resolve this name to reach the task on `:8080`. +> **ECS task-def naming**: Use the family name (resolves to latest ACTIVE revision) +> or `family:revision` (e.g., `your-bot:1`). ECS does not support Docker-style +> `:latest` tags. + #### 3. Create a VPC Link ```bash @@ -292,8 +296,9 @@ for Telegram/LINE. self-referencing SG rule on `:8080` usually covers it. If you see 503 from API Gateway, check your SG inbound rules first. -2. **VPC Link subnets must be private** - VPC Link attaches to private subnets only. If your ECS task uses public subnets +2. **VPC Link subnets should be private (recommended)** + VPC Link ENIs don't need public IPs, so private subnets are preferred. + Public subnets also work, but your VPC Link will still use private ENIs internally. If your ECS task uses public subnets with `assignPublicIp=ENABLED`, that's fine — the VPC Link still uses private subnets for its ENIs. @@ -441,10 +446,10 @@ aws secretsmanager create-secret \ "name": "openab", "image": "ghcr.io/openabdev/openab:0.9.0-beta.6-opencode", "portMappings": [{"containerPort": 8080}], - "environment": [ - {"name": "TELEGRAM_BOT_TOKEN", "value": "123:abc"}, - {"name": "LINE_CHANNEL_SECRET", "value": "xxx"}, - {"name": "LINE_CHANNEL_ACCESS_TOKEN", "value": "yyy"} + "secrets": [ + {"name": "TELEGRAM_BOT_TOKEN", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/telegram-bot-token:token::"}, + {"name": "LINE_CHANNEL_SECRET", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/line-shared:LINE_CHANNEL_SECRET::"}, + {"name": "LINE_CHANNEL_ACCESS_TOKEN", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/line-shared:LINE_CHANNEL_ACCESS_TOKEN::"} ], "essential": true } From 647d4e43f94707db3136de625d90df91ecb003c8 Mon Sep 17 00:00:00 2001 From: smallgun01 Date: Thu, 2 Jul 2026 09:07:20 +0800 Subject: [PATCH 3/6] =?UTF-8?q?docs:=20fix=20VPC=20Link=20integration=20?= =?UTF-8?q?=E2=80=94=20use=20SRV=20record=20+=20Cloud=20Map=20ARN?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes based on chaodu-agent's live E2E finding in #1275: - Cloud Map service must use SRV records (not A) — the port comes from SRV - Integration URI must be the Cloud Map service ARN (not http:// URL) - API Gateway HTTP API VPC_LINK rejects http:// URIs with: BadRequestException: integration uri should be a valid Cloud Map service ARN - Capture SERVICE_ID from create-service for the service ARN lookup - Also fix --task-definition your-bot:latest → your-bot (ECS family name) - Update architecture diagram, path passthrough callout, and DNS record descriptions to match the corrected configuration --- docs/refarch/running-telegram-line-on-aws.md | 52 ++++++++++++-------- 1 file changed, 31 insertions(+), 21 deletions(-) diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md index 7be83d29a..273e1f771 100644 --- a/docs/refarch/running-telegram-line-on-aws.md +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -74,8 +74,8 @@ API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcod │ AWS Cloud Map │ ~$0 (private DNS) │ namespace: oab │ │ service: your-bot │ - │ → DNS A record │ - │ resolves to task IP │ + │ → DNS SRV record │ + │ resolves to IP:port │ └───────────┬───────────┘ │ ▼ @@ -94,10 +94,12 @@ API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcod - The task's security group - AWS CLI or Console access -> **DNS record type: use A, not SRV** -> API Gateway HTTP API integration resolves DNS A records — SRV records do **not** work -> for VPC Link targets. Always use `{Type="A"}` in your Cloud Map service configuration. -> The port (`:8080`) is specified in the integration URI, not the DNS record. +> **DNS record type: use SRV, not A** +> API Gateway HTTP API VPC Link integrations require the Cloud Map service **ARN** +> (not an `http://` URL) as the integration URI. The port comes from the **SRV** record — +> A records do **not** carry port information and will fail with +> `BadRequestException: integration uri should be a valid Cloud Map service ARN`. +> Use `{Type="SRV"}` in your Cloud Map service configuration. ### Step-by-Step @@ -110,12 +112,13 @@ aws servicediscovery create-private-dns-namespace \ --vpc vpc-abc123 \ --region us-east-1 -# Service with DNS A record (auto-registers task IP) +# Service with DNS SRV record (auto-registers task IP + port) aws servicediscovery create-service \ --name your-bot \ --namespace-id ns-xxx \ - --dns-config 'NamespaceId="ns-xxx",DnsRecords=[{Type="A",TTL="60"}]' \ - --region us-east-1 + --dns-config 'NamespaceId="ns-xxx",DnsRecords=[{Type="SRV",TTL="60"}]' \ + --region us-east-1 \ + --query 'Service.Id' --output text # save this as SERVICE_ID ``` #### 2. Enable Service Discovery on Your ECS Service @@ -131,7 +134,7 @@ aws ecs delete-service --cluster oab --service your-bot --region us-east-1 aws ecs create-service \ --cluster oab \ --service-name your-bot \ - --task-definition your-bot:latest \ + --task-definition your-bot \ --desired-count 0 \ --launch-type FARGATE \ --network-configuration \ @@ -143,9 +146,9 @@ aws ecs create-service \ aws ecs update-service --cluster oab --service your-bot --desired-count 1 --region us-east-1 ``` -When the task starts, ECS auto-registers its private IP as the DNS A record for -`your-bot.oab`. Other services in the same VPC resolve this name to reach the task on -`:8080`. +When the task starts, ECS auto-registers its private IP and port as the DNS SRV record +for `your-bot.oab`. The SRV record carries both address and port, which the VPC Link +uses to route traffic. > **ECS task-def naming**: Use the family name (resolves to latest ACTIVE revision) > or `family:revision` (e.g., `your-bot:1`). ECS does not support Docker-style @@ -174,13 +177,19 @@ API_ID=$(aws apigatewayv2 create-api \ --region us-east-1 \ --query 'ApiId' --output text) -# Create the integration FIRST: VPC Link → Cloud Map service endpoint -# Endpoint = your-bot.oab:8080 (the Cloud Map DNS name) +# Get the Cloud Map service ARN (needed for the integration URI) +SERVICE_ARN=$(aws servicediscovery get-service \ + --id $SERVICE_ID \ + --region us-east-1 \ + --query 'Service.Arn' --output text) + +# Create the integration FIRST: VPC Link → Cloud Map service ARN +# The integration URI must be the Cloud Map service ARN (not an http:// URL) INTEGRATION_ID=$(aws apigatewayv2 create-integration \ --api-id $API_ID \ --integration-type HTTP_PROXY \ --integration-method POST \ - --integration-uri "http://your-bot.oab:8080" \ + --integration-uri "$SERVICE_ARN" \ --connection-type VPC_LINK \ --connection-id $VPC_LINK_ID \ --payload-format-version "1.0" \ @@ -216,13 +225,14 @@ https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/line ``` > **Path passthrough is automatic**: API Gateway HTTP API appends the route path to the -> integration URI. The route `POST /webhook/telegram` combined with integration URI -> `http://your-bot.oab:8080` results in requests hitting -> `http://your-bot.oab:8080/webhook/telegram` on the container. No path rewriting -> needed. +> integration target. The route `POST /webhook/telegram` results in +> `POST /webhook/telegram` hitting the container. No path rewriting needed. +> +> When the Cloud Map service is registered via ECS service discovery, the SRV record +> carries the container port — API Gateway does not need to specify it separately. This URL **never changes** — even when tasks restart and get new private IPs, Cloud Map -auto-updates the DNS record and the VPC Link resolves the new IP transparently. +auto-updates the DNS SRV record and the VPC Link resolves the new IP transparently. #### 5. Security Group: Inbound Rules From bb415976fbf8f0089cfe394b36346fa9c8e893f6 Mon Sep 17 00:00:00 2001 From: smallgun01 Date: Thu, 2 Jul 2026 13:51:55 +0800 Subject: [PATCH 4/6] docs: fix VPC Link cost + add oabctl cross-reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit F1: VPC Link is NOT free — ENI-hour billing (~$0.01/hr, ~$7/month). Updated cost breakdown (Option 1: $5-10 → $12-17), architecture diagram, pros/cons, multi-platform total, and summary table. F2: Add cross-reference to operator/README.md for oabctl ingress automation. --- docs/refarch/running-telegram-line-on-aws.md | 30 +++++++++++++------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md index 273e1f771..e389389ca 100644 --- a/docs/refarch/running-telegram-line-on-aws.md +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -32,7 +32,7 @@ traffic to your Fargate task's `:8080` without paying for an ALB.** | # | Option | Monthly Cost | Best For | |---|--------|-------------|----------| -| 1 | **API Gateway HTTP API + VPC Link + Cloud Map** | ~$5–10 | Single/multi bot, no ALB, lowest AWS-native cost | +| 1 | **API Gateway HTTP API + VPC Link + Cloud Map** | ~$12–17 | Single/multi bot, no ALB, lowest AWS-native cost | | 2 | **ALB + ECS Fargate** | ~$20+ | Health checks, auto-scaling, enterprise | | 3 | **Cloudflare Tunnel sidecar** | ~$5–10 | Already have Cloudflare, simplest setup | @@ -50,6 +50,11 @@ traffic to your Fargate task's `:8080` without paying for an ALB.** This is the cheapest AWS-native path. It replaces a $16+/month ALB with a ~$1/month API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcoded IPs. +> **Automation tip**: The `oabctl` operator (see [`operator/README.md`](../../operator/README.md)) +> automates this exact Option 1 setup — API Gateway + VPC Link + Cloud Map + +> security group — via a declarative `spec.ingress` block. Once you understand the +> manual steps below, consider using `oabctl apply` for production deployments. + ### Architecture Diagram ``` @@ -68,7 +73,7 @@ API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcod │ /webhook/telegram │ │ /webhook/line │ └───────────┬───────────┘ - │ VPC Link ($0 — free for HTTP API) + │ VPC Link (~$0.01/hr/ENI) ▼ ┌───────────────────────┐ │ AWS Cloud Map │ ~$0 (private DNS) @@ -291,13 +296,15 @@ for Telegram/LINE. | Resource | Monthly Cost | |----------|-------------| | API Gateway HTTP API | ~$1.00/million requests (effectively $0 for small bots) | -| VPC Link | $0 (free for HTTP API; hourly charge only applies to REST API) | +| VPC Link | ~$0.01/hr per ENI (~$7/month for a single ENI) | | Cloud Map | $0 (private DNS, minimal queries) | | ECS Fargate (256/512, 12h/day) | ~$5–10/month | -| **Total** | **~$5–10/month** | +| **Total** | **~$12–17/month** | -> The HTTP API VPC Link is free — unlike REST API VPC Links which charge ~$0.01/hour. -> This makes API Gateway the cheapest AWS-native ingress option for low-traffic bots. +> VPC Link pricing is based on ENI-hour billing (approximately $0.01/hour per ENI). +> REST API VPC Links are more expensive. This makes API Gateway the cheapest +> AWS-native ingress option for low-traffic bots, though not as close to zero as +> Cloudflare Tunnel. ### Pitfalls We Hit @@ -326,7 +333,7 @@ for Telegram/LINE. | Pros | Cons | |------|------| -| Cheapest AWS-native path (~$5–10/mo total) | VPC Link adds an extra network hop | +| Cheapest AWS-native path (~$12–17/mo total) | VPC Link adds an extra network hop | | Webhook URL never changes (API Gateway endpoint is static) | One extra service to manage (API Gateway) | | Same infra for Telegram + LINE + Discord | Initial setup has more moving parts | | No ALB, no Kubernetes, no Cloudflare dependency | Debugging requires checking VPC Link logs | @@ -389,8 +396,9 @@ Cloudflare, this is the simplest path. | ECS Fargate (256/512, 12h/day) | ~$5–10/month | | **Total** | **~$5–10/month** | -> Same total as Option 1 — the difference is Cloudflare vs AWS for the ingress layer, -> not the Fargate cost. Choose based on which ecosystem you already use. +> Cheaper than Option 1 (~$12–17) — the difference is Cloudflare (free tunnel) vs +> AWS VPC Link (~$7/month) for the ingress layer. Choose based on which ecosystem +> you already use. ### Architecture (ECS Fargate) @@ -537,7 +545,7 @@ POST /webhook/line ─┼── same VPC Link → same Cloud Map → same t ``` **Cost efficiency**: One task replaces three. The API Gateway + VPC Link is shared -across all platforms. Total: **~$5–10/month for Telegram + LINE + Discord on a single +across all platforms. Total: **~$12–17/month for Telegram + LINE + Discord on a single Fargate task.** --- @@ -546,7 +554,7 @@ Fargate task.** | Factor | Option 1 (API GW + CM) | Option 2 (ALB) | Option 3 (CF Tunnel) | |--------|----------------------|----------------|---------------------| -| Monthly cost | ~$5–10 | ~$18–21 | ~$5–10 | +| Monthly cost | ~$12–17 | ~$18–21 | ~$5–10 | | Setup complexity | Medium | Low | Low | | AWS-native | ✅ Yes | ✅ Yes | ❌ No (CF dependency) | | Health checks | Manual | ✅ Built-in | Manual | From 4ec2bfc5c99a23b8fccdb7b840e8508fa9755cbc Mon Sep 17 00:00:00 2001 From: smallgun01 Date: Mon, 6 Jul 2026 08:19:45 +0800 Subject: [PATCH 5/6] docs(refarch): address review F1 (webhook signature validation) + F2 (beta.4 version alignment) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit F1: Add TELEGRAM_SECRET_TOKEN guidance to keep Telegram webhook signature verification intact when exposing :8080 to the public internet. New Security callout in Option 1 §7 explains that LINE auto-verifies via LINE_CHANNEL_SECRET (OpenAB built-in HMAC-SHA256), while Telegram requires TELEGRAM_SECRET_TOKEN env var matching the setWebhook secret_token parameter. Applied to: - Option 1 §6 setWebhook command (now uses `curl` + `openssl rand`) - Option 1 §7 env var block + new Security callout - Option 3 (Cloudflare Tunnel) task-def secrets block - Multi-Platform adapter table F2: Align version note with docs/telegram.md (which states 'default since v0.9.0-beta.4'). Drop imprecise beta.2/beta.5/beta.6 references; keep the architecture advice version-agnostic. Signed-off-by: smallgun01 --- docs/refarch/running-telegram-line-on-aws.md | 50 +++++++++++++++----- 1 file changed, 37 insertions(+), 13 deletions(-) diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md index e389389ca..820a4a438 100644 --- a/docs/refarch/running-telegram-line-on-aws.md +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -36,12 +36,12 @@ traffic to your Fargate task's `:8080` without paying for an ALB.** | 2 | **ALB + ECS Fargate** | ~$20+ | Health checks, auto-scaling, enterprise | | 3 | **Cloudflare Tunnel sidecar** | ~$5–10 | Already have Cloudflare, simplest setup | -> **Version note (callout)** -> OpenAB beta.5+ supports **unified webhook mode** — set `TELEGRAM_BOT_TOKEN` or -> `LINE_CHANNEL_SECRET` + `LINE_CHANNEL_ACCESS_TOKEN` as env vars directly on your bot -> container, no separate gateway service needed. The infra paths below work identically -> from beta.2 through beta.6. The only difference: beta.2 required a standalone gateway -> service + `[gateway]` config block; beta.5+ does not. +> **Version note** +> OpenAB v0.9.0-beta.4+ ships **unified webhook mode** by default — set +> `TELEGRAM_BOT_TOKEN` or `LINE_CHANNEL_SECRET` + `LINE_CHANNEL_ACCESS_TOKEN` +> as env vars directly on your bot container, no separate gateway service +> needed. The infra paths below work the same regardless of how the platform +> adapter is wired in; only the deployment shape changes. --- @@ -264,8 +264,16 @@ The biggest pitfall when moving from Discord-only to Telegram/LINE. #### 6. Set Webhook URLs **Telegram (BotFather / setWebhook):** -``` -https://api.telegram.org/bot/setWebhook?url=https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/telegram + +Generate a random webhook secret first, then use the same value for both the env +var and the `secret_token` query parameter — Telegram echoes it back as the +`X-Telegram-Bot-Api-Secret-Token` header on every webhook, and OpenAB rejects +requests whose header doesn't match (see the **Security** callout in §7 below). + +```bash +TELEGRAM_SECRET_TOKEN="$(openssl rand -hex 32)" + +curl "https://api.telegram.org/bot/setWebhook?url=https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/telegram&secret_token=${TELEGRAM_SECRET_TOKEN}" ``` **LINE (LINE Developers Console):** @@ -275,11 +283,25 @@ Webhook URL: https://{api-id}.execute-api.us-east-1.amazonaws.com/prod/webhook/l #### 7. OpenAB Environment Variables +> **Security: webhook signature validation** +> Exposing `:8080` to the public internet means anyone who discovers your webhook +> URL can POST forged events. Both platforms ship built-in defenses, but they +> require different config: +> +> - **LINE**: OpenAB automatically verifies the `X-Line-Signature` header using +> `LINE_CHANNEL_SECRET` (HMAC-SHA256). No extra config needed — just set the +> env var. +> - **Telegram**: Set `TELEGRAM_SECRET_TOKEN` (1–256 chars, alphanumeric + `-_`). +> Telegram sends it back as `X-Telegram-Bot-Api-Secret-Token` on every webhook; +> OpenAB rejects requests whose header doesn't match. The same value **must** +> be passed as `secret_token` to the `setWebhook` call above. + ```bash -# Telegram +# Telegram (secret_token must match the setWebhook call in §6) TELEGRAM_BOT_TOKEN=123:abc +TELEGRAM_SECRET_TOKEN="$(openssl rand -hex 32)" -# LINE +# LINE (LINE_CHANNEL_SECRET is used for automatic webhook signature verification) LINE_CHANNEL_SECRET=xxx LINE_CHANNEL_ACCESS_TOKEN=yyy @@ -287,9 +309,9 @@ LINE_CHANNEL_ACCESS_TOKEN=yyy DISCORD_BOT_TOKEN=zzz ``` -With unified webhook mode (beta.5+), the OpenAB binary auto-detects which env vars are -set and starts the corresponding platform adapters. No `[gateway]` config block needed -for Telegram/LINE. +With unified webhook mode (v0.9.0-beta.4+), the OpenAB binary auto-detects which +env vars are set and starts the corresponding platform adapters. No `[gateway]` +config block needed for Telegram/LINE. ### Cost Breakdown @@ -466,6 +488,7 @@ aws secretsmanager create-secret \ "portMappings": [{"containerPort": 8080}], "secrets": [ {"name": "TELEGRAM_BOT_TOKEN", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/telegram-bot-token:token::"}, + {"name": "TELEGRAM_SECRET_TOKEN", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/telegram-bot-token:TELEGRAM_SECRET_TOKEN::"}, {"name": "LINE_CHANNEL_SECRET", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/line-shared:LINE_CHANNEL_SECRET::"}, {"name": "LINE_CHANNEL_ACCESS_TOKEN", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456:secret:openab/line-shared:LINE_CHANNEL_ACCESS_TOKEN::"} ], @@ -535,6 +558,7 @@ The binary auto-detects which env vars are present and starts each platform adap |----------------|---------|-------------| | `DISCORD_BOT_TOKEN` | Discord | (outbound WS, no path) | | `TELEGRAM_BOT_TOKEN` | Telegram | `/webhook/telegram` | +| `TELEGRAM_SECRET_TOKEN` | (Telegram only) | **Recommended.** Verifies `X-Telegram-Bot-Api-Secret-Token` on inbound webhooks; must match the `secret_token` passed to `setWebhook`. | | `LINE_CHANNEL_SECRET` + `LINE_CHANNEL_ACCESS_TOKEN` | LINE | `/webhook/line` | **API Gateway routing**: Add one route per platform, all pointing to the same VPC Link: From 704abf390339d0df16b59b291e428f6314fe77b2 Mon Sep 17 00:00:00 2001 From: smallgun01 Date: Tue, 7 Jul 2026 20:53:09 +0800 Subject: [PATCH 6/6] docs: strengthen oabctl recommendation in Option 1 after live E2E Verified oabctl ingress on us-east-1 (2026-07-07): - One provisions the full stack in ~2.5min - Replaces all 7 manual AWS CLI steps - Two pitfalls documented: us-east-1e AZ unsupported, configFrom required Switches from 'automation tip' to 'Recommended' callout at Option 1 top. --- docs/refarch/running-telegram-line-on-aws.md | 26 +++++++++++++++++--- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/docs/refarch/running-telegram-line-on-aws.md b/docs/refarch/running-telegram-line-on-aws.md index 820a4a438..2e794047d 100644 --- a/docs/refarch/running-telegram-line-on-aws.md +++ b/docs/refarch/running-telegram-line-on-aws.md @@ -50,10 +50,28 @@ traffic to your Fargate task's `:8080` without paying for an ALB.** This is the cheapest AWS-native path. It replaces a $16+/month ALB with a ~$1/month API Gateway HTTP API and uses Cloud Map for service discovery instead of hardcoded IPs. -> **Automation tip**: The `oabctl` operator (see [`operator/README.md`](../../operator/README.md)) -> automates this exact Option 1 setup — API Gateway + VPC Link + Cloud Map + -> security group — via a declarative `spec.ingress` block. Once you understand the -> manual steps below, consider using `oabctl apply` for production deployments. +> **✅ Recommended: use `oabctl`** — one `oabctl apply -f` command provisions this +> entire Option 1 stack (Cloud Map → VPC Link → API Gateway → routes → stage) in +> ~2.5 minutes, replacing all 7 manual steps below. Tested and verified on +> `us-east-1` (2026-07-07). +> +> ```bash +> # Minimal manifest (save as bot.yaml, then oabctl apply -f bot.yaml): +> spec: +> ingress: +> type: apigateway +> paths: +> - /webhook/telegram +> ``` +> +> See [`docs/oabctl.md`](../oabctl.md#ingress--inbound-webhooks-telegram--line) +> for the full ingress reference. Known pitfalls: +> - `us-east-1e` (use1-az3) does **not** support API Gateway VPC Link — avoid +> subnets in that AZ. +> - `configFrom` (S3 config path) is required even for infra-only testing. +> +> The manual CLI steps below are preserved for readers who want to understand +> the underlying AWS plumbing or who are not using `oabctl`. ### Architecture Diagram