Skip to content

Repository navigation

[DevOps] Migrate EC2 deployment from SSH to AWS Systems Manager #9

New issue
New issue
Closed
Closed
[DevOps] Migrate EC2 deployment from SSH to AWS Systems Manager#9
Assignees
ayushcmd
Labels
CI/CD

Description

@ayushcmd
ayushcmd
opened on Oct 27, 2026

Outcome

Completed and verified on 1 November 2026.

CloudMonitor's production deployment path has been migrated from runner-originated SSH to GitHub Actions OIDC plus AWS Systems Manager Run Command.

The first controlled deployment through the SSM workflow succeeded. GitHub Actions assumed the dedicated deployment role, verified that the EC2 instance was Online in Systems Manager, invoked AWS-RunShellScript, deployed the main revision and passed production health checks.

Acceptance criteria

  • EC2 is registered and Online in Systems Manager.
  • GitHub Actions can securely assume the AWS deployment role through OIDC and invoke the approved SSM command.
  • GitHub-hosted runners do not require inbound SSH access to deploy.
  • Production deployment and frontend/backend/Prometheus health checks succeed.
  • PostgreSQL data and the existing named volume remain intact.
  • Existing HTTP application access continues to work; HTTPS/TLS remains tracked separately under Issue [Security] Harden CloudMonitor for production #12.
  • AWS roles, trust relationship, deployment policy and rollback procedure are documented.
  • Rollback and troubleshooting steps are documented.

Production verification

  • Deployed commit: f536dad97518a761c12b83ff578ff3194e9ed04f
  • Frontend: HTTP 200.
  • Backend: /health returned status=healthy.
  • PostgreSQL: healthy.
  • Prometheus: required targets alertmanager, cloudmonitor-backend, node-exporter and prometheus all reported up.
  • PostgreSQL volume: cloudmonitor-prod_postgres_data, preserved.
  • Alembic revision checked before deployment: 003_os_and_version.

Safety and remaining scope

  • The GitHub Actions CD workflow is manual-only (workflow_dispatch). Deploy and rollback require explicit production confirmation.
  • The workflow does not run docker compose down -v.
  • The rollback handler restores application source/build state; it does not restore PostgreSQL data or reverse database migrations.
  • Restricted administrator SSH remains a break-glass path; GitHub-hosted runners no longer need inbound SSH for deployment.
  • Broader production-security items, including HTTPS/TLS, unauthenticated Prometheus read access and agent Docker-socket hardening, remain in Issue #12.

Documentation

This issue is closed as completed.

Activity

  1. ayushcmd
    added
    CI/CD
    on Oct 27, 2026
  2. ayushcmd
    self-assigned this
    on Oct 30, 2026

  3. ayushcmd commented on Oct 31, 2026

    @ayushcmd
    ayushcmd
    on Oct 31, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    Implementation prepared in PR #22: #22. The SSH-based CD workflow is replaced on a separate branch with a manual-only OIDC + SSM workflow (oidc-check, ssm-preflight, confirmed deploy, confirmed rollback) and an AWS setup/runbook. PR CI passed (backend/agent, frontend, Compose). AWS-side setup and live verification are still pending: EC2 instance profile/SSM Agent Online, GitHub OIDC role and environment variables, read-only SSM preflight, then a controlled deployment and rollback test. The workflow remains manual-only until those steps are complete; no AWS resources have been changed and no production deployment was run. Keep administrator SSH as a break-glass path.

  4. ayushcmd commented on Nov 1, 2026

    @ayushcmd
    ayushcmd
    on Nov 1, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    Completed — production SSM deployment verified

    The SSH-based GitHub Actions deployment path has been replaced with GitHub OIDC + AWS Systems Manager Run Command for CloudMonitor production.

    Verified outcomes

    • EC2 instance profile role CloudMonitorEC2SSMRole attached with AmazonSSMManagedInstanceCore.
    • SSM Agent diagnostics passed and CloudMonitor appeared Online in Systems Manager Fleet Manager.
    • GitHub OIDC provider token.actions.githubusercontent.com uses audience sts.amazonaws.com.
    • Dedicated deployment role CloudMonitorGitHubDeployRole successfully assumed by GitHub Actions using OIDC.
    • SSM preflight completed successfully.
    • Production deployment workflow completed successfully from main.
    • Deployed commit and last-successful-deploy-sha matched: f536dad97518a761c12b83ff578ff3194e9ed04f.
    • Frontend returned HTTP 200; backend reported status=healthy; the four required Prometheus targets (Prometheus, backend, Node Exporter, Alertmanager) reported UP.
    • PostgreSQL remained healthy and the existing named volume cloudmonitor-prod_postgres_data was preserved.
    • A fresh PostgreSQL custom-format backup was created and validated before deployment.

    Operational notes

    • The current CD workflow is intentionally manual via workflow_dispatch; after starting a deploy operation and confirming production, the deployment and health checks run automatically through SSM. A push to main does not currently auto-deploy.
    • GitHub-hosted runners no longer require inbound SSH connectivity to EC2. Keep trusted administrator SSH/break-glass access until separately reviewed; this issue does not require removing human access.
    • Production currently serves HTTP. HTTPS/TLS and remaining production security hardening are tracked separately under Issue [Security] Harden CloudMonitor for production #12.

    A detailed button-by-button IAM, OIDC, GitHub Environment, and SSM operations guide is being added under docs/ as the follow-up documentation change.

    Closing this migration issue as completed.

  5. ayushcmd
    closed this as completedon Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

CI/CD

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions