devops-terraform-buildkite-scripts

Modern Terraform infrastructure deployment pipeline using Buildkite, Clivern Lynx state backend, and comprehensive security scanning.

---

📋 Table of Contents

• Overview
• Features
• Architecture
• Prerequisites
• Quick Start
• Project Structure
• Pipeline Stages
• Configuration
• Security
• Deployment
• Troubleshooting
• Contributing
• License

---

🎯 Overview

This repository contains Terraform infrastructure-as-code that deploys and manages cloud resources using a modern CI/CD pipeline with:

• 🏗️ Infrastructure as Code: OpenTofu/Terraform for declarative infrastructure
• 🔄 CI/CD Automation: Buildkite for pipeline orchestration
• 💾 State Management: Clivern Lynx HTTP backend with distributed locking
• 🔐 Security: Comprehensive scanning with TFSec, Checkov, Terrascan, KICS
• 🛡️ Compliance: Mondoo security posture management
• 🔑 Secrets: HashiCorp Vault for secure credential management
• 🤖 AI Analysis: Fabric AI and Overmind for plan analysis

---

✨ Features

🚀 Deployment

• ✅ Multi-environment support (dev, staging, production)
• ✅ Progressive deployment with approval gates
• ✅ Automated state backups before changes
• ✅ Rollback capabilities
• ✅ Environment-specific configurations

🔒 Security

• ✅ TFSec - Terraform static analysis
• ✅ Checkov - Policy-as-code scanning
• ✅ Terrascan - Infrastructure security scanner
• ✅ KICS - Comprehensive IaC scanner
• ✅ GitGuardian - Secret detection
• ✅ Mondoo - Security posture management
• ✅ Compliance score enforcement (minimum 85%)

🤖 Intelligent Analysis

• ✅ Fabric AI - Natural language plan analysis
• ✅ Overmind - Blast radius calculation
• ✅ Change impact assessment
• ✅ Risk scoring and recommendations

🏗️ Infrastructure

• ✅ Module-based architecture
• ✅ Reusable Terraform modules
• ✅ Version-pinned dependencies
• ✅ Automated documentation generation
• ✅ Drift detection

---

🏛️ Architecture

🏛️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Buildkite CI/CD                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐        │
│  │ Validation   │→ │  Security    │→ │  Deployment  │        │
│  │ & Formatting │  │  Scanning    │  │  Pipeline    │        │
│  └──────────────┘  └──────────────┘  └──────────────┘        │
└────────────────────────────┬────────────────────────────────────┘
                             │
                 ┌───────────┴───────────┐
                 ▼                       ▼
        ┌─────────────────┐     ┌─────────────────┐
        │  HashiCorp      │     │  Clivern Lynx   │
        │  Vault          │     │  State Backend  │
        │  (Secrets)      │     │  (HTTP + Lock)  │
        └─────────────────┘     └─────────────────┘
                 │                       │
                 └───────────┬───────────┘
                             ▼
                    ┌─────────────────┐
                    │   Cloud         │
                    │   Infrastructure│
                    │   (AWS/Azure)   │
                    └─────────────────┘

---

📦 Prerequisites

Required Tools

Tool	Minimum Version	Purpose
OpenTofu	1.6.0+	Infrastructure provisioning
PowerShell	7.0+	Script execution
Buildkite Agent	3.x+	CI/CD pipeline execution
Vault CLI	1.15.0+	Secret management

Optional Tools

Tool	Purpose
Fabric AI	AI-powered plan analysis
Overmind CLI	Blast radius analysis
terraform-docs	Documentation generation

Access Requirements

• ✅ HashiCorp Vault access with JWT authentication
• ✅ Clivern Lynx HTTP backend endpoint
• ✅ Buildkite organization and pipeline
• ✅ Cloud provider credentials (AWS/Azure/GCP)
• ✅ Mondoo service account token

---

🚀 Quick Start

1. Clone Repository

git clone https://github.com/yourorg/terraform-infrastructure.git
cd terraform-infrastructure

2. Configure Pipeline

Edit .buildkite/pipeline.yml:

env:
  PROJECT_NAME: "my-infrastructure"
  SERVICE_NAME: "my-service"
  TARGET_ENVIRONMENTS: "dev,stg,prd"
  VAULT_NAMESPACE: "DevOps/prd/my-project"
  LYNX_BASE_URL: "https://lynx.company.com"

3. Set Up Vault Secrets

# Lynx backend credentials
vault kv put secret/lynx/terraform \
  username="terraform-user" \
  password="secure-password"

# Mondoo token
vault kv put secret/mondoo \
  token="your-mondoo-token"

4. Create Environment Directories

mkdir -p dev stg prd
cd dev

cat > main.tf <<EOF
terraform {
  required_version = ">= 1.6.0"
  
  backend "http" {
    # Configured by Initialize-TofuBackend
  }
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"
  
  name = "my-vpc"
  cidr = "10.0.0.0/16"
}
EOF

5. Commit and Push

git add .
git commit -m "feat: initial infrastructure setup"
git push origin main

6. Watch Pipeline Execute

Visit your Buildkite dashboard to see the pipeline run! 🎉

---

📁 Project Structure