Merge #20: [#14] Phase 2: Twelve-Factor App Refactoring - Persistent Data Volume Implementation

21df7d8 cleanup: remove backup Makefile files (Jose Celano)
ab2e551 cleanup: remove obsolete deployment and installation scripts (Jose Celano)
d3e9ed5 refactor: simplify deployment script configuration file handling (Jose Celano)
9877fa6 refactor: move Docker Compose .env to persistent volume (Jose Celano)
ba4d174 fix: [#14] copy generated configuration files to persistent volume (Jose Celano)
a0062c9 docs: [#14] add comprehensive persistent volume documentation (Jose Celano)
2cc9fb8 refactor: [#14] standardize environment file path structure (Jose Celano)
ed2c45a style: [#14] remove trailing spaces from multi-provider abstraction README (Jose Celano)
f13b08d feat: [#14] implement persistent data volume for VM data persistence (Jose Celano)
1f49a67 docs: [#14] add multi-provider infrastructure abstraction plan (Jose Celano)
bc9a9e2 feat: add git status warning and environment config documentation (Jose Celano)
15e142d refactor: [#1] rename local.env to local.env.tpl for consistency (Jose Celano)
ac254c4 feat: [#1] improve API error reporting and complete token standardization (Jose Celano)
d6e6a11 fix: [#1] update default API token from local-dev-admin-token-12345 to MyAccessToken in local environment (Jose Celano)
cf05572 docs: [#14] update twelve-factor refactoring status to reflect current implementation (Jose Celano)

Pull request description:

  ## 🎯 Overview

  This pull request implements **Phase 2** of the twelve-factor app refactoring for the Torrust Tracker Demo, focusing on persistent data volume implementation and deployment script simplification.

  Resolves #14

  ## 📋 Summary of Changes

  ### 🗄️ **Persistent Data Volume Implementation**
  - **Centralized Storage**: All persistent application data now stored in `/var/lib/torrust/` volume
  - **Data Separation**: Clean separation between application code and persistent data
  - **Volume Structure**: Organized subdirectories for tracker, database, monitoring, and proxy configurations

  ### 🚀 **Deployment Script Simplification**
  - **63 lines of code removed**: Eliminated complex backup/restore logic during repository updates
  - **Direct Transfer**: Configuration files copied directly from local machine to persistent volume via SCP
  - **Streamlined Flow**: Local generation → Direct transfer → Service start (no intermediate steps)
  - **Error Reduction**: Fewer file operations and reduced chance of deployment failures

  ### 📁 **Configuration Management**
  - **Local Generation**: All config files generated locally using environment templates
  - **Direct Placement**: Configuration files placed directly in final persistent volume locations
  - **Template System**: Proper environment-based configuration rendering

  ## 🔧 Technical Implementation

  ### Persistent Volume Structure
  ```
  /var/lib/torrust/
  ├── tracker/
  │   ├── etc/tracker.toml          # Tracker configuration
  │   ├── lib/database/             # Database storage
  │   └── log/                      # Tracker logs
  ├── prometheus/
  │   ├── data/                     # Prometheus time-series data
  │   └── etc/prometheus.yml        # Prometheus configuration
  ├── proxy/
  │   ├── webroot/                  # Nginx webroot
  │   └── etc/nginx-conf/nginx.conf # Nginx configuration
  ├── mysql/
  │   └── init/                     # MySQL initialization scripts
  ├── certbot/
  │   ├── etc/                      # SSL certificates
  │   └── lib/                      # Certbot data
  ├── dhparam/                      # DH parameters
  └── compose/
      └── .env                      # Docker Compose environment
  ```

  ### Deployment Workflow Improvements
  - **Before**: Repository → Storage backup → Configuration → Restore → Deploy
  - **After**: Repository → Configuration direct copy → Deploy

  ### Configuration File Management
  - Local generation via `infrastructure/scripts/configure-env.sh`
  - Direct SCP transfer to `/var/lib/torrust/*/` locations
  - Proper ownership and permissions set during transfer

  ## ✅ Testing & Validation

  ### Test Coverage
  - [x] **Unit Tests**: All infrastructure and application layer tests pass
  - [x] **Integration Tests**: Complete E2E deployment workflow validated
  - [x] **Deployment Tests**: Services start correctly and health checks pass
  - [x] **Persistent Data**: Data survives container restarts and redeployments

  ### Validation Results
  - All Docker Compose services start successfully
  - Health check endpoints respond correctly
  - Persistent data maintained across deployments
  - Configuration files correctly generated and deployed

  ## 🔄 **Twelve-Factor Compliance Status**

  This PR fully implements **Factors 1-12**:

  | Factor | Status | Implementation |
  |--------|--------|----------------|
  | 1. Codebase | ✅ Complete | Git repository with proper branching |
  | 2. Dependencies | ✅ Complete | Docker Compose service definitions |
  | 3. Config | ✅ Complete | Environment-based template system |
  | 4. Backing Services | ✅ Complete | MySQL, Prometheus as attached resources |
  | 5. Build/Release/Run | ✅ Complete | Clean separation with `make infra-apply` + `make app-deploy` |
  | 6. Processes | ✅ Complete | Stateless containers with persistent volume |
  | 7. Port Binding | ✅ Complete | Service export via defined ports |
  | 8. Concurrency | ✅ Complete | Docker Compose orchestration |
  | 9. Disposability | ✅ Complete | Fast startup/graceful shutdown |
  | 10. Dev/Prod Parity | ✅ Complete | Environment template system |
  | 11. Logs | ✅ Complete | Docker logging + persistent log storage |
  | 12. Admin Processes | ✅ Complete | Health checks and monitoring tools |

  ## 📖 **Key Commits**

  1. **`f13b08d`**: Implement persistent data volume for VM data persistence
  2. **`2cc9fb8`**: Standardize environment file path structure
  3. **`ba4d174`**: Copy generated configuration files to persistent volume
  4. **`9877fa6`**: Move Docker Compose .env to persistent volume
  5. **`d3e9ed5`**: Simplify deployment script configuration file handling

  ## 🚀 **Benefits**

  - **Reliability**: Simplified deployment with fewer failure points
  - **Maintainability**: Clean separation of concerns between code and data
  - **Performance**: Persistent data survives redeployments without regeneration
  - **Debugging**: Easier to troubleshoot with direct file transfers
  - **Scalability**: Foundation for multi-environment deployments

  ## 🔄 **Migration Impact**

  - **Breaking Changes**: None - new deployments work seamlessly
  - **Data Migration**: Automatic - persistent data properly handled
  - **Rollback**: Simple branch revert if needed

  ## 📝 **Next Steps**

  This completes Phase 2 of the twelve-factor refactoring. Future phases will include:
  - Production environment deployment (Hetzner migration)
  - SSL certificate automation
  - Multi-environment configuration management
  - Performance optimization

  ---

  **Ready for review and merge into `main`** 🚀

Top commit has no ACKs.

Tree-SHA512: 54d33fe49e3ae8820b791093e9ca9807626b8957e2492bf33b481e5ac1e08698cb143efc9893846306bb3392b2fdb7bd4abc9ef3897816e43dcc3c19fa13f763

Author: josecelano

.github/copilot-instructions.md

@@ -40,6 +40,48 @@ We are migrating the tracker to a new infrastructure on Hetzner, involving:
 - Migrating the database from SQLite to MySQL
 - Implementing Infrastructure as Code for reproducible deployments
 
+## 🏗️ Twelve-Factor Architecture
+
+This project implements a complete twelve-factor app architecture with clear separation between infrastructure provisioning and application deployment:
+
+### Current Working Architecture (Twelve-Factor Compliant)
+
+```text
+┌─────────────────────────────────────────────────────────────┐
+│                 Configuration Management                    │
+├─────────────────────────────────────────────────────────────┤
+│  • Environment Templates (local.env.tpl, production.env.tpl)  │
+│  • Configuration Processing (configure-env.sh)              │
+│  • Template Rendering (.tpl → actual configs)               │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                     Infrastructure Layer                    │
+├─────────────────────────────────────────────────────────────┤
+│  • VM Provisioning (provision-infrastructure.sh)            │
+│  • Environment-specific Setup (templated cloud-init)        │
+│  • Provider Abstraction (local implemented, cloud planned)  │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                    Application Layer                        │
+├─────────────────────────────────────────────────────────────┤
+│  • Environment-aware Deployment (templated configs)         │
+│  • Dynamic Service Configuration                            │
+│  • Comprehensive Health Validation                          │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Key Implementation Features
+
+- **Complete Twelve-Factor Compliance**: All 12 factors implemented and operational
+- **Infrastructure/Application Separation**: Clean separation with `make infra-apply` and `make app-deploy`
+- **Environment-based Configuration**: Template system with environment switching
+- **Build/Release/Run Stages**: Proper separation of configuration processing, deployment, and execution
+- **Multi-environment Support**: Local development with production parity
+
 ## 📁 Repository Structure
 
 ```text