feat: [#14] Complete integration test refactoring and verification

- Fixed integration test script git archive extraction logic
- Added .env file transfer to VM in integration test setup
- Updated API endpoint testing to use correct authentication tokens
- Fixed test endpoints to use nginx proxy (port 80) instead of internal ports
- Updated integration testing guide with successful test results and critical details
- Added authentication requirements and correct API usage examples
- Documented network architecture and port access patterns

Integration tests now pass completely:
✅ VM deployment and SSH access
✅ Docker services (MySQL, Tracker, Prometheus, Grafana, Nginx)
✅ API endpoints with authentication (health check, stats API)
✅ UDP tracker ports (6868, 6969) listening
✅ Monitoring services health checks

This completes the 12-factor app refactoring with working end-to-end validation.

Author: josecelano

docs/guides/integration-testing-guide.md

@@ -1543,60 +1543,77 @@ This guide provides a complete integration testing workflow that:
 
 ### Integration Testing Results Summary
 
+✅ **INTEGRATION TESTS NOW PASS COMPLETELY!**
+
 This guide provides a complete integration testing workflow that:
 
 1. **Creates fresh infrastructure** in ~3-5 minutes
-2. **Generates configuration files** from templates (~2 seconds)
+2. **Generates configuration files** from templates (~2 seconds)  
 3. **Refreshes OpenTofu state** to detect VM IP (~3 seconds)
 4. **Waits for cloud-init** to complete (~2-3 minutes)
 5. **Runs comprehensive tests** covering all services (~3-5 minutes)
 6. **Performs external smoke testing** using official Torrust client tools (~2-3 minutes)
 7. **Verifies end-to-end functionality** of the Torrust Tracker
 8. **Cleans up resources** when complete (~1 minute)
 
-### Successful Test Results
+**Total Time**: ~8-12 minutes for complete cycle
+
+### ✅ Successful Test Results (Latest Run)
+
+During the most recent testing cycle, the following components were validated successfully:
+
+#### Infrastructure Tests
+
+- ✅ **VM Access**: SSH connectivity working at `192.168.122.54`
+- ✅ **Docker Installation**: Docker 28.3.1 and Docker Compose V2.38.1 working  
+- ✅ **Service Health**: All containers running with healthy status
+
+#### Service Deployment  
 
-During this testing cycle, the following components were validated:
+- ✅ **MySQL**: Database running healthy with proper credentials
+- ✅ **Tracker**: Torrust Tracker running with all endpoints active
+- ✅ **Prometheus**: Metrics collection working
+- ✅ **Grafana**: Dashboard service healthy (version 11.4.0)
+- ✅ **Nginx Proxy**: Reverse proxy routing working correctly
 
-#### ✅ Infrastructure Tests
+#### API and Endpoint Tests
 
-- VM deployment and configuration
-- Cloud-init provisioning
-- SSH connectivity and authentication
-- Docker and Docker Compose installation
+- ✅ **Health Check API**: `{"status":"Ok"}` via nginx proxy on port 80
+- ✅ **Statistics API**: Full stats JSON with admin token authentication
+- ✅ **UDP Tracker Ports**: 6868 and 6969 listening on both IPv4 and IPv6
+- ✅ **Monitoring Services**: Grafana and Prometheus both healthy
 
-#### ✅ Service Integration Tests
+#### Final Test Output
 
-- All Docker services startup (tracker, mysql, prometheus, grafana, nginx)
-- Service health checks and inter-service communication
-- Environment configuration and secrets management
+```console
+[SUCCESS] All integration tests passed!
+```
 
-#### ✅ API and Endpoint Tests
+### Critical Configuration Details
 
-- Health check API: `{"status":"Ok"}`
-- Stats API with authentication: Complete metrics data
-- Nginx proxy configuration with correct headers
+#### Authentication Requirements
 
-#### ✅ External Smoke Tests (Official Client Tools)
+- **Health Check API**: `/api/health_check` - No authentication required
+- **Stats API**: `/api/v1/stats` - **Requires authentication token**
+- **Admin Token**: `local-dev-admin-token-12345` (from `.env` file)
 
-- **UDP Tracker (6868)**: `{"AnnounceIpv4": {"seeders": 2, "peers": ["192.168.122.1:17548"]}}`
-- **UDP Tracker (6969)**: `{"AnnounceIpv4": {"seeders": 4, "peers": [...]}}`
-- **HTTP Tracker (via proxy)**: `{"complete": 1, "interval": 120, "peers": []}`
-- **Comprehensive Checker**: All operations (Setup, Connect, Announce, Scrape) successful
+#### Correct API Testing Examples
 
-#### ✅ Performance Validation
+```bash
+# Health check (no auth needed)
+curl -s http://$VM_IP/api/health_check | jq .
 
-- **UDP Response Time**: < 160ms
-- **HTTP Response Time**: < 187ms
-- **All protocols**: Sub-second response times under test load
+# Stats API (auth required) 
+curl -s "http://$VM_IP/api/v1/stats?token=local-dev-admin-token-12345" | jq .
+```
 
-### Critical Configuration Fixes Applied
+#### Network Architecture
 
-During testing, several critical configuration issues were identified and resolved:
+The deployment uses **nginx proxy** on port 80 that routes to internal services:
 
-1. **Tracker Visibility**: Changed `private = true` to `private = false` in tracker.toml
-2. **Nginx Proxy Headers**: Added missing `X-Forwarded-For` headers for HTTP tracker functionality
-3. **Authentication Setup**: Validated admin token configuration for API access
+- `/api/*` → routes to tracker service (internal port 1212)
+- Internal Docker ports (1212, 7070, 9090) are NOT accessible from outside the VM
+- UDP ports (6868, 6969) are directly exposed for tracker protocol
 
 ### Key Lessons Learned
 

infrastructure/scripts/validate-config.sh

@@ -189,8 +189,45 @@ validate_environment_config() {
     log_info "Validating environment-specific configuration..."
 
     case "${ENVIRONMENT}" in
-    "local" | "production")
-        # Both environments use the same sensible defaults
+    "local")
+        # Local environment allows public mode for integration testing
+        if grep -q 'threshold = "info"' "${tracker_config}"; then
+            [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: Info logging enabled"
+        else
+            log_error "${ENVIRONMENT}: Info logging not enabled"
+            return 1
+        fi
+
+        if grep -q 'on_reverse_proxy = true' "${tracker_config}"; then
+            [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: Reverse proxy enabled"
+        else
+            log_error "${ENVIRONMENT}: Reverse proxy should be enabled"
+            return 1
+        fi
+
+        if grep -q 'private = false' "${tracker_config}"; then
+            [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: Public tracker mode enabled (for integration testing)"
+        else
+            log_error "${ENVIRONMENT}: Public tracker mode should be enabled for integration testing"
+            return 1
+        fi
+
+        if grep -q 'driver = "mysql"' "${tracker_config}"; then
+            [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: MySQL database configured"
+        else
+            log_error "${ENVIRONMENT}: MySQL database not configured"
+            return 1
+        fi
+
+        if grep -q 'external_ip = "0.0.0.0"' "${tracker_config}"; then
+            [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: External IP set to 0.0.0.0"
+        else
+            log_warning "${ENVIRONMENT}: External IP not set to 0.0.0.0 (this may be intentional)"
+        fi
+        ;;
+
+    "production")
+        # Production environment requires private mode for security
         if grep -q 'threshold = "info"' "${tracker_config}"; then
             [[ "${VERBOSE}" == "true" ]] && log_info "${ENVIRONMENT}: Info logging enabled"
         else

infrastructure/tests/test-integration.sh

@@ -175,9 +175,8 @@ setup_torrust_tracker() {
         return 1
     fi
 
-    # Extract archive on VM
-    vm_exec "${vm_ip}" "cd /home/torrust/github/torrust && tar -xzf /tmp/$(basename "${temp_archive}") && mv torrust-tracker-demo torrust-tracker-demo-temp || true" "Extracting archive"
-    vm_exec "${vm_ip}" "cd /home/torrust/github/torrust && mv torrust-tracker-demo-temp torrust-tracker-demo" "Moving to final location"
+    # Extract archive on VM (git archive doesn't create parent directory)
+    vm_exec "${vm_ip}" "cd /home/torrust/github/torrust && mkdir -p torrust-tracker-demo && cd torrust-tracker-demo && tar -xzf /tmp/$(basename "${temp_archive}")" "Extracting archive"
     vm_exec "${vm_ip}" "rm -f /tmp/$(basename "${temp_archive}")" "Cleaning up archive"
 
     # Clean up local temp file
@@ -223,7 +222,7 @@ setup_torrust_tracker() {
 
     log_success "Installation script completed successfully"
 
-    # Step 5: Copy the configured storage folder to the VM
+    # Step 5: Copy the configured storage folder and .env file to the VM
     log_info "Copying configured storage folder to VM..."
 
     # Ensure storage directory exists and has proper structure
@@ -241,6 +240,15 @@ setup_torrust_tracker() {
         return 1
     fi
 
+    # Copy .env file to VM
+    log_info "Copying .env file to VM..."
+    if ! scp -o StrictHostKeyChecking=no \
+        "${PROJECT_ROOT}/application/.env" \
+        "torrust@${vm_ip}:/home/torrust/github/torrust/torrust-tracker-demo/application/.env"; then
+        log_error "Failed to copy .env file to VM"
+        return 1
+    fi
+
     # Verify critical configuration files exist on VM
     log_info "Verifying configuration files on VM..."
 
@@ -307,25 +315,25 @@ test_tracker_endpoints() {
     local vm_ip
     vm_ip=$(get_vm_ip)
 
-    # Test HTTP API endpoint
+    # Test HTTP API endpoint through nginx proxy using Host header
     log_info "Testing HTTP API endpoint..."
-    if vm_exec "${vm_ip}" "curl -f -s http://localhost:7070/api/v1/stats" "Checking HTTP API"; then
+    if vm_exec "${vm_ip}" "curl -f -s -H 'Host: tracker.torrust-demo.com' http://localhost:80/api/health_check" "Checking HTTP API"; then
         log_success "HTTP API is responding"
     else
         log_error "HTTP API is not responding"
         return 1
     fi
 
-    # Test metrics endpoint
-    log_info "Testing metrics endpoint..."
-    if vm_exec "${vm_ip}" "curl -f -s http://localhost:1212/metrics" "Checking metrics endpoint"; then
-        log_success "Metrics endpoint is responding"
+    # Test tracker statistics API
+    log_info "Testing tracker statistics API..."
+    if vm_exec "${vm_ip}" "curl -f -s -H 'Host: tracker.torrust-demo.com' 'http://localhost:80/api/v1/stats?token=local-dev-admin-token-12345'" "Checking statistics API"; then
+        log_success "Statistics API is responding"
     else
-        log_error "Metrics endpoint is not responding"
+        log_error "Statistics API is not responding"
         return 1
     fi
 
-    # Test if UDP ports are listening
+    # Test if UDP ports are listening (these are directly exposed)
     log_info "Testing UDP tracker ports..."
     if vm_exec "${vm_ip}" "ss -ul | grep -E ':6868|:6969'" "Checking UDP ports"; then
         log_success "UDP tracker ports are listening"
@@ -343,21 +351,21 @@ test_monitoring() {
     local vm_ip
     vm_ip=$(get_vm_ip)
 
-    # Test Prometheus
-    log_info "Testing Prometheus..."
-    if vm_exec "${vm_ip}" "curl -f -s http://localhost:9090/-/healthy" "Checking Prometheus health"; then
-        log_success "Prometheus is healthy"
+    # Test Grafana through nginx proxy using Host header
+    log_info "Testing Grafana..."
+    if vm_exec "${vm_ip}" "curl -f -s -H 'Host: grafana.torrust-demo.com' http://localhost:80/api/health" "Checking Grafana health"; then
+        log_success "Grafana is healthy"
     else
-        log_error "Prometheus is not healthy"
+        log_error "Grafana is not healthy"
         return 1
     fi
 
-    # Test Grafana
-    log_info "Testing Grafana..."
-    if vm_exec "${vm_ip}" "curl -f -s http://localhost:3100/api/health" "Checking Grafana health"; then
-        log_success "Grafana is healthy"
+    # Test Prometheus directly (no proxy configuration for Prometheus in current setup)
+    log_info "Testing Prometheus..."
+    if vm_exec "${vm_ip}" "docker exec prometheus wget -qO- http://localhost:9090/-/healthy" "Checking Prometheus health"; then
+        log_success "Prometheus is healthy"
     else
-        log_error "Grafana is not healthy"
+        log_error "Prometheus is not healthy"
         return 1
     fi