ci: [#14] separate CI-compatible tests from virtualization-required tests

- Add test-ci and test-local targets to Makefile for clear test separation
- Update GitHub Actions workflow to run make test-ci with all dependencies
- Create orchestration scripts for CI (test-ci.sh) and local (test-local.sh) testing
- Add unit test scripts for config, scripts, and infrastructure validation
- Remove deprecated test-integration.sh and test-local-setup.sh
- Document testing strategy in ci-vs-local-test-analysis.md
- Update infrastructure test documentation and project references
- Improve Makefile help output to clarify testing workflow

This enables running syntax validation, config validation, and unit tests
in GitHub Actions while keeping full E2E infrastructure tests for local
development with virtualization support.

Author: josecelano

.github/workflows/testing.yml

@@ -7,19 +7,24 @@ on:
     branches: [main, develop]
 
 jobs:
-  lint:
+  ci-tests:
     runs-on: ubuntu-latest
+    name: CI-Compatible Tests
 
     steps:
       - name: Checkout code
         uses: actions/checkout@v4
 
-      - name: Install linting tools
+      - name: Install dependencies
         run: |
           sudo apt-get update
-          sudo apt-get install -y yamllint shellcheck
+          sudo apt-get install -y yamllint shellcheck docker-compose
           sudo npm install -g markdownlint-cli
 
-      - name: Run linting script
-        run: |
-          ./scripts/lint.sh
+          # Install OpenTofu
+          curl -fsSL https://get.opentofu.org/install-opentofu.sh -o install-opentofu.sh
+          chmod +x install-opentofu.sh
+          sudo ./install-opentofu.sh --install-method deb
+
+      - name: Run CI test suite
+        run: make test-ci

Makefile

@@ -9,7 +9,8 @@
 VM_NAME ?= torrust-tracker-demo
 ENVIRONMENT ?= local
 TERRAFORM_DIR = infrastructure/terraform
-TESTS_DIR = infrastructure/tests
+INFRA_TESTS_DIR = infrastructure/tests
+TESTS_DIR = tests
 SCRIPTS_DIR = infrastructure/scripts
 
 # Help target
@@ -21,6 +22,13 @@ help: ## Show this help message
 	@echo "  2. app-deploy      - Deploy application (Build + Release + Run stages)"
 	@echo "  3. health-check    - Validate deployment"
 	@echo ""
+	@echo "=== TESTING WORKFLOW ==="
+	@echo "  1. test-syntax     - Fast syntax validation (30s)"
+	@echo "  2. test-unit       - Unit tests without deployment (1-2min)" 
+	@echo "  3. test-ci         - CI-compatible tests (syntax + config + scripts)"
+	@echo "  4. test-local      - Local-only tests (requires virtualization)"
+	@echo "  5. test            - Full E2E test with deployment (5-8min)"
+	@echo ""
 	@echo "Available targets:"
 	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "  %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
 	@echo ""
@@ -123,14 +131,38 @@ validate-config: ## Validate configuration for all environments
 # TESTING AND QUALITY ASSURANCE
 # =============================================================================
 
-test: ## Run comprehensive test suite
-	@echo "Running comprehensive test suite..."
-	$(TESTS_DIR)/test-local-setup.sh
+test-prereq: ## Test system prerequisites for development
+	@echo "Testing prerequisites..."
+	$(INFRA_TESTS_DIR)/test-unit-infrastructure.sh vm-prereq
+
+test: ## Run comprehensive end-to-end test (follows integration guide)
+	@echo "Running comprehensive end-to-end test..."
+	$(TESTS_DIR)/test-e2e.sh $(ENVIRONMENT)
+
+test-unit: ## Run unit tests (configuration, scripts, syntax)
+	@echo "Running unit tests..."
+	@echo "1. Configuration and syntax validation..."
+	$(INFRA_TESTS_DIR)/test-unit-config.sh
+	@echo "2. Infrastructure scripts validation..."
+	$(INFRA_TESTS_DIR)/test-unit-scripts.sh
 
 test-syntax: ## Run syntax validation only
 	@echo "Running syntax validation..."
 	./scripts/lint.sh
 
+test-ci: ## Run CI-compatible tests (syntax + config + scripts)
+	@echo "Running CI-compatible tests..."
+	$(INFRA_TESTS_DIR)/test-ci.sh
+
+test-local: ## Run local-only tests (requires virtualization)
+	@echo "Running local-only tests..."
+	$(INFRA_TESTS_DIR)/test-local.sh
+
+test-legacy: ## [DEPRECATED] Legacy test scripts have been removed
+	@echo "⚠️  DEPRECATED: Legacy test scripts have been removed"
+	@echo "Use 'make test-unit' for unit tests or 'make test' for E2E tests"
+	@exit 1
+
 lint: test-syntax ## Run all linting (alias for test-syntax)
 
 clean: ## Clean up temporary files and caches

README.md

@@ -12,7 +12,7 @@ It's also used to track issues in production.
 
 ## 🏗️ Repository Structure
 
-This repository is organized into two main concerns:
+This repository is organized into distinct concerns:
 
 ### 📦 [`infrastructure/`](infrastructure/)
 
@@ -21,7 +21,7 @@ This repository is organized into two main concerns:
 - OpenTofu/Terraform for VM provisioning
 - cloud-init templates for system setup
 - libvirt/KVM for local testing
-- Infrastructure testing and validation
+- Infrastructure unit tests and validation
 
 ### 🚀 [`application/`](application/)
 
@@ -32,6 +32,14 @@ This repository is organized into two main concerns:
 - Nginx, Prometheus, Grafana setup
 - Application scripts and utilities
 
+### 🧪 [`tests/`](tests/)
+
+**End-to-end testing** - Complete system validation
+
+- E2E deployment workflow tests
+- Integration testing automation
+- System-wide validation
+
 ### 📚 [`docs/`](docs/)
 
 **Project documentation** - Guides, security, and reference materials

docs/guides/integration-testing-guide.md

@@ -942,8 +942,12 @@ for better compatibility with modern compose.yaml files.
 ### 4.1 Test VM Access
 
 ```bash
-# [PROJECT_ROOT] Test basic VM connectivity
-time ./infrastructure/tests/test-integration.sh access
+# [PROJECT_ROOT] Test basic VM connectivity using SSH
+make ssh
+
+# Or test connectivity manually
+VM_IP=$(cd infrastructure/terraform && tofu output -raw vm_ip)
+ssh torrust@$VM_IP "echo 'VM is accessible'"
 ```
 
 **Expected Output**:
@@ -955,8 +959,14 @@ time ./infrastructure/tests/test-integration.sh access
 ### 4.2 Test Docker Installation
 
 ```bash
-# [PROJECT_ROOT] Test Docker functionality
-time ./infrastructure/tests/test-integration.sh docker
+# [PROJECT_ROOT] Test Docker functionality via health check
+make health-check
+
+# Or test Docker manually via SSH
+make ssh
+# Then inside VM:
+docker --version
+docker compose version
 ```
 
 **Expected Output**:
@@ -973,8 +983,8 @@ available and uses the appropriate command.
 ### 4.3 Setup Torrust Tracker Demo
 
 ```bash
-# [PROJECT_ROOT] Clone and setup the Torrust Tracker repository
-time ./infrastructure/tests/test-integration.sh setup
+# [PROJECT_ROOT] Deploy the application using twelve-factor workflow
+make app-deploy
 ```
 
 **Expected Output**:
@@ -989,8 +999,10 @@ configuration.
 ### 4.4 Start Torrust Tracker Services
 
 ```bash
-# [PROJECT_ROOT] Pull images and start all services
-time ./infrastructure/tests/test-integration.sh start
+# [PROJECT_ROOT] Application deployment includes starting services
+# Services are automatically started by 'make app-deploy'
+# To verify services are running:
+make health-check
 ```
 
 **Expected Output**:
@@ -1010,8 +1022,8 @@ time ./infrastructure/tests/test-integration.sh start
 ### 4.5 Test Service Endpoints
 
 ```bash
-# [PROJECT_ROOT] Test all API endpoints
-time ./infrastructure/tests/test-integration.sh endpoints
+# [PROJECT_ROOT] Test all endpoints via comprehensive health check
+make health-check
 ```
 
 **Expected Output**:
@@ -1028,8 +1040,11 @@ requirements. For manual testing, see Step 5.2 for the correct endpoint testing
 ### 4.6 Test Monitoring Services
 
 ```bash
-# [PROJECT_ROOT] Test Prometheus and Grafana
-time ./infrastructure/tests/test-integration.sh monitoring
+# [PROJECT_ROOT] Test Prometheus and Grafana via health check
+make health-check
+
+# For detailed monitoring, connect via SSH to inspect services directly
+make ssh
 ```
 
 **Expected Output**:
@@ -1041,8 +1056,8 @@ time ./infrastructure/tests/test-integration.sh monitoring
 ### 4.7 Run Complete Integration Test Suite
 
 ```bash
-# [PROJECT_ROOT] Run all tests in sequence
-time ./infrastructure/tests/test-integration.sh full-test
+# [PROJECT_ROOT] Run complete E2E test (infrastructure + application + health)
+make test
 ```
 
 **Expected Output**:
@@ -1541,8 +1556,11 @@ time (cd "$TRACKER_DIR" && cargo run -p torrust-tracker-client --bin http_tracke
 ### 8.1 Stop Services (if needed)
 
 ```bash
-# [PROJECT_ROOT] Stop all services cleanly
-./infrastructure/tests/test-integration.sh stop
+# [PROJECT_ROOT] Stop services via SSH if needed
+make ssh
+# Then inside VM:
+cd /home/torrust/github/torrust/torrust-tracker-demo/application
+docker compose down
 ```
 
 ### 8.2 Destroy VM and Clean Up
@@ -1669,17 +1687,17 @@ curl http://$VM_IP/api/v1/stats
 curl "http://$VM_IP/api/v1/stats?token=local-dev-admin-token-12345"
 ```
 
-### 9.4 Integration Test Script Limitations
+### 9.4 Health Check Limitations
 
-The automated integration test script (`./infrastructure/tests/test-integration.sh endpoints`)
-may fail because:
+The automated health check script (`make health-check`) provides comprehensive
+validation but may need tuning for specific scenarios:
 
-1. **Authentication**: Script doesn't include token for stats API
-2. **Port Assumptions**: May test internal ports instead of nginx proxy
-3. **JSON Parsing**: Doesn't use `jq` for response validation
+1. **Timeouts**: Some tests use conservative timeouts that may be slow
+2. **Test Coverage**: Focuses on connectivity rather than functional testing
+3. **Verbose Output**: Use `VERBOSE=true make health-check` for detailed results
 
-**Manual testing** (as shown in this guide) provides more reliable results and
-better insight into the actual API functionality.
+**Manual testing** (as shown in this guide) provides more detailed functional
+validation and better insight into the actual API behavior.
 
 ### 9.5 Useful Testing Commands
 
@@ -1798,7 +1816,7 @@ ls -la | grep -E "(Makefile|infrastructure|application)"
 
 - `make: *** No rule to make target 'configure-local'. Stop.`
 - `make: *** No such file or directory. Stop.`
-- `./infrastructure/tests/test-integration.sh: No such file or directory`
+- Commands like `make infra-apply` failing with file not found errors
 
 **Solution**: Always ensure you're in the project root directory before running commands.