This repository was archived by the owner on Oct 10, 2025. It is now read-only.
Commit e4076d1
committed
Merge #11: Add Local KVM/libvirt Testing Infrastructure
8fac056 fix: [#10] add X-Forwarded-For header to nginx HTTP config (Jose Celano)
75df631 refactor: comment out Rust dependencies for Docker-only deployment (Jose Celano)
58c7294 docs: [#10] add ADR-002 documenting Docker for all services decision (Jose Celano)
a2e0554 docs: [#10] add troubleshooting for VM IP detection issue (Jose Celano)
4c0edc0 feat: [#10] add Rust installation to cloud-init configuration (Jose Celano)
6203f29 feat: upgrade Docker installation to use official Docker repository (Jose Celano)
22ee5f3 docs: [#10] add twelve-factor app refactoring plan and guides (Jose Celano)
e5f29a2 security: disable password authentication by default (Jose Celano)
ed1bcb0 fix: modernize cloud-init user password configuration (Jose Celano)
e4833aa docs: [#10] update all Ubuntu version references from 22.04 to 24.04 (Jose Celano)
3a3746c docs: [#10] add DHCP lease behavior explanation to libvirt setup guide (Jose Celano)
b272f1b docs: organize SSH bug documentation into structured archive (Jose Celano)
c292adb fix: [#10] resolve SSH authentication failure in cloud-init configuration (Jose Celano)
53b7591 docs: [#10] add nullglob to project dictionary (Jose Celano)
9dc6b00 docs: [#10] add mandatory linting requirement to copilot instructions (Jose Celano)
3d5c1ee refactor: [#10] simplify lint.sh to use tools' built-in file discovery (Jose Celano)
a4a5e5f refactor: [#10] rename workflow from infrastructure to testing (Jose Celano)
5b70235 feat: [#10] implement comprehensive linting infrastructure (Jose Celano)
fdf1a95 fix: resolve YAML line length in user-data.yaml.tpl (Jose Celano)
cf61dfd docs: add preferred working methodology to copilot instructions (Jose Celano)
7f3251a docs: enforce GPG commit signing requirement in copilot instructions (Jose Celano)
d7d9ddf feat: add VM console access commands and documentation (Jose Celano)
94d01d4 feat: update infrastructure to use Ubuntu 24.04 and fix cloud-init configuration (Jose Celano)
374d763 feat: [#10] enhance development workflow and SSH debugging (Jose Celano)
b9e6606 feat: [#10] add Torrust Tracker dependencies for future source compilation (Jose Celano)
a786666 docs: [#10] add explicit git permission requirements to AI Assistant guidelines (Jose Celano)
9fbf7bd fix: [#10] correct typo in install script message (Jose Celano)
70639c5 fix: [#10] remove undefined service dependencies from Docker Compose (Jose Celano)
a43d130 chore: [#10] remove obsolete MAKEFILE_TESTING_TODO.md file (Jose Celano)
091029f fix: resolve Docker Compose compatibility and firewall SSH blocking issues (Jose Celano)
832fc68 docs: update makefile validation checklist with comprehensive test results (Jose Celano)
34750e1 refactor: reorganize repo into infrastructure and application components (Jose Celano)
8a33e42 feat: add local KVM/libvirt testing infrastructure with automated fixes (Jose Celano)
Pull request description:
Relates to: #10
This PR implements a complete local testing infrastructure using KVM/libvirt to enable reliable local development and testing before production deployment to Hetzner.
### 📋 **What's Included**
#### **🔧 Infrastructure Setup**
- **OpenTofu/Terraform** configuration for local VM deployment
- **Cloud-init** templates for automated Ubuntu 22.04 setup with Docker
- **Automated libvirt** permission fixes and storage pool configuration
- **Network configuration** with proper interface detection (`ens3`)
- **Security hardening** with UFW firewall and automatic updates
#### **⚙️ Makefile Automation**
- Complete workflow automation with intuitive targets
- SSH key injection from local config (secure, never stored in repo)
- Real-time VM monitoring and cloud-init progress tracking
- Minimal configuration support for debugging
- Comprehensive libvirt troubleshooting automation
#### **🧪 Testing & Monitoring**
- Automated infrastructure validation tests
- Real-time cloud-init monitoring script
- VM status checking and SSH connectivity validation
- Network optimization for BitTorrent traffic
#### **📚 Documentation**
- Quick Start Guide
- Complete Setup Guide
- libvirt Troubleshooting
- Updated main README with local testing section
### 🎯 **Current Status**
✅ **Working Infrastructure**
- ✅ VM deployment and network connectivity
- ✅ SSH access with key-based authentication
- ✅ Docker installation and configuration
- ✅ UFW firewall with Torrust Tracker ports
- ✅ Network optimizations for BitTorrent
- ✅ Cloud-init automation
### 📝 **TODO List for Review**
#### **🔍 Manual Testing Required**
- [x] **Test all Makefile commands** generated by AI assistant
- [x] `make install-deps` - Dependency installation
- [x] `make init` - OpenTofu initialization
- [x] `make plan` - Infrastructure planning
- [x] `make apply` - VM deployment
- [x] `make apply-minimal` - Minimal configuration deployment
- [x] `make monitor-cloud-init` - Real-time monitoring
- [x] `make restart-and-monitor` - Complete restart workflow
- [x] `make ssh` - SSH connection
- [x] `make destroy` - VM cleanup
- [x] `make test` - Infrastructure tests
#### **🧪 Infrastructure Validation**
- [x] **Run infrastructure tests**
- [x] `tests/infrastructure/test-local-setup.sh prerequisites`
- [x] `tests/infrastructure/test-local-setup.sh full-test`
- [x] test-integration.sh
- [x] **Verify GitHub Actions workflow** (infrastructure.yml)
- [x] **Test libvirt permission fixes** on fresh Ubuntu/Debian systems
#### **🎯 Torrust Tracker Installation Testing**
DISCARDED: Compilation from sources was discarded. We have prioritised easy update over performance for the demo.
- [ ] **Install tracker from Rust sources**
- [ ] Clone torrust-tracker repo in VM
- [ ] Build with Rust/Cargo
- [ ] Configure and run tracker
- [ ] Test HTTP and UDP endpoints
- [x] **Install with Docker Compose** (legacy torrust-demo style)
- [x] Clone repo in VM
- [x] Run with docker-compose
- [x] Verify all services start correctly
- [x] Test tracker functionality
#### **📋 Code Review**
- [x] Review OpenTofu configuration security
- [x] Validate cloud-init templates
- [x] Check Makefile target implementation
- [x] Verify .gitignore excludes sensitive files
- [x] Review documentation completeness
### 🎯 **Next PRs Scope**
The **next PR** will focus on production deployment:
- [ ] **Hetzner production configuration**
- [ ] Terraform/OpenTofu config for Hetzner Cloud
- [ ] Production cloud-init templates
- [ ] SSL/TLS configuration with Let's Encrypt
- [ ] Production security hardening
- [ ] Monitoring and alerting setup
### 🔒 **Security Notes**
- ✅ SSH keys properly use template variables, never stored in repo
- ✅ `local.tfvars` is git-ignored and contains actual secrets locally
- ✅ All sensitive files properly excluded via .gitignore
- ✅ Cloud-init templates use secure templating approach
### 🎯 **Testing the Infrastructure**
```bash
# Quick setup (requires Ubuntu/Debian with sudo)
make install-deps # Install dependencies
make setup-ssh-key # Configure SSH key
make apply # Deploy VM
make ssh # Connect to VM
# Monitor deployment
make monitor-cloud-init
# Run tests
make test
# Cleanup
make destroy
```
### 📊 **Files Changed**
- **Infrastructure**: 15+ new files (OpenTofu, cloud-init, scripts)
- **Documentation**: 4 new guides + updated README
- **Automation**: Enhanced Makefile with 15+ targets
- **Testing**: Infrastructure test suite + GitHub Actions workflow
🚀 This provides a solid foundation for local testing before production Hetzner deployment.
ACKs for top commit:
josecelano:
ACK 8fac056
Tree-SHA512: 94426a5e1e1e58743808e8f4443ea595db9df03c67c51faa9827fb34b488eaf970c8263817e5de829f32208e593300c90197aca31a6913b59c0433f4a82a5c77File tree
93 files changed
+11959
-193
lines changed- .github
- workflows
- .vscode
- application
- docs
- media
- docs
- adr
- guides
- infrastructure
- testing
- infrastructure
- cloud-init
- docs
- bugs
- 001-ssh-authentication-failure
- test-configs
- refactoring/twelve-factor-refactor
- testing
- third-party
- scripts
- terraform
- tests
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
93 files changed
+11959
-193
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
0 commit comments