feat: isolate dev environment for multi-agent worktree support by wrn14897 · Pull Request #1994 · hyperdxio/hyperdx

wrn14897 · 2026-03-26T05:42:36Z

Summary

Isolate dev, E2E, and integration test environments so multiple git worktrees can run all three simultaneously without port conflicts
Each worktree gets a deterministic slot (0-99) with unique port ranges: dev (30100-31199), E2E (20320-21399), CI integration (14320-40098)
Dev portal dashboard (http://localhost:9900) auto-discovers all running stacks, streams logs, and provides a History tab for past run logs

Port Isolation

Environment	Port Range	Project Name
Dev stack	30100-31199	`hdx-dev-<slot>`
E2E tests	20320-21399	`e2e-<slot>`
CI integration	14320-40098	`int-<slot>`

All three can run simultaneously from the same worktree with zero port conflicts.

Dev Portal Features

Live tab:

Auto-discovers dev, E2E, and integration Docker containers + local services (API, App)
Groups all environments for the same worktree into a single card
SSE log streaming with ANSI color rendering, capped at 5000 lines
Auto-starts in background from make dev, make dev-e2e, make dev-int

History tab:

Logs archived to ~/.config/hyperdx/dev-slots/<slot>/history/ on exit (instead of deleted)
Each archived run includes meta.json with worktree/branch metadata
Grouped by worktree with collapsible cards, search by worktree/branch
View any past log file in the same log panel, delete individual runs or clear all
Custom dark-themed confirm modal (no native browser dialogs)

What Changed

scripts/dev-env.sh — Slot-based port assignments, portal auto-start, log archival on exit
scripts/test-e2e.sh — E2E port range (20320-21399), log capture via tee, portal auto-start, log archival
scripts/ensure-dev-portal.sh — Shared singleton portal launcher (works sourced or executed)
scripts/dev-portal/server.js — Discovery for dev/E2E/CI containers, history API (list/read/delete), local service port probing
scripts/dev-portal/index.html — Live/History tabs, worktree-grouped cards, search, collapse/expand, custom confirm modal, ANSI color log rendering
docker-compose.dev.yml — Parameterized ports/volumes/project name with hdx.dev.* labels
packages/app/tests/e2e/docker-compose.yml — Updated to new E2E port defaults
Makefile — dev-int/dev-e2e targets with log capture + portal auto-start; dev-portal-stop; dev-clean stops everything + wipes slot data
.env files — Ports use ${VAR:-default} syntax across dev, E2E, and CI environments
agent_docs/development.md — Full documentation for isolation, port tables, E2E/CI port ranges

How to Use

# Start dev stack (auto-starts portal)
make dev

# Run E2E tests (auto-starts portal, separate ports)
make dev-e2e FILE=navigation

# Run integration tests (auto-starts portal, separate ports)
make dev-int FILE=alerts

# All three can run simultaneously from the same worktree
# Portal at http://localhost:9900 shows everything

# Stop portal
make dev-portal-stop

# Clean up everything (all stacks + portal + history)
make dev-clean

Dev Portal

Test Plan

Run make dev — verify services start with slot-assigned ports
Run make dev in a second worktree — verify different ports, no conflicts
Run make dev-e2e and make dev-int simultaneously — no port conflicts
Open http://localhost:9900 — verify all stacks grouped by worktree
Click a service to view logs — verify ANSI colors render correctly
Stop a stack — verify logs archived to History tab with correct worktree
History tab — search, collapse/expand, view archived logs, delete
make dev-clean — stops everything, wipes slot data and history

changeset-bot · 2026-03-26T05:42:42Z

⚠️ No Changeset found

Latest commit: 842fea2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

vercel · 2026-03-26T05:42:42Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	Mar 30, 2026 6:19am

github-actions · 2026-03-26T05:44:15Z

Knip - Unused Code Analysis

🔴 +1 change in total issues (0 on main → 1 on PR)

Category	main	PR	Diff
Unused devDependencies	0	1	+1 🔴

Details

Unused devDependencies

🔴 package.json:@dotenvx/dotenvx

What is this?

Knip finds unused files, dependencies, and exports in your codebase.
This comment compares the PR branch against main to detect regressions.

Run yarn knip locally to see full details.

github-actions · 2026-03-26T05:44:21Z

PR Review

This is a dev-tooling-only PR (no production code changes). The server runs only locally, so security severity is reduced accordingly.

⚠️ JSON injection in dev-env.sh heredoc (lines 90–109, 127–129): HDX_DEV_BRANCH, HDX_DEV_WORKTREE, and PWD are embedded directly into JSON without escaping. Branch names like feat/my-"branch" or directories with backslashes produce malformed JSON, breaking dev portal discovery. → Use jq -n --arg or escape special chars (at minimum sed 's/\\/\\\\/g; s/"/\\"/g').
⚠️ Slot collision risk (dev-env.sh line 38): cksum | awk '{print $1 % 100}' gives only 100 possible slots. With multiple simultaneous worktrees, collisions are non-trivial (two worktrees sharing the same slot would stomp each other's ports and slot metadata). → Either increase the modulus (e.g. mod 1000 with a wider port range), or detect and resolve collisions, or document this as a known limitation.
⚠️ Exit trap set in sourced shell (dev-env.sh line 133): trap _hdx_cleanup_slot EXIT fires when the developer closes their terminal/shell, not just when make dev exits. This could unexpectedly archive logs just from closing a terminal window. → Consider a subshell pattern instead of relying on the parent shell's EXIT trap.
ℹ️ docker ps --format "{{json .}}" label parsing (server.js line 85–90): Label values are parsed by splitting on ,, but Docker label values can legally contain commas. This would silently mangle labels with commas in their values. Low risk given the specific labels used here, but fragile.

✅ Path traversal in getHistoryLog/deleteHistoryEntry is well-protected by HISTORY_DIR_RE. Local log reads in getLocalLogs are safe via whitelist lookup. Docker command arguments use whitelisted values. No critical security issues for a local-only tool.

github-actions · 2026-03-26T05:49:41Z

E2E Test Results

✅ All tests passed • 101 passed • 3 skipped • 1008s

Status	Count
✅ Passed	101
❌ Failed	0
⚠️ Flaky	3
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

brandon-pereira · 2026-03-27T15:25:35Z

Did a quick quick test - one question - is it intentional that the docker volumes are not shared - when I ran yarn dev it did not remember my created account, I went to the sign up page.

wrn14897 · 2026-03-27T15:48:15Z

Did a quick quick test - one question - is it intentional that the docker volumes are not shared - when I ran yarn dev it did not remember my created account, I went to the sign up page.

It really depends on your worktree setup. If you want the volume to be shared, make sure to add the copy step to your wt pre-create hook.

brandon-pereira · 2026-03-27T19:26:39Z

Did a quick quick test - one question - is it intentional that the docker volumes are not shared - when I ran yarn dev it did not remember my created account, I went to the sign up page.

It really depends on your worktree setup. If you want the volume to be shared, make sure to add the copy step to your wt pre-create hook.

This doesn't seem to work.. I tried again. I tried on main and got to the login screen, then I tried on this branch and got sign up 🤔

Didn't explore the code - lmk if you want me to investigate more

wrn14897 · 2026-03-27T21:10:10Z

Did a quick quick test - one question - is it intentional that the docker volumes are not shared - when I ran yarn dev it did not remember my created account, I went to the sign up page.

It really depends on your worktree setup. If you want the volume to be shared, make sure to add the copy step to your wt pre-create hook.

This doesn't seem to work.. I tried again. I tried on main and got to the login screen, then I tried on this branch and got sign up 🤔

Didn't explore the code - lmk if you want me to investigate more

Think about it again. I'm not sure copying .volume is a good idea since the session or db states might be corrupted. Alternatively, we can setup a dev only flag to enable auto signup.

Add slot-based port isolation so multiple git worktrees can run the full dev stack simultaneously without port conflicts. - Compute deterministic slot (0-99) from worktree directory name - Allocate dev ports in 30100-31199 range (no overlap with CI or E2E) - Parameterize docker-compose.dev.yml ports, project name, and volumes - Add Docker labels (hdx.dev.*) for portal service discovery - Remove fluentd logging driver from dev compose - Add dev portal microservice (localhost:9900) with live log streaming - Add make dev, make dev-down, make dev-portal targets - Tee local service logs to ~/.config/hyperdx/dev-slots/<slot>/logs/ - Update development docs with isolation workflow and port table

Reduce initial log payload sizes, batch DOM updates via rAF with a 5000-line cap, and convert ANSI escape codes to styled HTML for proper color rendering in the log viewer.

Register alerts and common-utils as local services in the dashboard so each concurrently process gets its own clickable log viewer.

Launch the portal server automatically when sourcing dev-env.sh. Skips if port 9900 is already in use (another worktree started it first). The portal process is cleaned up on exit alongside slot files.

… keybinding, plain port text - Use exact HyperDX dark mode color palette from _tokens.scss - Replace placeholder logo with HyperDX logomark SVG - Add favicon from packages/app - Expand instance cards to full width - Add Escape keybinding to close log panel - Make yarn dev mirror make dev (both source dev-env.sh) - Port numbers in table are now plain text, not clickable links

Use conditional tee (${HDX_DEV_LOGS_DIR:+path}) so app:dev writes log files when the env var is set (via dev-env.sh) and passes through to stdout when it's not. Single source of truth for which services are launched.

…onflict Set NEXT_DIST_DIR=.next-e2e in E2E playwright config so the E2E Next.js dev server uses a different build/lock directory than the main dev server, allowing both to run simultaneously from the same worktree.

… failures - dev-env.sh: remove stale .next/dev/lock before starting (preserves cache) - test-e2e.sh: remove entire .next-e2e directory for a clean slate each run

…e, log capture - Dev portal now discovers E2E (e2e-<slot>) and CI integration (int-<slot>) Docker containers alongside dev stacks, grouped by worktree in one card - Moved E2E tests to dedicated port range (20320-21399) so they no longer conflict with CI integration tests (14320-40098) or dev stack (30100-31199) - Added stdout log capture for make dev-e2e (tee to logs-e2e/e2e.log) and make dev-int (tee to logs-int/api-int.log) for portal log streaming - Local service port probing for E2E (API, App) and CI (API) processes - Auto-start dev portal from make dev-int and make dev-e2e via shared ensure-dev-portal.sh script - Added make dev-portal-stop and make dev-clean now stops portal + wipes ~/.config/hyperdx/dev-slots - Fixed stale E2E default ports across all E2E test files

- Logs are now archived to history/<envType>-<timestamp>/ on exit instead of being deleted, so devs can revisit past run logs - Added History tab to dev-portal with worktree-grouped cards matching the Live view layout - History API: GET /api/history (list), GET /api/history/:slot/:dir/:file (read), DELETE /api/history/:slot/:dir (remove) - Custom dark-themed confirm modal replaces native browser confirm() for Delete and Clear All actions - make dev-clean still wipes everything including history - Makefile uses reusable archive-int-logs function for DRY cleanup

- Write meta.json alongside archived logs with worktree/branch/path so history entries retain correct identity after the originating dev stack shuts down and its slot JSON file is deleted - Fix discoverHistory() fallback: only use process.cwd() for the local slot, inherit worktree info from sibling meta.json files for other slots - Add global search in History tab filtering by worktree and branch name - History cards collapsed by default, with expand/collapse toggle button in the top-right corner of each card - Search resets expanded state so all cards fold when typing

vercel bot deployed to Preview March 26, 2026 22:13 View deployment

vercel bot deployed to Preview March 26, 2026 22:35 View deployment

wrn14897 force-pushed the warren/HDX-3797-isolate-dev-environment branch from 88708db to e9d4e0a Compare March 27, 2026 16:16

vercel bot deployed to Preview March 27, 2026 16:19 View deployment

vercel bot deployed to Preview March 27, 2026 20:57 View deployment

vercel bot deployed to Preview March 27, 2026 21:57 View deployment

vercel bot deployed to Preview March 30, 2026 01:14 View deployment

vercel bot deployed to Preview March 30, 2026 04:54 View deployment

vercel bot deployed to Preview March 30, 2026 05:24 View deployment

vercel bot deployed to Preview March 30, 2026 05:30 View deployment

vercel bot deployed to Preview March 30, 2026 05:40 View deployment

vercel bot deployed to Preview March 30, 2026 05:48 View deployment

wrn14897 added 9 commits March 29, 2026 23:16

fix: optimize dev portal log rendering and add ANSI color support

b96dfe3

Reduce initial log payload sizes, batch DOM updates via rAF with a 5000-line cap, and convert ANSI escape codes to styled HTML for proper color rendering in the log viewer.

feat: show alerts and common-utils as separate log streams in dev portal

0c14e11

Register alerts and common-utils as local services in the dashboard so each concurrently process gets its own clickable log viewer.

feat: auto-start dev portal in background from make dev

7a030e8

Launch the portal server automatically when sourcing dev-env.sh. Skips if port 9900 is already in use (another worktree started it first). The portal process is cleaned up on exit alongside slot files.

fix: clean up stale Next.js lock/cache to prevent dev and E2E startup…

4979211

… failures - dev-env.sh: remove stale .next/dev/lock before starting (preserves cache) - test-e2e.sh: remove entire .next-e2e directory for a clean slate each run

wrn14897 added 5 commits March 29, 2026 23:16

docs: update AGENTS.md for make dev and multi-worktree support

3ed645e

fix: update E2E CI workflow ports to match new 20xxx-21xxx range

eccec6f

fix: replace bash-only PIPESTATUS with pipefail for CI /bin/sh compat

842fea2

wrn14897 force-pushed the warren/HDX-3797-isolate-dev-environment branch from 8447e74 to 842fea2 Compare March 30, 2026 06:16

vercel bot deployed to Preview March 30, 2026 06:19 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: isolate dev environment for multi-agent worktree support#1994

feat: isolate dev environment for multi-agent worktree support#1994
wrn14897 wants to merge 14 commits intomainfrom
warren/HDX-3797-isolate-dev-environment

wrn14897 commented Mar 26, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

vercel bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 26, 2026 •

edited

Loading

Uh oh!

brandon-pereira commented Mar 27, 2026 •

edited

Loading

Uh oh!

wrn14897 commented Mar 27, 2026

Uh oh!

brandon-pereira commented Mar 27, 2026

Uh oh!

wrn14897 commented Mar 27, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

wrn14897 commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Port Isolation

Dev Portal Features

What Changed

How to Use

Dev Portal

Test Plan

Uh oh!

changeset-bot bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

vercel bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Knip - Unused Code Analysis

Uh oh!

github-actions bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review

Uh oh!

github-actions bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Test Results

Uh oh!

brandon-pereira commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wrn14897 commented Mar 27, 2026

Uh oh!

brandon-pereira commented Mar 27, 2026

Uh oh!

wrn14897 commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wrn14897 commented Mar 26, 2026 •

edited

Loading

changeset-bot bot commented Mar 26, 2026 •

edited

Loading

vercel bot commented Mar 26, 2026 •

edited

Loading

github-actions bot commented Mar 26, 2026 •

edited

Loading

github-actions bot commented Mar 26, 2026 •

edited

Loading

github-actions bot commented Mar 26, 2026 •

edited

Loading

brandon-pereira commented Mar 27, 2026 •

edited

Loading

wrn14897 commented Mar 27, 2026 •

edited

Loading