Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions demos/sub-agent-multiplex/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# NanoClaw on Agent Substrate PoC
# Portable Dockerfile for OSS Substrate Migration

# Stage 1: Build the standalone bundles
FROM node:22-slim AS builder

WORKDIR /app

# Install build dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
&& rm -rf /var/lib/apt/lists/*

# Copy standalone package files
COPY package.json ./
# Use npm install for simplicity and portability in the standalone package
RUN npm install

# Copy source code
COPY ui/ ./ui/
COPY workload/ ./workload/
COPY broker/ ./broker/

# Build zero-dependency bundles
RUN ./node_modules/.bin/esbuild workload/agent.ts \
--bundle \
--platform=node \
--target=node22 \
--outfile=dist/agent.js \
--external:node:*

RUN ./node_modules/.bin/esbuild ui/demo-ui.ts \
--bundle \
--platform=node \
--target=node22 \
--outfile=dist/demo-ui.js \
--external:node:*

RUN ./node_modules/.bin/esbuild broker/server.ts \
--bundle \
--platform=node \
--target=node22 \
--outfile=dist/broker.js \
--external:node:*

# Stage 2: Final Production Image
FROM node:22-slim AS runner

WORKDIR /app

# Copy the entire context to check for local binaries
COPY . .

# Install runtime dependencies (tini for signal forwarding, kubectl for dashboard sync)
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
tini \
&& curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
&& chmod +x kubectl \
&& mv kubectl /usr/local/bin/ \
&& rm -rf /var/lib/apt/lists/*

# Copy built assets
COPY --from=builder /app/dist/ ./dist/
# Copy kubectl-ate binary if it exists in context, otherwise download it
# This makes the Dockerfile portable across environments
RUN if [ -f "./kubectl-ate" ]; then \
mv ./kubectl-ate /usr/local/bin/kubectl-ate; \
else \
curl -L -o /usr/local/bin/kubectl-ate https://github.com/agent-substrate/substrate/releases/latest/download/kubectl-ate-linux-amd64; \
fi && chmod +x /usr/local/bin/kubectl-ate

# Create a /pause hook for Substrate rehydration
RUN echo '#!/bin/sh' > /pause && \
echo 'echo "[pause] Starting NanoClaw agent..."' >> /pause && \
echo 'exec /usr/bin/tini -- /usr/local/bin/node /app/dist/agent.js' >> /pause && \
chmod +x /pause

# Default entrypoint (can be overridden by deployment to run demo-ui)
ENTRYPOINT ["/usr/bin/tini", "--", "node", "dist/agent.js"]
98 changes: 98 additions & 0 deletions demos/sub-agent-multiplex/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Substrate Multiplex Demo: NanoClaw 1.5x Overcommit

This demo demonstrates the extreme efficiency gains possible with Google Substrate by multiplexing **3 logical NanoClaw agents** onto **2 physical substrate workers** (1.5x density ratio).

## System Information

- **Agent Framework**: NanoClaw (v2.x)
- **Source**: `github.com/nanocoai/nanoclaw`
- **Substrate Mode**: Multi-Actor Multiplexing (1.5x oversubscription)
- **Runtime**: Bun (Node.js compatible) inside Debian Slim
- **Isolation**: gVisor (runsc)

## What this shows

- **High-Density Multiplexing**: Three logical agent identities running on only two physical pods (1.5x oversubscription).
- **State Persistence**: A `taskCounter` maintained in the Node.js process memory survives multiple suspend/resume cycles.
- **Dynamic Rotation**: Agents finish work at different times (3-6s), forcing Substrate to constantly rotate pod ownership.
- **Visual Identity Tracking**: Color-coded agents (Blue/Pink/Gold) and live log tailing to make infrastructure sharing intuitively obvious.

## Audience

This guide is intended for engineers exploring Agent Substrate for hosting large-scale agentic workloads where cost-efficiency and stateful rehydration are critical.

## Prerequisites

- **Agent Substrate Cluster**: A Kubernetes cluster with Substrate installed.
- **Docker**: For building and pushing the unified actor/UI image.
- **GCS Bucket**: Configured for Substrate state snapshots (e.g., `gs://snapshot-substrate-gke-ai-eco-dev/`).
- **kubectl & kubectl-ate**: The Substrate CLI tool for managing logical actors.

## Components

| Path | Purpose |
|---|---|
| `workload/agent.ts` | The workload: A NanoClaw/Hono server with persistent memory state. |
| `ui/demo-ui.ts` | The dashboard: A Node.js backend providing live logs, task queueing, and visual tracking. |
| `sub-agent-multiplex.yaml.tmpl` | Kubernetes manifests for ActorTemplates and WorkerPools. |
| `Dockerfile` | Unified OCI image containing both the actor workload and the dashboard UI. |

## How to Run

### 1. Provision Hardware
Scale the physical `WorkerPool` to the desired replica count (2 for this demo):
```bash
kubectl apply -f sub-agent-multiplex.yaml
```

### 2. Deploy Logical Agents
Create the three "fun-named" actors using the Substrate CLI.
```bash
kubectl-ate create actor agent-luna-v12 --template sub-agent/sub-agent-agent
kubectl-ate create actor agent-mars-v12 --template sub-agent/sub-agent-agent
kubectl-ate create actor agent-nova-v11 --template sub-agent/sub-agent-agent
```

### 3. Launch the Dashboard
The dashboard runs as a standard Kubernetes Deployment with a LoadBalancer.
```bash
kubectl apply -f demo-ui.yaml
```

## Drive the Demo

Open the dashboard and use the following interaction patterns:

- **Pulse (Manual Wakeup)**: Trigger tasks across the registry. Watch the **colored icons** rapidly cycle through the 2 worker slots.
- **Live Logs**: Observe the Fleet Decision Stream. You will see agent registrations and task dispatching events proving that physical hardware is being recycled in real-time.
- **Cron Tracker**: Observe real-time countdowns as the automated schedule triggers orchestration events.

## Integrating a Real LLM API

Integrating an LLM into a NanoClaw logical actor is straightforward. Because Substrate persists the **entire process memory**, any in-memory conversation history or KV-cache will survive multiple suspend/resume cycles without requiring an external database.

### 1. Add the LLM SDK
Add your preferred SDK (e.g., OpenAI or Anthropic) to the `package.json`:
```bash
npm install openai
```

### 2. Update the Actor Logic
Modify `workload/agent.ts` to initialize the client and maintain a local chat history:
```typescript
import OpenAI from "openai";

const openai = new OpenAI({ apiKey: process.env.LLM_API_KEY });
let history: any[] = []; // This array will survive Substrate snapshots!

app.post("/v1/chat", async (c) => {
const { message } = await c.req.json();
history.push({ role: "user", content: message });

const response = await openai.chat.completions.create({
model: "gpt-4",
messages: history,
});
// ... process response
});
```
38 changes: 38 additions & 0 deletions demos/sub-agent-multiplex/broker-deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: nano-broker
namespace: sub-agent
labels:
app: nano-broker
spec:
replicas: 1
selector:
matchLabels:
app: nano-broker
template:
metadata:
labels:
app: nano-broker
spec:
containers:
- name: broker
image: gcr.io/gke-ai-eco-dev/substrate-demos/sub-agent:latest
command: ["/usr/bin/tini", "--", "node", "dist/broker.js"]
ports:
- containerPort: 8091
env:
- name: ATE_ENDPOINT
value: "api.ate-system.svc.cluster.local:443"
---
apiVersion: v1
kind: Service
metadata:
name: nano-broker
namespace: sub-agent
spec:
selector:
app: nano-broker
ports:
- port: 8091
targetPort: 8091
178 changes: 178 additions & 0 deletions demos/sub-agent-multiplex/broker/server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
import { Hono } from "hono";
import { serve } from "@hono/node-server";
import { exec } from "node:child_process";

const app = new Hono();

// --- Configuration ---
const ATE_ENDPOINT = process.env.ATE_ENDPOINT || "api.ate-system.svc.cluster.local:443";
const AGENT_TASKS = [
"Inventory reconciliation audit",
"Security patch verification",
"Log aggregation summary",
"API endpoint health check",
"Database index optimization analysis"
];

// --- Types ---
interface RegisteredAgent {
actorId: string;
lastSeen: number;
status: "idle" | "working" | "error";
taskCount: number;
}

interface BrokerLog {
timestamp: string;
module: "registry" | "orchestrator" | "substrate";
message: string;
level: "info" | "warn" | "error";
}

// --- State ---
const registry: Record<string, RegisteredAgent> = {};
const logs: BrokerLog[] = [];
let isOrchestrating = false;

// --- Helpers ---
const log = (module: BrokerLog["module"], message: string, level: BrokerLog["level"] = "info") => {
const entry: BrokerLog = {
timestamp: new Date().toISOString().slice(11, 19),
module,
message,
level
};
logs.push(entry);
if (logs.length > 100) logs.shift();
console.log(`[${entry.timestamp}] [${module}] ${message}`);
};

const runCmd = (cmd: string): Promise<string> => {
return new Promise((resolve, reject) => {
exec(cmd, (error, stdout, stderr) => {
if (error) reject(new Error(stderr || error.message));
else resolve(stdout);
});
});
};

// --- API Endpoints ---

// NanoClaw Agents call this on boot
app.post("/register", async (c) => {
const { actorId } = await c.req.json();
if (!actorId) return c.json({ error: "actorId required" }, 400);

registry[actorId] = {
actorId,
lastSeen: Date.now(),
status: registry[actorId]?.status || "idle",
taskCount: registry[actorId]?.taskCount || 0
};

log("registry", `Agent **${actorId}** self-registered successfully.`);
return c.json({ status: "registered", broker: "FleetBroker-v1" });
});

// Infrastructure calls this when it detects due work
app.post("/notify-due", async (c) => {
const { sessionId, dueCount } = await c.req.json();
log("orchestrator", `Infrastructure Alert: **${sessionId}** has ${dueCount} tasks pending.`);

// Custom Platform Policy: Always trigger if work is due
performTask(sessionId);
return c.json({ status: "processed" });
});

// Dashboard polls this for "Platform View"
app.get("/status", (c) => {
return c.json({
registry: Object.values(registry),
logs: logs,
orchestrating: isOrchestrating
});
});

// Trigger a manual task from Dashboard
app.post("/trigger/:actorId", async (c) => {
const actorId = c.req.param("actorId");
if (!registry[actorId]) return c.json({ error: "Agent not registered" }, 404);

// Fire and forget task execution
performTask(actorId);
return c.json({ status: "triggered" });
});

// --- Orchestration Logic ---

async function performTask(actorId: string) {
if (registry[actorId].status === "working") {
log("orchestrator", `Task skipped for ${actorId}: already working.`, "warn");
return;
}

const task = AGENT_TASKS[Math.floor(Math.random() * AGENT_TASKS.length)];
registry[actorId].status = "working";
log("orchestrator", `Policy Trigger: Dispatching '${task}' to **${actorId}**.`);

try {
// 1. Substrate Resume
log("substrate", `> kubectl-ate resume actor ${actorId}`);
await runCmd(`kubectl-ate --endpoint ${ATE_ENDPOINT} resume actor ${actorId}`);

// 2. Wait for Rehydration
let actorIP = "";
for (let i = 0; i < 30; i++) {
const out = await runCmd(`kubectl-ate --endpoint ${ATE_ENDPOINT} get actor ${actorId} -o json`);
const actor = JSON.parse(out).actors?.[0] || JSON.parse(out);
if (actor.status === "STATUS_RUNNING" && actor.ateomPodIp) {
actorIP = actor.ateomPodIp;
break;
}
await new Promise(r => setTimeout(r, 1000));
}

if (!actorIP) throw new Error("Rehydration Timeout");

// 3. Execute NanoClaw Task
log("substrate", `Connected to ${actorId} at ${actorIP}. Injecting payload...`);
await new Promise(r => setTimeout(r, 4000)); // Network settle

await runCmd(`curl -s -f -m 10 -X POST http://${actorIP}:8080/task -H "Content-Type: application/json" -d '{"task": "${task}"}'`);

log("orchestrator", `Task completed by **${actorId}**. Platform yielding hardware.`);
registry[actorId].taskCount++;

// 4. Substrate Suspend
log("substrate", `> kubectl-ate suspend actor ${actorId}`);
await runCmd(`kubectl-ate --endpoint ${ATE_ENDPOINT} suspend actor ${actorId}`);

registry[actorId].status = "idle";
} catch (e: any) {
log("substrate", `Orchestration failed for ${actorId}: ${e.message}`, "error");
registry[actorId].status = "error";
// Safety yield
await runCmd(`kubectl-ate --endpoint ${ATE_ENDPOINT} suspend actor ${actorId}`).catch(() => {});
}
}

// Simulated Customer Policy: Every 2 minutes, pick a registered agent to work
async function runAutoPolicy() {
if (!isOrchestrating) return;

const activeAgents = Object.keys(registry);
if (activeAgents.length > 0) {
const pick = activeAgents[Math.floor(Math.random() * activeAgents.length)];
performTask(pick);
}

setTimeout(runAutoPolicy, 120000); // 2 minute cycle
}

// --- Start Server ---
const port = 8091;
serve({ fetch: app.fetch, port, hostname: "0.0.0.0" });

log("registry", "Fleet Management Broker active on port 8091.");
isOrchestrating = true;
runAutoPolicy();
Loading
Loading