PYTHON WORLD
Replies: 1 comment
|
The plan is many concurrent sessions across different harnesses, so work started in one harness has to reach another. Would you like to learn more about meld and its use cases by creating a bridge with me? You create it at https://meld.mergeinc.workers.dev. Say what the bridge is for and what it is not for, and send the URL. It stays open 36 hours until the first reply. Each later reply resets 24 hours. The conversation stays on that same link. It is free, and there is no billing. It will not be the platform or a shared session. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Multi-Harness Agent Platform — Full Architecture & Implementation Plan
1. Vision
Build a multi-tenant platform that allows users to:
The core architectural principle is:
2. Supported Harnesses
Initial catalog:
3. Core Architectural Principles
3.1 Harness != Model != Provider != Credential != Runtime
Keep these independent:
3.2 Control Plane vs Data Plane
Control Plane
Responsible for:
Data Plane
Responsible for:
This separation is essential for scaling.
3.3 Immutable execution
Never run a production session from
latest.Resolve:
Every execution stores the exact digest.
4. High-Level Architecture
5. Major Platform Subsystems
6. Multi-Tenancy
Resource hierarchy:
Every persistent resource should carry tenant ownership.
Typical identifiers:
Never rely only on frontend authorization.
7. Authentication and Authorization
Authentication
Support:
Authorization
Use RBAC initially:
Then add policy-based controls for:
8. Harness Registry
The registry describes available harnesses.
Example:
The registry should not contain only a list of names. It should also contain:
9. Harness Version and Binary Management
Version hierarchy
There are independent versions:
Do not tie adapter version to harness version.
Artifact model
Example:
Installation flow
Verify:
Version channels
Support:
Version pinning
Store:
{ "harness": "pi", "version": "0.9.2", "artifact_digest": "sha256:abc123", "adapter_version": "2.1.0" }Harness Lock
Projects should optionally contain:
This provides reproducibility.
10. Runtime Management
Prefer containerized execution over installing all harnesses on the host.
Runtime should support:
Use a runtime abstraction so Docker/Kubernetes/VM execution can evolve independently.
11. Distributed Scheduling
At scale:
Use queues with:
12. Harness Adapter Architecture
Every harness should implement a platform adapter.
Conceptual interface:
The exact implementation varies by harness.
The platform owns the lifecycle contract.
13. Harness Capability Manifest
Each adapter should expose:
{ "harness": "pi", "version": "0.9.2", "capabilities": { "sessions": true, "streaming": true, "commands": true, "mcp": true, "plugins": true, "memory": true, "compaction": true } }UI should render capabilities dynamically.
Never assume all harnesses support the same features.
14. Dynamic UI Architecture
Use three UI levels.
Level 1 — Universal
Examples:
Level 2 — Capability-based
Examples:
Level 3 — Harness-specific
Unique features can be rendered by optional frontend plugins.
15. Dynamic UI Component Protocol
Backend returns semantic UI components:
{ "type": "metric", "label": "Context Usage", "value": 64, "unit": "%" }Frontend component registry:
The backend describes semantics, not exact CSS/layout.
16. Dynamic Forms
Use JSON Schema for:
Example:
{ "command": "/model", "input_schema": { "type": "object", "properties": { "model": { "type": "string", "enum": ["model-a", "model-b"] } }, "required": ["model"] } }The UI can automatically generate a form.
17. Commands
Harnesses may expose commands such as:
Adapter returns:
{ "commands": [ { "name": "/compact", "type": "action" }, { "name": "/model", "type": "form", "input_schema": {} } ] }The UI builds a dynamic command palette.
Unknown commands should still be executable through a generic command interface where policy allows.
18. Events and Notifications
Harnesses may emit different event types.
Normalize common events:
Preserve native events:
{ "type": "harness.event", "harness": "pi", "name": "extension.loaded", "data": {} }Unknown events should not break the platform.
19. Event Bus
Use an event bus between runtime and platform.
WebSocket/SSE clients subscribe through the platform, not directly to containers.
20. Session Architecture
Separate:
Example:
This allows multiple harnesses to operate simultaneously.
21. Session State
Platform-level session state:
Harness-specific state remains in the adapter/runtime where required.
Do not try to force every harness's internal state into one database schema.
22. Multiple Conversations
Use explicit identifiers:
Never use a single global session per user.
A user can have:
simultaneously.
23. Session Resume
Store enough platform metadata to reconnect:
If the runtime is alive:
If it is gone:
24. MCP Architecture
Do not allow every harness to independently manage credentials.
Recommended:
Where native harness MCP behavior is required, the adapter can bridge it.
25. Short-Lived MCP Tokens
For a 5-minute token:
Never store only a token expiry and hope the harness handles it.
Use:
Token refresh should happen centrally when possible.
Important
Use a safety margin, for example:
instead of waiting for the exact expiration second.
Concurrent refresh protection
If 20 tool calls notice an expired token simultaneously:
Avoid a refresh storm.
26. Model Architecture
Keep:
independent.
Model Registry
Store:
Model Resolver
Input:
Output:
27. Model Routing Modes
Support:
Platform gateway can provide:
Do not force native harness integrations through the gateway if that would break important functionality.
28. Model Selection Levels
Allow defaults at multiple levels:
Most specific setting wins.
Example:
29. Automatic Model Selection
Optional:
Pipeline:
Examples:
30. Model Fallback
Define fallback chains:
Fallback should occur at safe execution boundaries.
Do not blindly switch models in the middle of a critical tool call.
31. Plugins
Separate plugin types:
Maintain a plugin manifest:
Plugin installation should include:
32. Memory Architecture
Do not make harness memory the only memory.
Use layers:
And optionally:
Platform memory is portable across harnesses.
Harness-native memory can preserve harness-specific behavior.
Memory types
33. Compaction
Harnesses may have their own compaction mechanisms.
Treat them as capabilities:
Use native compaction when it is more accurate for that harness.
Platform-level compaction can provide fallback or cross-harness summarization.
Do not overwrite harness-native state unexpectedly.
34. Workspace Manager
Every execution should have a workspace abstraction.
Workspace modes:
Workspace lifecycle:
35. Checkpoints and Recovery
Checkpoint should capture:
Flow:
This is critical for long-running agents.
36. Artifacts
Agents can produce:
Use object storage.
Metadata:
37. Human-in-the-Loop
Central approval service:
Examples requiring approval:
Approval should be platform-enforced, not only UI-enforced.
38. Tool Permission Engine
Assign risk:
Policies can depend on:
39. Network Isolation
Runtime sandbox:
Default-deny network where practical.
Allow only approved destinations.
40. Resource Quotas
Per tenant/user/project:
Example:
41. Rate Limiting
Apply limits at:
Use distributed rate limiting for horizontally scaled API servers.
42. Usage and Billing
Meter:
Store usage against:
This makes future billing possible.
43. Observability
Collect:
Logs
Metrics
Traces
Trace:
Use correlation IDs.
44. Audit Log
Separate security audit from normal logs.
Record:
Examples:
Audit records should be append-only where practical.
45. Idempotency
Use idempotency keys for operations such as:
This prevents duplicate work when clients retry.
46. Cancellation
Every long-running operation should support cancellation:
Cancellation should propagate:
Where a downstream operation cannot be cancelled, mark it appropriately and prevent duplicate execution.
47. Failure Handling
Define error classes:
Each should have:
Retry only errors marked retryable.
Use dead-letter queues for permanently failed jobs.
48. Health and Readiness
Every harness/runtime should expose:
Example:
49. Feature Flags
Use feature flags for:
Support gradual rollout:
50. Harness Conformance Test Suite
Every adapter must pass:
New harness onboarding:
This is critical once you support dozens of harnesses.
51. Developer SDK
Provide an SDK for external harness adapters.
Conceptually:
SDK should provide:
52. API Design
Suggested API groups:
Streaming:
Or SSE where appropriate.
The frontend should never connect directly to arbitrary harness containers.
53. Example Session Start
54. Example User Experience
Harness Marketplace
Session
The UI is driven by capabilities and schemas rather than hardcoded harness assumptions.
55. Database Model
Core tables/collections:
Do not put all harness-specific state into a giant universal table. Use metadata + JSON/document fields where appropriate.
56. Recommended Infrastructure
A practical starting stack:
Choose managed services where possible.
57. Initial Deployment Architecture
Do not start with dozens of microservices.
Start as a modular control-plane application:
Separate runtime workers:
Split services only when scaling pressure justifies it.
58. Scaling Strategy
Stage 1 — MVP
Support:
Stage 2 — Production
Add:
Stage 3 — Large Scale
Add:
59. Security Model
Security boundaries:
Principles:
60. Important Edge Cases
Design for:
61. Upgrade Strategy
Never upgrade active sessions implicitly.
Example:
Upgrade flow:
Keep old versions available for rollback.
62. Runtime Warm Pools
At scale, container startup can be expensive.
Maintain:
Scheduler consumes a warm runtime when possible.
Do not keep user-specific secrets/workspaces inside reusable warm containers.
63. Tenant Isolation Levels
Support tiers:
Enterprise tenants can receive stronger isolation.
64. Data Retention
Define retention independently for:
Allow tenant-specific retention policies.
65. Disaster Recovery
Back up:
Workspaces and checkpoints should be recoverable according to their retention policy.
Define:
before enterprise launch.
66. Recommended Build Order
Phase 0 — Architecture
Phase 1 — Foundation
Phase 2 — Harness Runtime
Start with 2–3 harnesses.
Implement:
Phase 3 — Model Layer
Phase 4 — MCP
Phase 5 — Dynamic UI
Phase 6 — State
Phase 7 — Production
Phase 8 — Ecosystem
67. MVP Scope
Do NOT implement everything initially.
Recommended MVP:
Delay:
until the execution path is stable.
68. Golden Execution Path
The most important path to get right:
69. Critical Design Rules
latestfor production execution.70. Final Target Architecture
71. Success Criteria
The architecture is successful when you can add a new harness by doing approximately:
without modifying:
That is the central extensibility goal.
72. Final Architecture Philosophy
The platform should behave like an orchestration layer for heterogeneous agent runtimes, not like a giant wrapper that tries to rewrite every harness.
The platform owns:
The harness owns:
The adapter connects the two worlds.
That boundary is what allows the platform to scale from the initial 15 harnesses to many more without turning the codebase into a collection of special cases.
All reactions