Skip to content

feat: add local-first data layer for BI, workflow analytics, and dashboard-ready metrics#1199

Open
danielfoch wants to merge 2 commits intogarrytan:mainfrom
danielfoch:feat/gstack-data-layer
Open

feat: add local-first data layer for BI, workflow analytics, and dashboard-ready metrics#1199
danielfoch wants to merge 2 commits intogarrytan:mainfrom
danielfoch:feat/gstack-data-layer

Conversation

@danielfoch
Copy link
Copy Markdown

@danielfoch danielfoch commented Apr 25, 2026

Summary

Adds a local-first GStack Data Layer for BI, workflow analytics, cron timelines, resource tracking, and dashboard-ready metrics. It gives GStack/OpenClaw/Hermes users a way to normalize local agent activity, connect it to business events and KPIs, track scheduled runs, and export clean data plus a local visual dashboard.

Why

GStack already helps builders plan, review, QA, ship, benchmark, retro, and learn. The missing layer is business and resource visibility: what the harness is doing, which workflows are working, when crons are firing, how many agents are active, where tokens/costs are going, what outcomes they affect, and which repeated workflows are becoming reusable intelligence.

What changed

  • Added /data-layer
  • Added schemas for agent events, business events, cron runs, category rules, activity series, metric definitions, KPI observations, workflow outcomes, dashboard specs, and dashboard reports
  • Added a local export helper for dashboard-ready JSON/JSONL/CSV artifacts
  • Added flexible time buckets with --bucket hour|day|week|month
  • Added cron run start/finish tracking and Gantt-ready cron timelines
  • Added user-defined task categories for resource tracking, such as personal, admin, work, financial, coding, or custom categories
  • Added active-agent and token/cost rollups
  • Added a dependency-free local dashboard.html plus daily-report.md for screenshot/report workflows
  • Added docs for the GStack Data Layer
  • Added sanitized vertical KPI, cron, category, dashboard, and daily report examples

Non-goals

  • No model training
  • No external API calls
  • No added remote telemetry
  • No hosted dashboard
  • No heavy BI dependencies
  • No real CRM/Stripe/Shopify integrations
  • No automatic screenshot delivery without explicit user-approved channel setup
  • Not real-estate-only

Privacy model

Local-first and opt-in. Business data stays under .gstack/data-layer/ by default, which is gitignored. Examples are sanitized. This feature distinguishes local analytics from remote telemetry and does not transmit data. Raw repo slugs and raw agent IDs are hashed when exported.

Example use cases

  • Real estate/OpenClaw: lead response time, showing requests, listing-copy QA, CMA turnaround, fair-housing review flags, human edit rate, scheduled follow-up sweeps, and market digest crons.
  • Agentic AI operations: agent runs by hour/day/week/month, cron runs per day, active agents, token/cost use by period, cost per successful run, acceptance/edit/rejection rates, error rates, category resource mix, and daily dashboard screenshot summaries.
  • SaaS/tech: activation rate, support volume, MRR/ARR movement, issue-to-PR cycle time, QA pass rate, and post-deploy incidents.
  • Ecommerce, agencies/services, legal ops, and finance ops: workflow outcomes and KPI templates for common operator dashboards.

Test plan

  • Ran bun run gen:skill-docs --host all
  • Ran bun run gen:skill-docs --host all --dry-run
  • Ran bun run skill:check
  • Ran jq empty schemas/*.schema.json examples/data-layer/*.json
  • Validated examples/data-layer/*.jsonl line-by-line with jq -c empty
  • Ran bun test test/data-layer-export.test.ts
  • Smoke-tested export helper with sanitized examples and confirmed dashboard.html and time-series exports are written
  • Ran git diff --check
  • Ran focused static tests: existing guard fails on tracked large fixture browse/test/fixtures/security-bench-haiku-responses.json
  • Ran full bun test: existing local failures from path-with-spaces assumptions and existing browser assertions; E2E/evals stayed skipped because EVALS=1 was not configured

Follow-ups

  • Grounded Data Review
  • Workflow Navigator
  • Context/eval/trace exports
  • Optional downstream dashboard starters
  • User-approved daily screenshot delivery integration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant