Update lakebase-autoscale skill with revised CLI commands and new fea…

[dustinvannoy-db]

…tures

• Fix autoscaling spread constraint: 8 CU → 16 CU across SKILL.md and computes.md
• Fix scale-to-zero wake-up latency: few hundred ms → ~100ms
• Update token refresh guidance: 50 min → 45 min (Claude suggested this because of this link, either seems ok)
• Move synced-table CLI from databricks database to databricks postgres group (v0.294.0+)
• Update SDK module from databricks.sdk.service.database to databricks.sdk.service.postgres
• Correct reverse-ETL throughput figures: snapshot 2k rows/s/CU, incremental 150 rows/s/CU (based on docs)
• Add High Availability section (secondaries vs read replicas, HA constraints)
• Add Data API section (PostgREST-compatible HTTP CRUD, Autoscaling-only)
• Add Lakehouse Sync Beta section (Postgres → UC Delta, AWS only)
• Add databricks apps init --features lakebase command and list-endpoints command

[dustinvannoy-db]

@cankoklu-db can you please review this ground truth and try evaluating these changes?

[cankoklu-db]

Review: Approve ✅ — Follow-ups Applied

The CLI-first direction is correct and technically validated. Three follow-ups identified in review are now applied.

---

Technical Facts Confirmed

Fact	main (old)	PR #497 (new)	Verified
Default endpoint name	ep-primary (wrong)	primary	✅ Live list-endpoints after create-project
Autoscaling spread	≤ 8 CU (wrong)	≤ 16 CU	✅ API rejects: "max - min must be <= 16 CU"
Token TTL	implied ~1h	exactly 3600s	✅ Live JWT decode; expire_time field present
Refresh cadence	implicit	45 min explicit	✅ Correct conservative buffer

---

Why the Eval Scores Looked Worse (They Weren't)

Proxy eval: main 0.690 → PR #497 0.216. The entire drop was one thing: ground_truth.yaml was written for the SDK approach and never updated. Judges were grading CLI answers against SDK expected facts.

Task 002 actual response WITH the PR #497 skill:

databricks postgres create-branch projects/my-app development \
    --json '{"spec": {"source_branch": "projects/my-app/branches/production", "ttl": "604800s"}}'

Correct — but judge marked all expected facts missing because it expected w.postgres.create_branch(), BranchSpec, Duration(seconds=604800). The skill was working. The rubric was wrong.

---

Three Follow-ups Applied

1. ground_truth.yaml — CLI facts for tasks 001–009. SDK expected_facts/expected_patterns replaced with CLI equivalents for management-plane tasks. Connection and DDL facts made approach-agnostic. Confirmed references/*.md are not loaded at eval time — evaluator reads only SKILL.md.

2. SKILL.md — psycopg3 snippet added inline to Credentials section. Moving it to references/connection-patterns.md caused a regression where task 003 responses defaulted to import psycopg2. Fixed by adding the snippet directly inline.

3. ep-primary → primary in ground_truth.yaml (tasks 007, 008, 009) + references/computes.md line 7. ep-primary was the direct cause of the task 009 floor (score 0.000 — agent used wrong endpoint path, connection failed).

---

Post-Fix Eval Results

Overall proxy score: 0.216 → 0.609 (+0.393)

Task	Pre-fix	Post-fix	corr	comp	guide	Δ
001 create_project	~0.19	0.608	no	yes	yes	+0.50
002 create_branch	~0.19	0.825	yes	yes	yes	+1.00
003 connect_notebook	~0.21	0.825	yes	yes	yes	+1.00
005 resize_compute	~0.19	0.825	yes	yes	yes	+1.00
006 cli_reference	~0.20	0.200	no	no	no	+0.00
007 full_project_setup	~0.16	0.442	no	yes	no	+0.50
008 schema_ddl	~0.22	0.505	yes	yes	yes	+0.00
009 support_cases_ddl	~0.21	0.640	yes	yes	no	+1.00

Skill effectiveness: 0.62 (all tasks NEEDS_SKILL). Tasks 002, 003, 005 all hit 0.825. Task 006 (cli_reference) is a residual ground truth issue for a follow-on pass. Task 008 delta=+0.00 is expected — pure DDL doesn't benefit from skill context.

LGTM.