[workshop-sim] Workshop Simulation Report — 2026-09-11 (Run #7, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-09-11
• Students simulated: 46 × 1000 Monte Carlo runs
• Workshop steps available: 30/30
• Overall success rate: 0% (95% Monte Carlo interval: 0%–0.01%)
• Highest-dropout step: 08-run-your-workflow (100% conditional dropout among 14,970 at-risk runs; 95% Monte Carlo interval: 99.97%–100%)
• Lowest curriculum quality step: 04-github-actions-intro.md (overall score 5.39/10)
• Learning KPI index: 2.9/10 (active_learning 4.2 · checkpoint_quality 0.0 · scaffolding 5.0)
• Model: 2026-07-survival-model-v2 / 2026-07-assumption-model-v2 (parameter hash 2024391902)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	15	6.69 / 10.0	±1.74
Part 2 — advanced (lessons 15+)	15	6.06 / 10.0	±0.37
Overall corpus	30	6.37 / 10.0	±1.19

No pages are classified as other; all 30 main steps fall into Part 1 or Part 2.

Critical Findings

1. Every simulated learner (100%) fails at 08-run-your-workflow. The step's own pre-flight checks reference billing/compile state that is repeatedly re-verified from Steps 7 and 7d without adding new diagnostic value, and the run condition (workflow-not-compiled) is the sole recorded failure category for this step across all 14,970 at-risk runs — this is a hard universal blocker, not a segment-specific one.
2. 07-first-workflow is the second-highest dropout step (32% conditional dropout, up from 22% before content-aware adjustment) — the terminal-only Copilot CLI pre-check gates every learner regardless of tool preference, and CCA/Agents-tab learners get no explicit /agentic-workflows authoring guidance on this page.
3. 04-github-actions-intro.md combines the highest concept density in Part 1 (28 new concepts in 756 words) with checkpoint_quality: 0.0 and no mid-page comprehension check, driving its 17.6% dropout via concept-overload.
4. Learning quality health is weak even for learners who stay in the workshop. The cohort-wide learning KPI index is 2.9/10 — every one of the 30 steps scores checkpoint_quality: 0.0 under the shared rubric (no page matches the exact ## ✅ Checkpoint heading pattern the assessor scans for), so completing a step provides little measurable reinforcement of practice.
5. The single most important repair is in Part 1 (00–14): fixing the 08-run-your-workflow compile/billing verification loop would remove the universal blocker that currently prevents any learner — regardless of level, personality, or tool — from completing the core path.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add a differential-diagnosis troubleshooting block to 08-run-your-workflow.md that helps learners distinguish a stale-lock-file failure from a mismatched-billing failure, instead of only repeating "return to Step 7d" (completion impact: ↑ · learning KPI impact: ↑, targets checkpoint_quality/scaffolding)
2. Add an explicit /agentic-workflows Agents-tab authoring callout and a short worked "your generated file looks different — here's why" comparison to 07-your-first-workflow.md (completion impact: ↑ · learning KPI impact: ↑, targets active_learning/scaffolding)
3. Insert one mid-page knowledge check between introducing trigger/job/step/runner in 04-github-actions-intro.md, and add a real ## ✅ Checkpoint heading so it is machine-recognized (completion impact: ↑ · learning KPI impact: ↑, targets checkpoint_quality/active_learning)

Dropout by step

Step	At-risk runs	Conditional dropout rate	95% MC interval	Failure mode	Top reason
08-run-your-workflow	14,970	100.0%	99.97%–100%	Access barrier	Workflow .lock.yml not compiled/committed to match the chosen billing method
07-first-workflow	22,032	32.1%	31.4%–32.7%	Access barrier	Split between authoring friction (translating the prompt into a valid workflow) and missing Copilot model access
05-agentic-intro	36,340	18.3%	17.9%–18.7%	Learning barrier	Conceptual gap in agentic vs. standard Actions distinction, plus a deployment-capability gap
04-actions-intro	44,124	17.6%	17.3%–18.0%	Learning barrier	Concept overload — 28 new terms introduced with no intermediate check
05c-agentic-practice	29,679	12.1%	11.8%–12.5%	Learning barrier	Practice activities assume mastery not yet consolidated from prior step
05b-agentic-security	26,076	9.7%	9.3%–10.1%	Learning barrier	Security concepts introduced densely without enough worked examples
06-install-gh-aw	23,546	6.4%	6.1%–6.8%	Access barrier	CLI install/auth friction
02-setup	46,000	4.1%	3.9%–4.3%	Access barrier	Codespace setup friction
00-welcome / 01-prerequisites	46,000	0.0%	0%–0.01%	—	No recorded failures

Curriculum quality and learning KPIs

Step file	Overall score	active_learning	checkpoint_quality	scaffolding	Learning KPI index	Lowest rubric dimension	Repair priority
04-github-actions-intro.md	5.39	3.9	0.0	5.0	2.65	checkpoint_quality	High
05-agentic-workflows-intro.md	5.43	2.4	0.0	5.0	2.24	checkpoint_quality	High
08-run-your-workflow.md	5.67	3.0	0.0	5.0	2.45	checkpoint_quality	Critical (access barrier)
07-your-first-workflow.md	6.25	6.3	0.0	5.0	3.65	checkpoint_quality	High
Cohort mean (30 steps)	6.37	4.17	0.0	5.0	2.88	checkpoint_quality	—

Segment breakdowns

By technical level

Level	n	Success rate
actions-user	11	0.0%
advanced	5	0.0%
beginner	11	0.0%
github-basic	19	0.0%

By personality

Personality	n	Success rate
confused	6	0.0%
curious	15	0.0%
impatient	6	0.0%
methodical	12	0.0%
skeptical	7	0.0%

By UI preference: ui_preferred: true 0.0% vs. ui_preferred: false 0.0% — no segment escapes the universal 08-run-your-workflow blocker in this run.

Notable student journeys (3)

• Unexpected dropout: Learner 002 (actions-user, devops background, CLI-preferred, 2,445 prior successes out of 6,000 runs) reached 08-run-your-workflow in 698 of 1,000 runs before failing there — strong prior mastery could not overcome the universal compile/billing verification gap.
• Content-gap case: Learner 001 (github-basic, confused, data-science background, VS Code + UI-preferred) hit 04-actions-intro (317 failures) most often before even reaching Step 7, showing the concept-overload problem compounds for learners without a coding background.
• Surprising near-miss: several advanced-level, CLI-first learners still cleared every early step (near-zero failures through 06-install-gh-aw) yet converged on the same 08-run-your-workflow wall, confirming the blocker is content-structural rather than skill-dependent.

Generated by 🔬 Workshop Student Simulator · copilot · auto · 186.5 AIC · ⌖ 20.1 AIC · ⊞ 15.2K · ◷

• expires on Sep 12, 2026, 1:56 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #3201

View newer issue

---

This action was performed automatically by the Workshop Student Simulator workflow.

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #3202

View newer issue

---

This action was performed automatically by the Workshop Student Simulator workflow.