PoC of repository ai bootstrap by JanKrivanek · Pull Request #7585 · dotnet/machinelearning

JanKrivanek · 2026-03-07T06:46:34Z

Motivation

Experimentation with improving the repository ai readiness through generated ai bootstrap (https://github.com/kotlarmilos/ai-dotnet-bootstrap)

==============================================

We are excited to review your PR.

So we can do the best job, please check:

There's a descriptive title that will make sense to other developers some time from now.
There's associated issues. All PR's should have issue(s) associated - unless a trivial self-evident change such as fixing a typo. You can use the format Fixes #nnnn in your description to cause GitHub to automatically close the issue(s) when your PR is merged.
Your change description explains what the change does, why you chose your approach, and anything else that reviewers should know.
You have included any necessary tests in the same PR.

kotlarmilos · 2026-03-11T11:27:41Z

@copilot Remove the skills for now. Merge README-AI.md and copilot-instructios.md, refine it with https://github.com/anthropics/skills/blob/main/skills/skill-creator/SKILL.md. For issues triage use https://github.github.com/gh-aw/ instead.

Copilot

Pull request overview

Adds “AI-native” repository scaffolding to help GitHub Copilot/agents build, test, and contribute to ML.NET, plus new GitHub Actions workflows for issue similarity detection and comment tone/heat monitoring.

Changes:

Added Copilot/AI instruction docs (global + scoped) and an onboarding README.
Introduced agent/skill markdown definitions to standardize PR/test workflows for AI agents.
Added two GitHub Actions workflows: AI similar-issue detection and an “inclusive heat sensor” for comments.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
README-AI.md	Documents the added AI-native infrastructure and how to use it.
.github/workflows/inclusive-heat-sensor.yml	Adds a workflow to detect heated/offensive language in issues/PR comments.
.github/workflows/find-similar-issues.yml	Adds an AI-assisted workflow to comment with similar issues on new issue creation.
.github/skills/verify-tests-fail/SKILL.md	Adds a skill definition to verify tests fail without a fix and pass with it.
.github/skills/try-fix/SKILL.md	Adds a single-attempt “try fix + test + revert” skill definition.
.github/skills/run-tests/SKILL.md	Adds guidance/commands for running builds and tests with filtering.
.github/skills/pr-finalize/SKILL.md	Adds a PR finalization/review checklist skill definition.
.github/skills/pr-build-status/SKILL.md	Adds guidance for reading CI status from Azure Pipelines and GitHub Actions.
.github/skills/learn-from-pr/SKILL.md	Adds a skill definition for extracting lessons from completed PRs.
.github/skills/issue-triage/SKILL.md	Adds a skill definition for triaging issues via `gh`.
.github/skills/find-reviewable-pr/SKILL.md	Adds a skill definition to find/prioritize PRs for review.
.github/skills/ai-summary-comment/SKILL.md	Adds a skill definition for posting/updating a unified PR progress comment.
.github/prompts/release-notes.prompt.md	Adds a prompt template for generating categorized release notes.
.github/instructions/tokenizers.instructions.md	Adds scoped Copilot guidance for the Tokenizers area.
.github/instructions/tests.instructions.md	Adds scoped Copilot guidance for test conventions and commands.
.github/instructions/genai.instructions.md	Adds scoped Copilot guidance for GenAI/TorchSharp projects.
.github/copilot-instructions.md	Adds repo-wide Copilot guidance: build/test commands, structure, conventions.
.github/agents/write-tests-agent.md	Adds an agent workflow for locating/writing tests per repo conventions.
.github/agents/pr/SHARED-RULES.md	Adds shared rules/config for the PR agent’s multi-phase workflow.
.github/agents/pr/post-gate.md	Adds post-gate phases for multi-model fix exploration and reporting.
.github/agents/pr.md	Adds pre-flight + gate phases for the PR agent workflow.
.github/agents/learn-from-pr.md	Adds an agent workflow to apply improvements based on PR retrospectives.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-11T11:49:07Z

README-AI.md

+| File | Type | Purpose |
+|------|------|---------|
+| `.github/copilot-instructions.md` | Generated | Global Copilot context — repo overview, build/test commands, conventions, project structure |
+| `.github/instructions/tests.instructions.md` | Generated | Test-specific guidance — xUnit patterns, base classes, assertion styles, naming |
+| `.github/instructions/genai.instructions.md` | Generated | GenAI component guidance — LLaMA/Phi/Mistral via TorchSharp, Semantic Kernel |
+| `.github/instructions/tokenizers.instructions.md` | Generated | Tokenizer guidance — BPE/WordPiece/SentencePiece, data packages |


Markdown tables are written with a leading double pipe (|| ...), which creates an extra empty first column and renders incorrectly in most markdown viewers. Use a single leading pipe (or no leading pipe) consistently for the table header/separator rows.

Copilot · 2026-03-11T11:49:08Z

.github/workflows/inclusive-heat-sensor.yml

+jobs:
+  detect-heat:
+    uses: jonathanpeppers/inclusive-heat-sensor/.github/workflows/comments.yml@v0.1.2
+    secrets: inherit


This workflow calls a reusable workflow from an external repository and also uses secrets: inherit, which forwards all available secrets to that third-party workflow. For security, avoid inheriting secrets to external workflows; explicitly pass only the minimal secrets required (often none beyond the default GITHUB_TOKEN) and consider pinning the reusable workflow to an immutable commit SHA.

Suggested change

secrets: inherit

Copilot · 2026-03-11T11:49:08Z

.github/workflows/find-similar-issues.yml

+        with:
+          node-version: '20'
+
+      - run: npm init -y && npm install @octokit/rest


npm install @octokit/rest without a version (and without a lockfile) makes the workflow non-deterministic and increases supply-chain risk, especially since the job has issues: write. Prefer actions/github-script (built-in Octokit) or pin a specific package version and use a checked-in lockfile (npm ci).

Suggested change

- run: npm init -y && npm install @octokit/rest

- run: npm init -y && npm install @octokit/rest@20.0.2

Copilot · 2026-03-11T11:49:08Z

.github/workflows/find-similar-issues.yml

+            const words=extractWords(`${title}\n${body}`);
+            const candidates=issues.map(i=>({issue:i,score:jaccard(words,extractWords(`${i.title}\n${i.body||''}`))}))
+              .filter(c=>c.score>0.1).sort((a,b)=>b.score-a.score).slice(0,30);
+
+            const results=[];
+            for(const{issue}of candidates){
+              try{
+                const r=await fetch(`${endpoint}/chat/completions`,{method:"POST",headers:{"Content-Type":"application/json","Authorization":`Bearer ${token}`},
+                  body:JSON.stringify({model,temperature:0.3,max_tokens:150,messages:[


This workflow can make up to ~30 model inference calls per newly opened issue (one per candidate in the candidates list). On a high-traffic repo this can become slow/expensive and may hit rate limits. Consider reducing the candidate cap, adding an opt-in label/command gate, and/or early-exiting after collecting enough high-confidence matches.

Copilot · 2026-03-11T11:49:09Z

.github/skills/find-reviewable-pr/SKILL.md

+# Recent needing review
+gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt


date -v-14d is macOS/BSD-specific and will fail on Linux, which is a common environment for running these commands. Use a portable alternative (e.g., GNU date -d '14 days ago' ...) or document separate commands per OS.

Suggested change

# Recent needing review

gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt

# Recent needing review (created in last 14 days)

# macOS / BSD:

gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt

# Linux (GNU date):

gh pr list --search "review:none created:>=$(date -d '14 days ago' +%Y-%m-%d)" --json number,title,author,createdAt

Copilot · 2026-03-11T11:49:09Z

.github/skills/try-fix/SKILL.md

+```bash
+git stash
+dotnet test test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj  # Confirm failure
+```
+
+### 2. Implement one fix
+Minimal changes. Read prior attempts (if any) and do something different.
+
+### 3. Test
+```bash
+./build.sh
+dotnet test test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj
+```
+
+### 4. Report
+```markdown
+## Try-Fix Attempt #N
+**Approach**: [what and why]
+**Changes**: `path/file` — [what changed]
+**Build**: ✅/❌
+**Tests**: ✅/❌
+**Verdict**: ✅ FIX WORKS / ❌ FAILED — [reason]
+```
+
+### 5. Revert
+```bash
+git checkout -- .
+git stash pop
+```


This skill relies on git stash/git stash pop. If there are no local changes when git stash runs, it won't create a stash entry and git stash pop will fail. Consider making the workflow robust (check for changes / stash existence) or use a deterministic cleanup approach like git checkout HEAD -- . + git clean -fd (which you already recommend elsewhere).

Copilot · 2026-03-11T11:49:09Z

.github/agents/pr/SHARED-RULES.md

+The agent should not run git commands that change branch state during PR review. Use read-only commands:
+
+- ✅ `gh pr diff`, `gh pr view`, `gh issue view`
+- ❌ `git checkout`, `git switch`, `git stash`, `git reset`
+
+Exception: `git checkout HEAD -- .` and `git clean -fd` are allowed for cleanup between try-fix attempts.


There’s an internal inconsistency: SHARED-RULES forbids git stash/git checkout during PR review, but the try-fix and verify-tests-fail skills instruct using git stash (and post-gate.md instructs git checkout HEAD -- .). Please align these docs (either broaden the allowed git commands for these phases, or update the skills to use the permitted cleanup commands) so the agent isn’t guided to violate its own rules.

Suggested change

The agent should not run git commands that change branch state during PR review. Use read-only commands:

- ✅ `gh pr diff`, `gh pr view`, `gh issue view`

- ❌ `git checkout`, `git switch`, `git stash`, `git reset`

Exception: `git checkout HEAD -- .` and `git clean -fd` are allowed for cleanup between try-fix attempts.

The agent should not run git commands that change branches, rewrite history, or otherwise alter repository state beyond what is needed for local cleanup during PR review.

- ✅ Read-only: `gh pr diff`, `gh pr view`, `gh issue view`

- ✅ Cleanup-only on the current branch (between try-fix / verify-tests-fail attempts): `git checkout HEAD -- .`, `git clean -fd`, `git stash push -u`, `git stash pop` (or `git stash apply`)

- ❌ Branch / history changes: `git checkout <branch>`, `git switch <branch>`, `git reset`, `git rebase`, `git commit --amend`, `git push --force`

**Rules for cleanup commands:**

- Use them only to restore a clean working tree between automated attempts on the **current** branch.

- Never use them to switch branches or modify commit history.

- Do not leave changes stashed at the end of a phase; if you `git stash push`, you must immediately `git stash pop`/`apply` (or otherwise ensure the stash is cleared as part of the cleanup flow).

Copilot · 2026-03-11T11:49:10Z

.github/skills/verify-tests-fail/SKILL.md

+# 1. Remove the fix, keep the tests
+git stash push -m "fix" -- <FIX_FILES>
+
+# 2. Build and run — should FAIL
+./build.sh
+dotnet test test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj --filter "FullyQualifiedName~RelevantTests"
+# Expected: ❌ FAIL (proves tests catch the bug)
+
+# 3. Restore fix
+git stash pop
+
+# 4. Build and run — should PASS
+./build.sh
+dotnet test test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj --filter "FullyQualifiedName~RelevantTests"
+# Expected: ✅ PASS (proves fix works)


This verification flow assumes git stash push ... always creates a stash, but if <FIX_FILES> have no local modifications (e.g., fix already committed or working tree clean), git stash pop will fail. Consider adding a guard/check for a created stash entry or documenting an alternative revert path (git checkout HEAD -- <FIX_FILES>).

kotlarmilos · 2026-03-11T12:02:15Z

@copilot Remove the skills for now. Merge README-AI.md and copilot-instructios.md, refine it with https://github.com/anthropics/skills/blob/main/skills/skill-creator/SKILL.md. For issues triage use https://github.github.com/gh-aw/ instead.

initiate ai

ce63e62

dotnet-policy-service bot assigned JanKrivanek Mar 7, 2026

JanKrivanek marked this pull request as ready for review March 11, 2026 11:44

Copilot AI review requested due to automatic review settings March 11, 2026 11:44

Copilot started reviewing on behalf of JanKrivanek March 11, 2026 11:45 View session

Copilot AI reviewed Mar 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PoC of repository ai bootstrap#7585

PoC of repository ai bootstrap#7585
JanKrivanek wants to merge 1 commit intomainfrom
dev/jankrivanek/ai-bootstrap

JanKrivanek commented Mar 7, 2026

Uh oh!

kotlarmilos commented Mar 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

Copilot AI Mar 11, 2026

Uh oh!

kotlarmilos commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	- run: npm init -y && npm install @octokit/rest
	- run: npm init -y && npm install @octokit/rest@20.0.2

		# Recent needing review
		gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt

-# Recent needing review
-gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt
+# Recent needing review (created in last 14 days)
+# macOS / BSD:
+gh pr list --search "review:none created:>=$(date -v-14d +%Y-%m-%d)" --json number,title,author,createdAt
+# Linux (GNU date):
+gh pr list --search "review:none created:>=$(date -d '14 days ago' +%Y-%m-%d)" --json number,title,author,createdAt

-The agent should not run git commands that change branch state during PR review. Use read-only commands:
-- ✅ `gh pr diff`, `gh pr view`, `gh issue view`
-- ❌ `git checkout`, `git switch`, `git stash`, `git reset`
-Exception: `git checkout HEAD -- .` and `git clean -fd` are allowed for cleanup between try-fix attempts.
+The agent should not run git commands that change branches, rewrite history, or otherwise alter repository state beyond what is needed for local cleanup during PR review.
+- ✅ Read-only: `gh pr diff`, `gh pr view`, `gh issue view`
+- ✅ Cleanup-only on the current branch (between try-fix / verify-tests-fail attempts): `git checkout HEAD -- .`, `git clean -fd`, `git stash push -u`, `git stash pop` (or `git stash apply`)
+- ❌ Branch / history changes: `git checkout <branch>`, `git switch <branch>`, `git reset`, `git rebase`, `git commit --amend`, `git push --force`
+**Rules for cleanup commands:**
+- Use them only to restore a clean working tree between automated attempts on the **current** branch.
+- Never use them to switch branches or modify commit history.
+- Do not leave changes stashed at the end of a phase; if you `git stash push`, you must immediately `git stash pop`/`apply` (or otherwise ensure the stash is cleared as part of the cleanup flow).

Conversation

JanKrivanek commented Mar 7, 2026

Motivation

Uh oh!

kotlarmilos commented Mar 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

kotlarmilos commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants