Skip to content

Add Devin skills for testing, setup, codegen, CI, releases, templates, and triage - #1755

Draft
devin-ai-integration[bot] wants to merge 8 commits into
mainfrom
devin/1787338449-devin-skills
Draft

Add Devin skills for testing, setup, codegen, CI, releases, templates, and triage#1755
devin-ai-integration[bot] wants to merge 8 commits into
mainfrom
devin/1787338449-devin-skills

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds ten agent skills under .agents/skills/ (auto-loaded by Devin in this repo) and deletes the stale DEV.md.

Testing (scoped to the testing process only):

  • testing-js-sdk — vitest project layout (unit/template/connectionConfig/browser), which suites need E2B_API_KEY, single-file iteration, bun/deno runs, offline msw-mocked suites, CI staging-flake / staging-vs-production notes.
  • testing-python-sdk — uv/pytest usage, sync vs async layout (incl. the streaming exception to parity), offline-only runs, skip_debug marker, CI notes.
  • testing-cli — build dist/index.js and run it headlessly with E2B_API_KEY (no auth login), -d/exec -- bash -lc patterns, metadata-filter empty state, --format json validation, how vitest builds and spawns the real CLI.
  • verifying-cli-visual-output — renderer tests, direct npx tsx harness, eyeball checklist (uppercase headers, wcswidth alignment, no trailing whitespace), TTY vs piped ANSI behavior (FORCE_COLOR, script -qec), screenshot/GUI caveats.
  • setting-up-testing-environment — toolchain (.tool-versions), pnpm/uv installs, SDK/CLI builds, credential conventions (E2B_API_KEY, .env.local, ~/.e2b/config.json), smoke checks.

Operational:

  • codegen-and-specs — spec ownership (infra/belt via Copybara), spec/*-ref pins, make codegen, redocly filtering, the generated_files CI check.
  • debugging-ci — the sdk_tests.yml matrix, the required SDK Tests Status aggregate, known staging flakes, staging-before-production API skew, Cloudflare-deploy races, debug loop.
  • releasing — changesets, release.yml (main-only, preflight → tests → publish), release-candidate.yml prereleases.
  • building-templates — CLI Dockerfile flow (e2b template create, --ready-cmd, migrate), the js-sdk Template builder, the repo's templates/ and their manual build workflow.
  • triaging-issues — the automation's classification dimensions, label policy, spam handling, and triage comment format.

Also folds in insights mined from prior Devin sessions (known staging CI flakes, staging-before-production API skew, CJK/emoji width handling) and the content of the previous testing-cli-tables skill.

Commands were verified locally (CLI built and run against the real API; test entrypoints checked against package scripts and configs). Workflow claims verified against .github/workflows/. The existing skills/stripe-projects dir is untouched. No changeset: skill docs don't change the public package surface.

Link to Devin session: https://app.devin.ai/sessions/c6f4ad63608846c5840c70d5696d62d7
Requested by: @mishushakov

Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@changeset-bot

changeset-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 3daee27

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cla-bot cla-bot Bot added the cla-signed label Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Package Artifacts

Built from 744e723. Download artifacts from this workflow run.

JS SDK (e2b@2.45.1-devin-1787338449-devin-skills.0):

npm install ./e2b-2.45.1-devin-1787338449-devin-skills.0.tgz

CLI (@e2b/cli@2.17.2-devin-1787338449-devin-skills.0):

npm install ./e2b-cli-2.17.2-devin-1787338449-devin-skills.0.tgz

Python SDK (e2b==2.45.1+devin.1787338449.devin.skills):

pip install ./e2b-2.45.1+devin.1787338449.devin.skills-py3-none-any.whl

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against TASTE.md (e2b-dev/sdk-harness). This PR adds four agent skill docs and no SDK code, so most of the API-surface rules (T-3..T-27, T-31..T-74) don't apply. What is in scope is the guidance the skills give about SDK parity, since agents will follow it when they do change the surface: T-1/T-2 (three surfaces mirror 1:1, sync/async separate mirrors) and T-28 (the one sanctioned break in sync/async parity: sync Python never spawns background threads, so streaming callbacks move to wait() and watch_dir returns a polling handle).

2 violations, both the same issue: a blanket "sync and async must be equivalent / mirrored" instruction that contradicts T-28 and will push agents toward the hybrids T-28 explicitly forbids. Inline comments below.

Not tied to a changed line: I spot-checked the factual claims (vitest projects and excludes in vitest.config.mts, cli globalSetup: ['tests/setup.ts'], pytest.ini markers/timeout/pythonpath, package scripts, buildTableRows/sortSandboxes exports, --detach on sandbox create) and they match main. One suggestion for a follow-up, not a TASTE rule: none of the four skills point at the SDK design principles themselves, so an agent adding a public method learns how to test it but not that required args are positional (T-3), that lifecycles come from static factories (T-4), or that Url/Id casing is fixed (T-12). A one-line pointer in testing-js-sdk and testing-python-sdk would close that gap.

Comment thread .agents/skills/testing-js-sdk/SKILL.md Outdated
Comment thread .agents/skills/testing-python-sdk/SKILL.md Outdated
Comment thread .agents/skills/testing-python-sdk/SKILL.md Outdated
devin-ai-integration Bot and others added 7 commits August 21, 2026 18:59
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
…, and issue triage

Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
Co-Authored-By: mish@e2b.dev <mish@e2b.dev>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Added the streaming-parity qualifier in 3daee27.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

This line was removed when the skills were scoped to the testing process only (b056917).

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

This "Writing tests" section was removed when the skills were scoped to the testing process only (b056917); the remaining parity mention now carries the streaming exception (3daee27).

@devin-ai-integration devin-ai-integration Bot changed the title Add Devin skills for testing the SDKs and CLI Add Devin skills for testing, setup, codegen, CI, releases, templates, and triage Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant