docs: add how-to guide for reporting AWS Bedrock AgentCore environments - #356
docs: add how-to guide for reporting AWS Bedrock AgentCore environments#356dangrondahl wants to merge 8 commits into
Conversation
Documentation-driven development for #355. The page specifies the intended UX for `kosli snapshot agentcore` before the CLI and server work is scoped, so it is reviewed on the preview environment first. Every Kosli command is marked as proposed, since neither the command nor the `agentcore` environment type exists yet. The AWS commands are real and were checked against the current AWS reference docs, not only the spike corpus. The page leads with the CI-build-and-attest argument rather than treating it as an aside: `agentcore deploy` builds the image server-side in CodeBuild, so CI never produces an artifact to attest and the snapshot has nothing to match against. That is the substance of the guide, not a footnote. Scoped to container builds. CodeZip is named as an explicit limitation. Note the reason is narrower than the spike first concluded: `codeConfiguration` does define an optional `versionId`, but it is supplied by the deployer rather than computed by AgentCore, and it was absent from the captured response because nothing set it. There is still no service-computed content hash, so the limitation stands - but "the field does not exist" would have been wrong. Recommends pinning `containerUri` by digest. The documented pattern accepts `:tag` or `@sha256:digest`, and the digest form names exactly the image that was attested and cannot be repointed later. Granularity resolved as one Kosli environment per account and region covering all runtimes, with ECS-style include/exclude filtering flags. That matches the existing ECS and Lambda reporters and keeps one snapshot diff meaningful across the whole agent fleet. Reporting reads the live version rather than the newest one: endpoints pin versions independently, so DEFAULT can be on v4 while a named endpoint still serves v2.
The previous version repeated the rationale from #355: that `agentcore deploy` building the image server-side means "CI never produces an artifact it can attest, so there is nothing to match against what is running." That is wrong, and it was the load-bearing sentence of the section. `kosli attest artifact --artifact-type oci` resolves the digest from the registry manifest and does not care who built the image, so attesting the ECR image after `agentcore deploy` produces a fingerprint that matches the snapshot perfectly well. The real problem is ordering, not attribution. `agentcore deploy` builds the image and creates or updates the runtime in one CloudFormation operation, so there is no point at which the image exists and the runtime is not already pointed at it. Anything attested afterwards describes something already serving traffic: a policy reports a violation instead of preventing one, and scans and approvals land post-release. So the recommendation stands but the argument changes: post-hoc attestation buys an audit trail, separating build from release buys a gate. Both are now documented, with the tradeoff stated, which also makes the page a better answer to "how should we structure the pipeline". Cites AWS's own GitHub Actions reference pipeline as precedent - it builds and pushes to ECR from the Dockerfile, scans with Inspector, then creates the runtime from that image, and does not use `agentcore deploy`. Attesting to Kosli occupies the same slot as the scan. This reverses a decision recorded as settled in #355 and must be called out in the PR description rather than landing as a quiet edit.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
|
Claude finished @dangrondahl's task in 1m 37s —— View job Review: PR #356 — AgentCore how-to guide
Findings
|
| Category | Count |
|---|---|
| Critical | 0 |
| Improvement | 2 |
| Suggestion | 3 |
Overall: This is a thorough, well-structured page. The proposed-feature warnings are prominent, all links validate, the writing follows the project conventions (active voice, imperative mood, American spelling, root-relative links), and the IAM policy + regex guidance are practically useful. The PR description's decision log and correction table are unusually good context for reviewers. The two Improvement items are minor polish.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
The old version was one line saying configuration is not tracked, which undersells it. For ECS or Lambda the image is most of what the workload does, so a digest is a fair proxy for behavior. An AgentCore runtime is not like that: a role swap, a new EFS mount, a widened inbound auth audience or a changed model ID in an environment variable all change what the agent can do while the image digest stays identical. Names the four fields worth capturing first - roleArn, networkConfiguration, authorizerConfiguration, filesystemConfigurations - in a table with what a change to each one means, and calls out environmentVariables separately since it is the most decisive of the set but is marked sensitive in the AWS SDKs and must never be stored as values. Stated as "not yet" with the reason rather than as a roadmap commitment, since #355 scopes configuration tracking out. Ends with the mitigation a reader can act on today: pin those fields in the IaC that creates the runtime and review changes there. Note filesystemConfigurations and requestHeaderConfiguration postdate the spike corpus, so the corpus is no longer a complete picture of the response.
The page marked deferred work in four separate places: the top warning, a prerequisites aside, the CLI section warning and the Terraform note, plus the configuration limitation. Each marker is correct where it sits, but there was nowhere a reader could look to answer "what will this eventually do". Adds one <Info> at the end of Limitations collecting the four gaps, with a cross-reference to the Terraform section rather than restating it. This is the sixth callout on the page and CLAUDE.md says to use them sparingly, so it is justified only because it consolidates rather than adding another inline marker. Framed as direction with no order or dates implied. The page is already a specification for unbuilt work, and turning its gaps into an implied schedule would overpromise on something #355 has not scoped.
The page asserted that the running digest matches what CI attested but never showed how to confirm it. Adds a fourth step reading the snapshot back with `kosli get snapshot`, with the table output and the three columns that prove the pipeline is wired up: Fingerprint against the attested digest, FLOW and COMMIT for traceability, COMPLIANCE. Notes that Kosli stores the digest without its `sha256:` prefix, since the reader is comparing it by eye against what they pushed. Also names the failure signal: FLOW of N/A with NON-COMPLIANT means the image reached the runtime without being attested, which is what attesting after release looks like from the environment side. That makes the ordering argument earlier in the section concrete rather than theoretical. `kosli get snapshot` exists today so it needs no proposed marker, but it can only show an AgentCore snapshot once the reporter ships, and it depends on the reporting section below, so the step points there explicitly rather than leaving the order implicit. Drops `kosli get snapshot` from the closing list, where it is now redundant.
REPLICAS is len(creationTimestamp), which the server builds with one entry per running instance of an artifact: one per task for ECS, one per pod for K8s. An AgentCore reporter would emit one entry per runtime, so the column would read 1 for a single runtime while AgentCore may be running many session microVMs or none at all. Left unexplained it invites the reader to treat it as a concurrency count, which it is not for this environment type. Also records the consequence of grouping by fingerprint: two runtimes serving the same image collapse into one row, so their identity is lost. Added to the known gaps, since ECS and Cloud Run already solve this with a per-type context object on the artifact (task_arn/cluster_name/service_name and kind/service_name/revision_name), and AgentCore will need the equivalent to support one environment per account and region.
Review feedback on #356. `prod-*` and `dev-*` are glob habits, not regexes. Verified with Go's regexp: `prod-*` compiles but means `prod-` followed by zero or more hyphens, so it also matches a runtime called `prod`. Corrected to `prod-.*` and `dev-.*`. Rather than only fixing the two examples, added a paragraph naming the trap: the `-regex` flags take Go regular expressions, not shell globs, they are unanchored, and `^`/`$` are needed for an exact match. The examples were copied from the CLI's own help text, so a reader arriving with the same assumption will make the same mistake on the next flag they reach. Also replaced the descriptive link text with the AWS post's actual title, so the reference is recoverable if the URL moves.
de36f02 to
7016355
Compare
| <Step title="Build for ARM64"> | ||
| AgentCore runtimes are `aarch64`. If your CI runners are `x86_64`, cross-build |
There was a problem hiding this comment.
Suggestion: Three forms appear in this step — "ARM64" (title and line 118), aarch64 (here), and linux/arm64 (the Docker flag). The Docker flag is fixed syntax, but the prose could pick one term. ARM64 is friendlier for most readers; consider using it here too:
| <Step title="Build for ARM64"> | |
| AgentCore runtimes are `aarch64`. If your CI runners are `x86_64`, cross-build | |
| <Step> | |
| AgentCore runtimes are ARM64. If your CI runners are x86_64, cross-build |
Review feedback on #356 discussion r3720360121. The prose said `aarch64` while the step title and the paragraph below both said ARM64, and the Docker flag is `linux/arm64`. AWS's own documents use ARM64 in prose, so `aarch64` was the outlier and it appeared exactly once. Dropped the code formatting from ARM64 and x86_64 too. They read as prose here, not as literals to type, unlike the `--platform linux/arm64` flag in the code block below. The suggested patch replaced `<Step title="Build for ARM64">` with a bare `<Step>`, which would have removed the step heading. Took the terminology change only.
| creates or updates the runtime in the same operation. | ||
|
|
||
| The problem is not that AWS did the building. You can attest an image you did not | ||
| build - `kosli attest artifact` with `--artifact-type oci` reads the digest from |
There was a problem hiding this comment.
Improvement: kosli attest artifact is the command shown in the code block below (line 126), but this first mention doesn't link to its CLI reference page. Consider:
| build - `kosli attest artifact` with `--artifact-type oci` reads the digest from | |
| build - `kosli attest artifact` with `--artifact-type oci` reads the digest from |
or linking to /client_reference/kosli_attest_artifact inline, similar to how kosli attest generic is linked on line 137.
| The `-regex` flags take Go regular expressions, not shell globs, and they are | ||
| unanchored. Write `prod-.*` rather than `prod-*`: the latter is valid but means | ||
| `prod-` followed by any number of hyphens, so it also matches a runtime called | ||
| `prod`. Anchor with `^` and `$` when you want an exact match. |
There was a problem hiding this comment.
Improvement: The explanation of why prod-* is wrong is technically accurate but could be clearer for readers who don't know regex. Consider simplifying:
| The `-regex` flags take Go regular expressions, not shell globs, and they are | |
| unanchored. Write `prod-.*` rather than `prod-*`: the latter is valid but means | |
| `prod-` followed by any number of hyphens, so it also matches a runtime called | |
| `prod`. Anchor with `^` and `$` when you want an exact match. | |
| The `-regex` flags take Go regular expressions, not shell globs, and they are | |
| unanchored. Write `prod-.*` rather than `prod-*`: in regex, `*` repeats the | |
| previous character, so `prod-*` matches `prod` followed by zero or more hyphens | |
| — not "anything starting with prod-". Anchor with `^` and `$` when you want an exact match. |
1 participant
Adds
tutorials/report_agentcore_envs.mdand registers it inconfig/navigation.jsonunder Reporting environments, cross-linked both ways with/tutorials/report_aws_envs.Closes #355
Do not merge this pull request now
The page describes the command
kosli snapshot agentcore. This command does not exist. The Kosli server does not have anagentcoreenvironment type. Keep this pull request as a draft until these two items are available.Each Kosli command in the page is available now, or has a clear mark that shows it is a proposal. All AWS commands are available now.
Decisions 1 and 2: review these together
Decision 1. Use one Kosli environment for each AWS account and region.
The environment contains all AgentCore runtimes. Each runtime is one artifact. The filter flags are
--runtimes,--runtimes-regex,--excludeand--exclude-regex. This agrees withkosli snapshot ecsandkosli snapshot lambda. Issue #355 did not make this decision. I made it when I wrote the page.Decision 2. A snapshot must show which runtime is which.
Kosli puts artifacts into groups by fingerprint. Two runtimes that use the same image become one row. You get a correct list of digests, but you do not know what operates where.
ECS and Cloud Run solve this problem. Each one adds a context object to the artifact:
ecs_contexthastask_arn,cluster_nameandservice_name.cloud_run_contexthaskind,service_nameandrevision_name.AgentCore needs the same type of object.
Decision 1 is correct only if the server team adds this object. If they do not add it, use one environment for each runtime. Then the name of the environment shows which runtime it is. This agrees less with the other reporters, and it makes more environments. But it does not lose data.
Decision 3: issue #355 gives the wrong reason
Issue #355 says this:
This is not correct. I put this sentence in the page before I found the error. The command
kosli attest artifact --artifact-type ocigets the digest from the registry. It does not need to know who built the image. Useagentcore deploy, then attest the ECR image, and the fingerprint agrees with the snapshot.The recommendation does not change. The reason changes. The problem is the sequence, not the builder. The command
agentcore deploybuilds the image and updates the runtime in one operation. The image is never available before the runtime uses it. An attestation after this operation describes an agent that already has traffic. A policy then shows a problem. It cannot prevent the problem. Scans and approvals are too late.The page gives both methods and the result of each one:
The page also refers to the AWS pipeline "Deploy AI agents on Amazon Bedrock AgentCore using GitHub Actions". This pipeline builds the image, sends it to ECR, scans it with Amazon Inspector, and then makes the runtime. It does not use
agentcore deploy.Corrections to issue #355
Three claims in the issue did not survive checking against the current AWS reference docs.
codeConfiguration.code.s3has noversionIdfieldCodeSha256.containerUriis a tag, so Kosli must resolve it@sha256:digest. The page now recommends pinning by digest, which names exactly the image that was attested and cannot be repointed.DEFAULTfollows the newest version but a named endpoint does not, so you can haveDEFAULTon v4 whilePRODstill serves v2.What is verified, and what is not
Verified against current AWS docs: ARM64-only (enforced by ELF header validation, fails
CREATE_FAILED), the tagged-union artifact shape, version numbers as strings, and thatbedrock-agentcore:List*/Get*plusecr:BatchGetImage/ecr:DescribeImagescovers the read chain.Not verified: nobody has run reporting end to end against a runtime deployed from a pre-built ECR image. AWS documents the pattern as first-class, including CDK's
AgentRuntimeArtifact.fromEcrRepositoryandfromImageUri, but the spike only ever deployed viaagentcore deploy. The page says so in a<Note>and does not present the invocation as a tested recipe. Proving it is the next iteration.Also worth knowing:
filesystemConfigurationsandrequestHeaderConfigurationpostdate the spike corpus, so the corpus is no longer a complete picture of theGetAgentRuntimeresponse. Do not build the reporter from it alone.Open question for reviewers
Can
agentcore package("Package agent artifacts without deploying") produce a pushed ARM64 ECR image and a digest? If it can, the recommended path gets much cheaper to adopt: keep the tooling, just split the steps. If it cannot, the recommendation really does mean droppingagentcore deployfor container builds, which is a bigger ask and the page should say so. The spike evidence suggests the container build is bound to the CloudFormation deploy, but I did not confirm it.Scope
roleArn,networkConfiguration,authorizerConfigurationandfilesystemConfigurationsas the fields that change what an agent can do while the digest stays identical. It flagsenvironmentVariablesas the most decisive of all but SDK-sensitive, so never stored as values./tutorials/report_aws_envsalongside ECS, Lambda and S3. Not this PR.Checks
config/navigation.json./tutorials/report_aws_envs.mint broken-linksfound one failure, intutorials/working_with_controls.mdx. This branch did not change that file.valefound no errors.Review feedback applied: the
--runtimes-regexexamples used glob syntax, corrected toprod-.*anddev-.*, with a paragraph naming the glob-versus-regex trap. The AWS blog link now carries the post title.