Skip to content

feat(plugins): references_dir + plugin_reference_tool for agent-visible skill references - #27

Merged
offendingcommit merged 3 commits into
mainfrom
feat/plugin-skill-references-dir
Aug 18, 2026
Merged

feat(plugins): references_dir + plugin_reference_tool for agent-visible skill references#27
offendingcommit merged 3 commits into
mainfrom
feat/plugin-skill-references-dir

Conversation

@offendingcommit

@offendingcommit offendingcommit commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Plugin skills declared via plugin_skill() can only ever expose their bare SKILL.md — there's no way to ship a companion references/ directory alongside it, so a plugin that wants to give the agent a deeper lookup doc has nowhere to put it through this SDK's own registration path. This adds that declaration surface, and a working way to actually reach it today rather than waiting on a host change.

What's here

  1. references_dir on plugin_skill()/PluginSkill/register_plugin() — forward-compatible groundwork. The actual host-side capability to serve companion files lives in hermes-agent, not here, and hasn't shipped in any released host. This is the kit half of that contract: plugins that declare references_dir today start working automatically the moment a host adds support, no second kit-side change required.
  2. plugin_reference_tool(skill, *, toolset) — makes the directory agent-visible today, without waiting on that host change. It builds an ordinary @tool function (the same mechanism every other plugin capability already uses) that lists or reads files under references_dir directly, sidestepping register_skill/skill_view entirely.

Design decisions

  • Groundwork says it's groundwork. Every doc surface (README, this repo's own skills/hermes-plugins authoring reference, docstrings) states plainly that no released host reads references_dir via register_skill yet, rather than implying a capability that doesn't exist. plugin_reference_tool exists precisely because that gap has no timeline.
  • The signature probe alone isn't trustworthy, so the call site doesn't rely on it alone. A bare Mock(spec=...) test double or a decorator applied without functools.wraps both present a (*args, **kwargs) shape that reads as "the host accepts this" even when it doesn't — confirmed by reproducing both live. register_plugin() retries once without references_dir if the host rejects it with a TypeError naming it; Python raises that error at argument binding, before the callee's body executes, so retrying is safe even against a host with side effects.
  • references_dir must actually be scoped to the skill. Review reproduced, end-to-end, that an unscoped references_dir (e.g. pointing it at /etc) turns plugin_reference_tool into an arbitrary-file-read tool using only the documented public API. Fixed by requiring references_dir to resolve as a descendant of the skill's own directory — enforced in plugin_skill() at declaration time, and re-checked defensively in plugin_reference_tool() in case PluginSkill is constructed directly, bypassing that factory.
  • A real correctness bug surfaced in review, not just a hardening gap. plugin_skill() was resolving an optional skill's missing references_dir to None before register_plugin() ever ran its own check, so the "dropped with a warning" behavior the README promised never actually logged anything for the ordinary case. Fixed by having plugin_skill() only validate existence for required skills, exactly mirroring how SKILL.md itself is already handled.
  • Default tool-name derivation now handles every name plugin_skill() itself accepts. The first cut only swapped hyphens for underscores; any skill name with an uppercase letter or leading digit (both legal per plugin_skill()'s own name pattern) produced an invalid tool name and crashed. Fixed with a proper normalize-and-guard helper, reproduced and locked in with a test.

Test plan

  • just test — 181 unit tests pass: 13 for references_dir plumbing (declaration-time/registration-time validation, the capability probe and its retry-on-rejection safety net, an explicit-named-parameter host, a **kwargs-only host, uninspectable signatures, unrelated TypeError passthrough) and 15 for plugin_reference_tool (listing, reading, custom name/description, the containment fix from both enforcement points, the tool-name fix for uppercase/leading-digit skill names, absolute-path/relative-traversal/symlink rejection, non-string file_path, empty directories, and a directory passed as file_path).
  • just test-contract against the real hermes-agent checkout could not be run in this environment — fails on main too, before this change, on an unrelated pre-existing import gap. Confirmed via direct source reading (not just the failing import) that the real host's register_skill(self, name, path, description="") has no references_dir parameter and no **kwargs, so this change's fallback path — register without it, log a warning — is exactly what fires against production today.
  • uv run python -m compileall -q hermes_plugin_kit tests — clean.

Known Residuals

Reviewed and accepted, not applied, each for a stated reason:

  • plugin_reference_tool reads are not TOCTOU-safe against a references_dir writable by an untrusted process at runtime (the containment check and the eventual read are separate syscalls, not a held file descriptor). Fine for the intended case — a static directory shipped with the plugin — insufficient if that assumption doesn't hold for a given deployment; documented in the README and the docstring rather than solved with fd-pinning, which is disproportionate for a directory authored by the plugin's own build pipeline.
  • register_plugin() is not transactional. A failure partway through registers some surfaces and not others — true of every registration surface in this function already, not specific to this change.
  • tests/test_hermes_contract.py (just test-contract) was not extended with a case exercising references_dir against the real host signature — the local hermes-agent checkout can't currently be imported in this environment for unrelated reasons (see Test plan).

Post-Deploy Monitoring & Validation

No additional operational monitoring required for the references_dir plumbing — it ships no runtime behavior change for any consumer (no plugin in this ecosystem currently passes it). plugin_reference_tool is opt-in per plugin (a plugin must explicitly call it and register the result), so it also has no default-on behavior to monitor; once a plugin adopts it, watch that plugin's own tool-call logs for the new <skill>_read_reference tool name.


Compound Engineering
Claude Code

plugin_skill() and PluginSkill now accept an optional references_dir --
a companion directory of reference files sibling to SKILL.md -- validated
the same way SKILL.md is (required unless the skill is optional, in which
case a missing directory is dropped with a warning instead of raising).

register_plugin() forwards references_dir to the host's register_skill
only when the host's live signature actually accepts it (probed via
inspect.signature, honoring both an explicit references_dir parameter
and a **kwargs catch-all), so older hosts are never called with an
argument they don't understand.

This does not yet make any directory agent-visible: as of this writing,
no released Hermes Agent host reads or serves plugin-skill companion
files (confirmed by reading hermes-agent's _serve_plugin_skill, which
hardcodes linked_files=None and never receives a directory argument).
A warning is logged naming the skill so that gap stays visible instead
of silently doing nothing. This is groundwork so plugins that already
declare references_dir start working with no further kit-side change
once a host adds support -- the actual host-side fix is out of scope
for this repo and tracked separately.
The inspect.signature() capability probe alone is unsound: a bare
Mock(spec=...) test double or a decorator applied without
functools.wraps both present a (*args, **kwargs) shape that the probe
reads as "host accepts references_dir" even when the real host
doesn't -- and the actual ctx.register_skill(**kwargs) call had no
guard, so a wrong guess crashed registration entirely. register_plugin
now retries once without references_dir on a TypeError naming it;
Python raises that error at argument binding, before the callee's body
runs, so retrying is safe even against a host with side effects.

Also fixes a real correctness bug: plugin_skill() was resolving an
optional skill's missing references_dir to None before register_plugin
ever saw it, so the "dropped with a warning" behavior documented in
the README never actually logged anything for the common case (a
references_dir that doesn't exist yet at declare time). plugin_skill()
now only validates references_dir when the skill is required, mirroring
exactly how SKILL.md's own optional handling already works, so
register_plugin's existing re-check is the single place that warns and
drops it.

Extracts _signature_accepts_kwarg as a shared, named probe (previously
inlined once for this feature and duplicated in concept from the
existing _call_session_db_evolving pattern), updates
skills/hermes-plugins/references/plugin-kit.md and README.md per this
repo's own doc-sync rule, and adds 6 more tests covering the probe's
except-branch, an explicit-named-parameter host, the retry-on-rejection
safety net, and an unrelated-TypeError passthrough.
…rences

plugin_reference_tool(skill, *, toolset, name=None, description=None)
builds an ordinary @tool function that lists or reads files under a
skill's references_dir, sidestepping register_skill/skill_view
entirely -- the agent can reach the directory's contents today, on
any host, via a normal tool call, without waiting on hermes-agent to
add companion-file support.

Review surfaced two real bugs before either landed silently:

- references_dir accepted any directory with no requirement that it
  live near the skill; pointing it at /etc (or a mounted secrets
  volume) turned this into an arbitrary-file-read tool using only the
  documented public API. plugin_skill now requires references_dir to
  resolve as a descendant of the skill's own directory, and
  plugin_reference_tool re-checks the same constraint defensively in
  case PluginSkill is constructed directly, bypassing that factory.
- The default tool-name derivation only swapped hyphens for
  underscores, so any skill name plugin_skill legally accepts but
  containing an uppercase letter or a leading digit (e.g.
  "Sample-Skill", "2fa-setup") produced an invalid tool name and
  crashed with ValueError. Fixed with a proper normalize-and-guard
  helper.

Also rejects non-string file_path with a clean error instead of
leaking a raw TypeError, and adds test coverage for the containment
fix, the naming fix, absolute-path/non-string/empty-directory/
directory-as-file_path edge cases, and registering the tool through
register_plugin like any other capability.
@offendingcommit offendingcommit changed the title feat(plugins): add forward-compatible references_dir for plugin skills feat(plugins): references_dir + plugin_reference_tool for agent-visible skill references Aug 18, 2026
@offendingcommit
offendingcommit marked this pull request as ready for review August 18, 2026 20:29
@offendingcommit
offendingcommit merged commit b56df77 into main Aug 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant