Skip to content

feat: v0.3.0 — authoritative resolver, structured extraction, --llm flag - #11

Open
nithval wants to merge 4 commits into
mainfrom
feat/v0.3.0-resolver-structured-extraction
Open

nithval wants to merge 4 commits into
mainfrom
feat/v0.3.0-resolver-structured-extraction

Conversation

@nithval

@nithval nithval commented Apr 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • Authoritative URL resolver (`internal/resolver`) — queries npm, PyPI, crates.io, and pkg.go.dev for maintainer-declared docs URLs before falling back to web search. Resolution order: package registry → curated DocHints → DuckDuckGo.
  • LLM-free structured extraction (`internal/skill/extractor`) — new `StructuredExtractor` implements the `Compiler` interface using heading-pattern matching. No API key required by default.
  • `--llm` flag — structured extraction is now the default on all commands (`add`, `scan --add-all`, `update`). Pass `--llm` to use the configured LLM provider. Config loading is skipped unless `--llm` is set.
  • Registry expanded from 26 → 65+ libraries with `CategoryForLibrary()` helper (exact + prefix matching).
  • Version bump `0.2.1 → 0.3.0`.

New files

File Purpose
`internal/resolver/resolver.go` Resolver struct, ecosystem detection, dedup
`internal/resolver/npm.go` npm registry (`registry.npmjs.org`)
`internal/resolver/pypi.go` PyPI (`pypi.org/pypi//json`)
`internal/resolver/crates.go` crates.io + pkg.go.dev for Go modules
`internal/resolver/resolver_test.go` 9 tests with `httptest` servers (no live network)
`internal/skill/extractor.go` `StructuredExtractor` + heading extraction logic
`internal/skill/extractor_test.go` 6 tests for section extraction, matching, compile output

Test plan

  • `go build ./...` — clean
  • `go test ./...` — all packages pass (15 new tests added)
  • Resolver tests use `httptest.NewServer` — no live network calls in CI
  • Existing `compiler_test.go` updated to inject noop resolver so LLM fallback path is still exercised in isolation
  • `matchSection` ordering verified: "Migration Guide" → `versions`, not `concepts`

Resolution strategy

markdocs' differentiation = authoritative doc URL resolution, not search. SkillKit curates which skill repos exist. markdocs resolves the maintainer-declared docs URL for any library in npm/PyPI/crates/Go — including the long tail that no skill repo has covered yet.

🤖 Generated with Claude Code

nithval and others added 4 commits April 19, 2026 14:08
## Authoritative URL Resolver (internal/resolver)
- New package queries npm, PyPI, crates.io, and pkg.go.dev before web search
- Uses maintainer-declared docs URLs (homepage/documentation fields in registries)
- Three-layer resolution: package registry → curated DocHints → DuckDuckGo
- Resolver base URLs are injectable for isolated testing (no live network in CI)
- 9 tests covering npm, PyPI, crates.io, Go modules, dedup, cleanGitURL

## LLM-free Structured Extraction (internal/skill/extractor)
- New StructuredExtractor implementing Compiler interface — no API key needed
- Heading-based section mapping with ordered priority (specific before broad)
- Sub-headings under matched sections preserved as content (not dropped)
- Category from expanded registry (65+ libs) with exact + prefix matching
- Description from first paragraph; when_to_use auto-generated
- 6 tests covering section extraction, heading matching, and full compile output

## --llm Flag (cmd/root.go, cmd/add.go, cmd/scan.go, cmd/update.go)
- Structured extraction is now the default — no markdocs init required
- Pass --llm to use the configured LLM provider (existing behaviour)
- Config loading skipped unless --llm is set; commands work without API key

## Registry Expansion (internal/skill/registry.go)
- Expanded from 26 to 65+ libraries across all categories
- Added CategoryForLibrary() with exact-then-prefix matching
- Added Go backends (gin, fiber, echo, chi), state management (zustand, jotai,
  redux), data fetching (@tanstack/react-query, swr), auth (clerk, lucia),
  devtools (vite, esbuild, turbo), and more

## Version bump: 0.2.1 → 0.3.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After LLM compilation, users are shown a preview of the skill and
prompted to Accept, Edit in \$EDITOR, or Discard before any file is
written. Batch/automated callers (scan --add-all, update) pass
noInteractive=true to skip the prompt.

- ui: add ReviewDraft(), openInEditor(), ReviewAction constants
- pipeline: add noInteractive bool param; insert review gate between
  compile and write; nil,nil return signals user discard
- add/scan/update: thread the flag through to the pipeline call

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
markdocs draft <name> accepts free-form notes, documentation, or any
text content (via interactive textarea or stdin pipe) and compiles it
into a structured skill file using the configured compiler. Applies the
same draft-review step as `add` before writing to disk.

  cat api-docs.md | markdocs draft internal-api
  markdocs draft internal-api   # interactive textarea

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Update README and CLAUDE.md to reflect the new draft-review workflow
and the markdocs draft command. Reframes the tool from "auto-compile
library docs" to "draft and create custom skills from documentation".

- README: new tagline, 6-step How It Works, markdocs draft section,
  updated add flags table, keep badges
- CLAUDE.md: new description, updated directory structure, pipeline
  and ReviewDraft docs, draft command example

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant