docs(msa-search-nim): document task-specific database profiles for faster startup#20
Merged
Merged
Conversation
…ster startup Add NIM_MODEL_PROFILE / NIM_MODEL_NAME guidance so users can download only the databases a task needs instead of the full ~1.4 TB set. Notably, paired/complex MSA search needs only the UniRef30 profile (~500 GB), which skips the ColabFold envdb. Updates SKILL.md, references/api.md, and references/parameters.md.
…B profiles The databases:uniref30 profile is ~490 GB dominated by two huge files; the NIM's built-in downloader is per-file-parallel and stalls on them at the NGC per-connection throttle (>80 min, measured). Document a range-parallel aria2c download (~13.5 min measured) fed to the NIM via NIM_MODEL_NAME, plus the persist-the-cache follow-up.
Self-contained script that installs the skill, downloads the databases:uniref30 profile in parallel with aria2c, warm-starts the NIM via NIM_MODEL_NAME, and runs a real paired MSA search (C1GY11 + C1HCX1) as a functional check. Validated on an H100 node: 455 GB downloaded in 804 s, NIM ready 12 s after start, paired search HTTP 200 with 500 rows/chain (colabfold_paired) — total ~14 min cold.
Single command that makes the fast path automatic: parallel aria2c download of the requested DB (skipped if already present) then NIM launch via NIM_MODEL_NAME. Idempotent — warm start (~7 s, validated) when the DB dir is populated, ~14 min cold otherwise. A plain docker run with NIM_MODEL_PROFILE cannot use aria2c (the NIM's built-in downloader is not pluggable), so this wrapper is the entry point.
test_from_scratch.sh is self-contained and does not depend on it; dropping the redundant wrapper to keep the PR focused on the documented manual fast path.
Not required by the skill; the SKILL.md/reference docs are the deliverable. Keeping the PR to documentation only.
…ended default Restructures the deployment guidance so the aria2c + NIM_MODEL_NAME fast path is presented as the recommended default (banner at the top of Local Docker + section retitle), with the plain full-download docker run demoted to an explicit fallback for the full databases:all set or a pdb70 smoke test. The NIM's built-in downloader is not pluggable, so aria2c cannot auto-trigger at runtime; this makes it the default in guidance instead.
Adds a natural-language 'Fast Local Deployment (recommended)' prompt as the first example so the prompt-level default matches the SKILL.md deployment default: parallel UniRef30 download + NIM_MODEL_NAME launch.
…t, incl. databases:all Generalizes the aria2c recommendation beyond single-DB profiles: retitles the section, broadens the lead and banner, and adds a multi-database recipe (fetch each DB version into sibling dirs under one parent, launch once via NIM_MODEL_NAME). The CDN per-connection throttle applies to every database, so parallel download helps the full set proportionally.
…escription Behavioral fix: agents copy the first executable code block and are primed by the frontmatter description. Previous edits only added prose banners, which lost to the plain docker run sitting first. Now the parallel-download + NIM_MODEL_NAME recipe is the first code block in Local Docker, the plain docker run is demoted to an explicit 'Fallback' subsection, and the description names the fast path as the recommended default.
Run scripts/plugin_sync.py --write so the generated plugin copy under plugins/bionemo-agent-toolkit/skills/msa-search-nim/ matches the updated source (SKILL.md fast-path restructure + references). Fixes the plugin-sync CI check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Documents the MSA-Search NIM's task-specific database profiles, so users can download only the databases a task needs instead of the full ~1.4 TB set at every fresh launch.
Why
The skill currently states local setup requires "about 1.4 TB / 1660 GB of NVMe storage" and describes only the full download. In practice the NIM supports
NIM_MODEL_PROFILE(task-specific profiles) andNIM_MODEL_NAME(custom/individual databases), which cut both storage and startup time substantially. This is in NVIDIA's configuration docs but was absent from the skill.The most impactful case: paired MSA search for protein complexes needs only UniRef30 (
databases:uniref30, ~500 GB) —colabfold_envdb_202108has no taxonomy and cannot be used for species pairing anyway. Selecting that profile skips the envdb, the single largest part of the full set.Changes
NIM_MODEL_PROFILE, the four profile tags with storage sizes, metadata verification, and a custom-databaseNIM_MODEL_NAMEblock); amended the Local Docker storage line so 1.4 TB is no longer implied to be mandatory.NIM_MODEL_PROFILEandNIM_MODEL_NAMEenv-var rows; clarified the cache-mount storage figure.databasesparam (selects among already-downloaded DBs) fromNIM_MODEL_PROFILE(controls what is downloaded at startup).Notes / correctness
list-model-profiles, matching NVIDIA's "hashes may change between releases" warning.nvcr.io/nim/colabfold/msa-search:2(NIM 2.5.0)list-model-profilesoutput and NVIDIA's published sizes.