docs: explain ONNX runtime and local-only loading options

[kv-248]

Documents how to configure ONNX session thread counts and enable_cpu_mem_arena, then populate a cache online and reuse it with local_files_only=True. It also explains the missing-cache error so offline deployments can be prepared correctly.

Addresses the runtime-options and local-only-loading parts of #579. Complements #638, which documents general cache configuration.

Validation: Python snippet syntax checks and git diff --check pass. A real-model check on Windows/Python 3.10.11 populated a fresh cache, reproduced identical embeddings in a fresh local-only process with Python socket/DNS calls blocked (zero attempts), verified the empty-cache ValueError, and confirmed both ONNX thread counts and the memory-arena setting. See the validation follow-up comment for versions and scope. Full repository tests were not run locally.

AI assistance was used to prepare and review this change.

All Submissions:

• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 646ec7e4-1782-4309-8fa2-038a7a8e1373

📥 Commits

Reviewing files that changed from the base of the PR and between 0dab99c and a05599d.

📒 Files selected for processing (1)

• README.md

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

README.md adds an “ONNX Runtime and local-only loading” section. The section documents thread and CPU memory-arena settings, model cache prepopulation while online, offline initialization with local_files_only=True, and the ValueError raised when cached files are missing.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~3 minutes

Merge Risk: 🟡 Moderate · up to a0559

The new cache-preparation instructions may fail to download models into an empty cache, preventing the documented offline setup from working. Resolve the example or its described behavior before merge.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the documentation changes for ONNX Runtime configuration and local-only loading.
Description check	✅ Passed	The description directly explains the documented runtime options, cache workflow, offline loading behavior, validation, and issue scope.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kv-248]

I checked the cache-preparation concern against PR commit a05599d with the real BAAI/bge-small-en-v1.5 model in an isolated environment.

• Online initialization into a fresh cache succeeded; embedding two documents produced float32 vectors of shape (2, 384).
• In a fresh Python process, local_files_only=True reused that cache and produced exactly equal vectors (maximum absolute difference 0.0). Python socket connection and DNS functions were patched to reject and record attempts; none were recorded.
• With a separate empty cache, local_files_only=True raised ValueError: Could not load model BAAI/bge-small-en-v1.5 from any source. No patched network calls were attempted.
• The resulting ONNX session also reported intra-op/inter-op thread counts of 4 and enable_cpu_mem_arena=False, matching the runtime example.

Environment: Windows, Python 3.10.11, this checkout installed as FastEmbed 0.8.0, huggingface-hub 1.30.0, ONNX Runtime 1.23.2, NumPy 2.2.6. HF_HUB_OFFLINE was unset. The network check covers Python socket/DNS calls, not an OS-level network sandbox.

I couldn't reproduce the reported download failure, so I've left the examples unchanged and updated the PR's validation notes.