chore(docker): shrink prod image from >5GB to ~606MB for Railway#101
chore(docker): shrink prod image from >5GB to ~606MB for Railway#101JohnRDOrazio wants to merge 4 commits intodevfrom
Conversation
- Move sentence-transformers to an optional [local-embeddings] extra so the default install skips PyTorch (~2–3GB saved). - Multi-stage Dockerfile.prod: builder installs into /opt/venv, slim runtime copies only the venv. Drops libgit2-dev (pygit2 wheels bundle libgit2), pip cache, build tools, and the sentence-transformers model pre-download. - local_provider raises a RuntimeError pointing at the [local-embeddings] extra when sentence-transformers is missing at runtime. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 28 minutes and 1 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe pull request optimizes the Docker image by implementing a two-stage build process that isolates dependency installation, removes pre-downloaded model artifacts, and restructures Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependent services: what Railway will and won't host for us
Per-service options
Free-tier caveatRailway's free/hobby tier has a total resource budget across all services in the project. Running API + Postgres + Redis + MinIO + Zitadel + Zitadel-Login in one project will very likely exceed it. Image size was the first wall; RAM/egress will be the next. Recommended shape for staying on the free tier
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Making sentence-transformers an optional extra broke two checks in CI
(which installs the default extras-free environment):
- mypy could not resolve the sentence_transformers import — added a
per-module override with ignore_missing_imports.
- test_creates_new_config_when_none_exists triggered the real
get_embedding_provider("local", …).dimensions path, which now raises
our friendly RuntimeError. Mocked get_embedding_provider in the test,
matching the pattern already used elsewhere in the file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Dockerfile.prod`:
- Line 3: The Dockerfile.prod currently uses the untested base image
"python:3.13-slim" (seen in the FROM python:3.13-slim AS builder and the later
python:3.13-slim runtime stage); update both occurrences to "python:3.12-slim"
so the production image matches the supported/tested Python versions in
pyproject.toml and your tooling, or alternatively add Python 3.13 to pyproject
classifiers and CI before keeping 3.13.
- Around line 21-25: The builder stage copies only ontokit/version.py before
running pip install ., which produces an incomplete installed package and can
cause alembic migrations to import broken installed modules (e.g.,
ontokit.core.database, ontokit.models, ontokit.core.config); change the
Dockerfile builder step to COPY the entire ontokit/ directory (instead of just
ontokit/version.py) into /build before RUN pip install . so the installed
wheel/sdist is complete and consistent with the runtime source layout.
In `@ontokit/services/embedding_providers/local_provider.py`:
- Around line 17-24: Replace the static top-level import of SentenceTransformer
with a dynamic import using importlib.import_module so the optional dependency
is only required when actually instantiating the provider: inside the code path
that constructs or accesses the LocalEmbeddingProvider (e.g., within
get_embedding_provider() or the LocalEmbeddingProvider.__init__/factory method)
call importlib.import_module("sentence_transformers") and then get
SentenceTransformer = module.SentenceTransformer; keep the same
ImportError->RuntimeError conversion (raise RuntimeError(...) from exc) so
callers still get the friendly message, and ensure you only access the
dimensions property after the dynamic import so default installs without the
optional extra don’t trigger MyPy/ runtime import failures.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a7449f71-893f-4066-bc70-fc6178704b32
📒 Files selected for processing (3)
Dockerfile.prodontokit/services/embedding_providers/local_provider.pypyproject.toml
| FROM python:3.13-slim | ||
| # OntoKit API Production Dockerfile (multi-stage) | ||
| # ---------- Stage 1: builder ---------- | ||
| FROM python:3.13-slim AS builder |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Compare Docker production runtime against declared Python support/tooling targets.
rg -n -C2 'python:3\.13|Programming Language :: Python|python_version|target-version' Dockerfile.prod pyproject.tomlRepository: CatholicOS/ontokit-api
Length of output: 1202
Align the production Python version with supported/tested versions.
The Dockerfile runs Python 3.13, but pyproject.toml declares support for only Python 3.11 and 3.12, and tooling targets Python 3.11. This creates a production runtime that has never been tested or validated.
Change to python:3.12-slim on lines 3 and 33, or add Python 3.13 to classifiers and CI coverage before deployment.
Suggested change for Python 3.12
-FROM python:3.13-slim AS builder
+FROM python:3.12-slim AS builder-FROM python:3.13-slim AS runtime
+FROM python:3.12-slim AS runtime🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@Dockerfile.prod` at line 3, The Dockerfile.prod currently uses the untested
base image "python:3.13-slim" (seen in the FROM python:3.13-slim AS builder and
the later python:3.13-slim runtime stage); update both occurrences to
"python:3.12-slim" so the production image matches the supported/tested Python
versions in pyproject.toml and your tooling, or alternatively add Python 3.13 to
pyproject classifiers and CI before keeping 3.13.
| try: | ||
| from sentence_transformers import SentenceTransformer | ||
| except ImportError as exc: | ||
| raise RuntimeError( | ||
| "The local embedding provider requires 'sentence-transformers'. " | ||
| "Install it with: pip install 'ontokit[local-embeddings]' " | ||
| "(or choose the 'openai' or 'voyage' provider instead)." | ||
| ) from exc |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Confirm that sentence-transformers is optional while local provider remains reachable
# from default embedding configuration paths.
printf '\nOptional dependency declaration:\n'
rg -n -C2 'local-embeddings|sentence-transformers' pyproject.toml
printf '\nStatic optional import sites:\n'
rg -n -C3 'from sentence_transformers import SentenceTransformer' --iglob '*.py'
printf '\nDefault local provider paths:\n'
rg -n -C3 'cfg else "local"|provider_name = cfg.provider|get_embedding_provider\(config.provider' --iglob '*.py'Repository: CatholicOS/ontokit-api
Length of output: 2851
🏁 Script executed:
#!/bin/bash
# Check the LocalProvider class structure and when _get_model is called
printf '\n=== LocalProvider class initialization ===\n'
sed -n '1,60p' ontokit/services/embedding_providers/local_provider.py | cat -n
# Check if _get_model is called in __init__ or lazily
printf '\n=== Check for properties/methods that trigger _get_model ===\n'
rg -n 'def (dimensions|__init__|embed)' ontokit/services/embedding_providers/local_provider.py -A5
# Check the EmbeddingService singleton pattern
printf '\n=== EmbeddingService instantiation pattern ===\n'
rg -n 'def get_embedding_service|class EmbeddingService' ontokit/services/embedding_service.py -A3
# Check base provider to see if dimensions is a property
printf '\n=== Base EmbeddingProvider ===\n'
rg -n 'class EmbeddingProvider|def dimensions' ontokit/services/embedding_providers/base.py -A3Repository: CatholicOS/ontokit-api
Length of output: 3884
🏁 Script executed:
#!/bin/bash
# Check if MyPy is configured and what rules are enabled
printf '\n=== MyPy/Ruff configuration ===\n'
if [ -f pyproject.toml ]; then
sed -n '/\[tool.mypy\]/,/^\[/p' pyproject.toml | head -20
echo '---'
sed -n '/\[tool.ruff\]/,/^\[/p' pyproject.toml | head -30
fi
# Check how get_embedding_provider is called
printf '\n=== get_embedding_provider function signature ===\n'
rg -n 'def get_embedding_provider' ontokit/services/embedding_providers/__init__.py -A10Repository: CatholicOS/ontokit-api
Length of output: 889
🏁 Script executed:
#!/bin/bash
# Search for any type stubs or imports that might handle optional dependencies
printf '\n=== Optional dependency handling patterns ===\n'
rg -n 'TYPE_CHECKING|Optional\[|try:.*import|importlib' ontokit/services/embedding_providers/ --iglob '*.py' -B2 -A2Repository: CatholicOS/ontokit-api
Length of output: 112
Use dynamic import for optional sentence-transformers dependency.
With sentence-transformers optional in the default install, the static import at line 18 will fail MyPy strict type checking. Additionally, the dimensions property is accessed immediately when get_embedding_provider() is called (embedding_service.py:110), meaning default projects without the optional extra will hit this code path at runtime.
Use importlib.import_module() to defer the import and avoid breaking strict type checking in default environments:
Proposed fix
import asyncio
+from importlib import import_module
import logging
+from typing import Any, cast
from ontokit.services.embedding_providers.base import EmbeddingProvider
@@
"""Get or load a sentence-transformers model (cached)."""
if model_name not in _models:
try:
- from sentence_transformers import SentenceTransformer
- except ImportError as exc:
+ sentence_transformers = import_module("sentence_transformers")
+ except ModuleNotFoundError as exc:
+ if exc.name != "sentence_transformers":
+ raise
raise RuntimeError(
"The local embedding provider requires 'sentence-transformers'. "
"Install it with: pip install 'ontokit[local-embeddings]' "
"(or choose the 'openai' or 'voyage' provider instead)."
) from exc
+ SentenceTransformer = cast(Any, getattr(sentence_transformers, "SentenceTransformer"))🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@ontokit/services/embedding_providers/local_provider.py` around lines 17 - 24,
Replace the static top-level import of SentenceTransformer with a dynamic import
using importlib.import_module so the optional dependency is only required when
actually instantiating the provider: inside the code path that constructs or
accesses the LocalEmbeddingProvider (e.g., within get_embedding_provider() or
the LocalEmbeddingProvider.__init__/factory method) call
importlib.import_module("sentence_transformers") and then get
SentenceTransformer = module.SentenceTransformer; keep the same
ImportError->RuntimeError conversion (raise RuntimeError(...) from exc) so
callers still get the friendly message, and ensure you only access the
dimensions property after the dynamic import so default installs without the
optional extra don’t trigger MyPy/ runtime import failures.
The builder previously copied only ontokit/version.py before 'pip install .', which produced a stub install in site-packages (only version.py, no __init__.py or submodules). At runtime the stub was shadowed by the full source copied into WORKDIR — so things worked by accident, via uvicorn's --app-dir insertion and python -m alembic putting CWD first on sys.path. Copy the full ontokit/ tree in the builder so 'pip install .' produces a complete, self-contained installed package. Drop the now-redundant runtime source copy — site-packages is the authoritative location. Also sync uv.lock to reflect sentence-transformers being an optional extra (missed in the previous commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@damienriehl I started looking into railway, I made an account on the free tier and tried "building" ontokit-api, but the build failed because the image was over 5GB and the free tier only allows 4GB images. So the current PR is just exploratory, to see if it's possible to deploy to railway with a smaller footprint; I'm also seeing that the deploy doesn't include postgresql or zitadel, which would have to be deployed as separate railway deploys (but then how do separate railway deploys communicate with each other? seems a bit complicated to me, perhaps it's easier to test the services on a dedicated VPS). I do actually have a VPS that we could use to test this, from hostinger (I purchased it while I was traveling to India last month so that I could continue working on projects with Claude Code from remote); only thing that would be required to test successfully on the VPS is a domain name. But perhaps it's still early to purchase a domain name? Or do we want to go ahead and purchase a domain, that we can then hand over to the foundation when there is some funding? I could do so on gandi where I purchase all of my domains, but I'm really at my spending limit for now with the hostinger VPS purchase on top of the OVH VPS that I've had for years now 🤷 Maybe we just wait out a little longer in the hopes that some funding will soon arrive. |
…issing Adds a unit test that poisons sys.modules["sentence_transformers"] to simulate the optional extra not being installed, then verifies that LocalEmbeddingProvider.dimensions raises a RuntimeError pointing at the [local-embeddings] extra and preserves the original ImportError as its cause. Restores patch coverage on the new try/except branch introduced in 4447489. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 participant
Fixes the Railway build blocker from #100 — the 4 GB image-size cap on the free/hobby tier.
Summary
sentence-transformersis now an optional extra (ontokit[local-embeddings]). It was pulling in PyTorch + transformers + scipy (~2–3 GB) for a provider that is only one of three (local / openai / voyage). Install the extra if you want to run the local embedding provider.Dockerfile.prodis now multi-stage. Builder installs deps into/opt/venv, runtime stage copies only the venv. Build tools, pip cache,libgit2-devheaders (pygit2 wheels bundle libgit2), and the sentence-transformers model pre-download are all gone from the final image.LocalEmbeddingProviderraises a friendlyRuntimeErrorpointing atpip install 'ontokit[local-embeddings]'ifsentence_transformersis missing at runtime.Result
docker build -f Dockerfile.prod)Test plan
docker build -f Dockerfile.prod .succeedsdocker imagesreports ~606 MBdocker run --entrypoint python … -c "import ontokit.main"loads the FastAPI app withoutsentence_transformersinstalledLocalEmbeddingProvider().dimensionsraises the friendlyRuntimeErrorwhen the extra is not installedBehavior change (heads up)
Installs using default dependencies (including the Railway deployment) cannot use the
localembedding provider untilontokit[local-embeddings]is installed. Projects withprovider = localwill hit the newRuntimeErrorat first embedding call. Users on Railway should pickopenaiorvoyage.🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
Chores
local-embeddingsextraImprovements