refactor: remove models-store in favor of default ~/.docker/models#730
refactor: remove models-store in favor of default ~/.docker/models#730doringeman wants to merge 1 commit intodocker:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request aims to remove the local models-store in favor of the default ~/.docker/models path used by Docker Desktop. The changes in .gitignore, cmd/cli/Dockerfile, and demos/embeddings/indexer.js are correct and align with this goal. However, the modification to the Makefile breaks the docker-run functionality by removing the volume mount for the model store. I've provided a suggestion to fix this by using a default path for MODELS_PATH in the docker-run-impl target, ensuring that Docker-based runs continue to work as expected while adopting the new default model path.
| @echo "Starting service on port $(PORT)..." | ||
| @echo "Service will be available at: http://localhost:$(PORT)" | ||
| @echo "Example usage: curl http://localhost:$(PORT)/models" | ||
| @echo "" | ||
| PORT="$(PORT)" \ | ||
| MODELS_PATH="$(MODELS_PATH)" \ | ||
| DOCKER_IMAGE="$(DOCKER_IMAGE)" \ | ||
| LLAMA_ARGS="$(LLAMA_ARGS)" \ | ||
| DMR_ORIGINS="$(DMR_ORIGINS)" \ |
There was a problem hiding this comment.
These changes break the docker-run family of targets. While the Go application may have a default model path for local runs, when running in Docker, a volume must be mounted from the host to the container to share the model store. By removing MODELS_PATH from this rule, docker-run.sh will no longer mount any volume for models, and the container will be isolated from the host's ~/.docker/models directory.
To fix this while still defaulting to ~/.docker/models, you can use shell parameter expansion to provide a default value. This also keeps the ability for users to override the path via the command line (e.g., make docker-run MODELS_PATH=/custom/path).
@echo "Starting service on port $(PORT) with model storage at $${MODELS_PATH:-$${HOME}/.docker/models}..."
@echo "Service will be available at: http://localhost:$(PORT)"
@echo "Example usage: curl http://localhost:$(PORT)/models"
PORT="$(PORT)" \
MODELS_PATH="$${MODELS_PATH:-$${HOME}/.docker/models}" \
DOCKER_IMAGE="$(DOCKER_IMAGE)" \
LLAMA_ARGS="$(LLAMA_ARGS)" \
DMR_ORIGINS="$(DMR_ORIGINS)" \
762dd9c to
4087d46
Compare
The Go code already defaults MODELS_PATH to ~/.docker/models, so the Makefile override to a local models-store/ directory was unnecessary. This shares the model store with Docker Desktop's Model Runner. Signed-off-by: Dorin Geman <dorin.geman@docker.com>
4087d46 to
3731f27
Compare
The Go code already defaults MODELS_PATH to ~/.docker/models, so the Makefile override to a local models-store/ directory was unnecessary. This shares the model store with Docker Desktop's Model Runner.