Skip to content

Update docs for current development installation #228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
PGijsbers wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Update docs for current development installation #228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e5078de
write updated instructions and favor running containerized
PGijsbers Dec 24, 2025
cd3aac6
Add caveat to the migration guide about sync issues
PGijsbers Dec 24, 2025
ed597a2
Update docs/installation.md
PGijsbers Dec 25, 2025
204aafc
Update docs/installation.md
PGijsbers Dec 25, 2025
6c7ffc7
Update docs/installation.md
PGijsbers Dec 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .python-version
View file
Open in desktop

This file was deleted.

152 changes: 15 additions & 137 deletions docs/installation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,144 +1,22 @@
# Installation

*Current instructions tested on Mac, but likely work on most Unix systems.*
The primary way to run this service is through a Docker container.
The REST API needs to be able to connect to a MySQL database with the OpenML "openml" and "openml_expdb" databases.
The `docker-compose.yaml` file of this project defines these together out of the box.
This is useful for development purposes, but the database does not persist between restarts in the current configuration.
By default, the current code is also mounted into the Python REST API container (again, for development purposes).

The OpenML server will be developed and maintained for the latest minor release of
Python (Python 3.12 as of writing).
You can install the dependencies locally or work with docker containers.
For development, it should suffice to run the services from a fresh clone by running `docker compose --profile "python" up -d`.
The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit http://localhost:8001/docs.
Comment on lines +9 to +10
Copy link

@coderabbitai coderabbitai bot Dec 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Format bare URL as a markdown link per MD034.

Line 10 contains a bare URL that should be formatted as a markdown link to comply with markdown linting standards.

🔎 Proposed fix to format the URL as a markdown link 
- The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit http://localhost:8001/docs.
+ The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit [http://localhost:8001/docs](http://localhost:8001/docs).
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
For development, it should suffice to run the services from a fresh clone by running `docker compose --profile "python" up -d`.
The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit http://localhost:8001/docs.
For development, it should suffice to run the services from a fresh clone by running `docker compose --profile "python" up -d`.
The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit [http://localhost:8001/docs](http://localhost:8001/docs).
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)

10-10: Bare URL used

(MD034, no-bare-urls)

🤖 Prompt for AI Agents 
In docs/installation.md around lines 9 to 10, there's a bare URL
(http://localhost:8001/docs) on line 10 that violates MD034; replace the bare
URL with a Markdown link (for example: "To visit the Swagger Docs, visit
[Swagger Docs](http://localhost:8001/docs).") so the URL is formatted as link
text rather than a bare URL.


??? tip "Use `pyenv` to manage Python installations"
Once the containers are started, you can run tests with `docker exec -it openml-python-rest-api python -m pytest -m "not php_api" tests`.
For migration testing, which compares output of the Python-based REST API with the old PHP-based one, also start the PHP server (`docker compose --profile "php" --profile "python" up -d`) and include tests with the `php_api` marker/fixture: `docker exec -it openml-python-rest-api python -m pytest tests`.

We recommend using [`pyenv`](https://github.com/pyenv/pyenv) if you are working with
multiple local Python versions. After following the installation instructions for
`pyenv` check that you can execute it:
!!! note

```text
> pyenv local
3.12
```
The PHP REST API needs Elasticsearch. In some cases, it also needs the ES indices to be built.
The current set up does not automatically build ES indices, because that takes a long time.
When we start testing more upload functionality, for which the PHP API needs built indices, we'll work on an ES image with prebuilt indices.

If `pyenv` can't be found, please make sure to update the terminal environment
(either by `reset`ing it, or by closing and opening the terminal). If you get the message
`pyenv: no local version configured for this directory` first clone the repository
as described below and try again from the root of the cloned repository.

You can then install the Python version this project uses with:
`cat .python-version | pyenv install`


## Local Installation

These instructions assume [Python 3.12](https://www.python.org/downloads/)
and [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) are already installed.

!!! info "You may need to install Python3 and MySQL development headers."

It may be necessary to first install additional headers before proceeding with a
local installation of the `mysqlclient` dependency. They are documented under
["Installation"](https://github.com/PyMySQL/mysqlclient#linux) of the `mysqlclient`
documentation.

=== "For Users"

If you don't plan to make code changes, you can install directly from Github.
We recommend to install the OpenML server and its dependencies into a new virtual
environment.
```bash title="Installing the project into a new virtual environment"
python -m venv venv
source venv/bin/activate

python -m pip install git+https://github.com/openml/server-api.git
```
If you do plan to make code changes, we recommend you follow the instructions
under the "For Contributors" tab, even if you do not plan to contribute your
changes back into the project.


=== "For Contributors"

If you plan to make changes to this project, it will be useful to install
the project from a cloned fork. To fork the project, go to our
[project page](https://github.com/openml/server-api) and click "fork".
This makes a copy of the repository under your own Github account.
You can then clone your own fork (replace `USER_NAME` with your Github username):

```bash title="Cloning your fork"
git clone https://github.com/USER_NAME/server-api.git
cd server-api
```

Then we can install the project into a new virtual environment in edit mode:

```bash title="Installing the project into a new virtual environment"
python -m venv venv
source venv/bin/activate

python -m pip install -e ".[dev,docs]"
```
Note that this also installs optional dependencies for development and documentation
tools. We require this for contributors, but we also highly recommend it anyone
that plans to make code changes.

## Setting up a Database Server
Depending on your use of the server, there are multiple ways to set up your own
OpenML database. To simply connect to an existing database, see
[configuring the REST API Server](#configuring-the-rest-api-server) below.


### Setting up a new database
This sets up an entirely empty database with the expected OpenML tables in place.
This is intended for new deployments of OpenML, for example to host a private OpenML
server.

!!! Failure ""

Instructions are incomplete. See [issue#78](https://github.com/openml/server-api/issues/78).

### Setting up a test database

We provide a prebuilt docker image that already contains test data.

=== "Docker Compose"
To start the database through `docker compose`, run:

```bash
docker compose up database
```

which starts a database.

=== "Docker Run"

To start a test database as stand-alone container, run:

```bash
docker run --rm -e MYSQL_ROOT_PASSWORD=ok -p 3306:3306 -d --name openml-test-database openml/test-database:latest
```

You may opt to add the container to a network instead, to make it reachable
from other docker containers:

```bash
docker network create openml
docker run --rm -e MYSQL_ROOT_PASSWORD=ok -p 3306:3306 -d --name openml-test-database --network openml openml/test-database:latest
```

The container may take a minute to initialise, but afterwards you can connect to it.
Either from a local `mysql` client at `127.0.0.1:3306` or from a docker container
on the same network. For example:

```bash
docker run --network NETWORK --rm -it mysql mysql -hopenml-test-database -uroot -pok
```
where `NETWORK` is `openml` when using `docker run` when following the example,
and `NETWORK` is `server-api_default` if you used `docker compose` (specifically,
it is `DIRECTORY_NAME` + `_default`, so if you renamed the `server-api` directory to
something else, the network name reflects that).

## Configuring the REST API Server

The REST API is configured through a [TOML](https://toml.io) file.

!!! Failure ""

Instructions are incomplete. Please have patience while we are adding more documentation.
Information for a production deployment will follow, in a nutshell you need to configure the REST API to connect to a persistent database,
which can be the one defined in `docker-compose.yaml` if has an appropriately mounted volume.
3 changes: 3 additions & 0 deletions docs/migration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ There may be undocumented changes, especially in edge cases which may not have o
As the PHP API was underspecified, the re-implementation is based on a mix of reading old code and probing the API.
If there is a behavioral change which was not documented but affects you, please [open a bug report](https://github.com/openml/server-api/issues/new?assignees=&labels=bug%2C+triage&projects=&template=bug-report.md&title=).

It is possible this migration guide is out of sync for endpoints not yet deployed to production (currently that includes them all).
Before an endpoint is deployed to production we will ensure that the documentation is up-to-date to the best of our knowledge.

## All Endpoints
The following changes affect all endpoints.

Expand Down