diff --git a/.python-version b/.python-version deleted file mode 100644 index e4fba21..0000000 --- a/.python-version +++ /dev/null @@ -1 +0,0 @@ -3.12 diff --git a/docs/installation.md b/docs/installation.md index b239601..c506537 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,144 +1,22 @@ # Installation -*Current instructions tested on Mac, but likely work on most Unix systems.* +The primary way to run this service is through a Docker container. +The REST API needs to be able to connect to a MySQL database with the OpenML "openml" and "openml_expdb" databases. +The `docker-compose.yaml` file of this project defines these together out of the box. +This is useful for development purposes, but the database does not persist between restarts in the current configuration. +By default, the current code is also mounted into the Python REST API container (again, for development purposes). -The OpenML server will be developed and maintained for the latest minor release of -Python (Python 3.12 as of writing). -You can install the dependencies locally or work with docker containers. +For development, it should suffice to run the services from a fresh clone by running `docker compose --profile "python" up -d`. +The REST API will be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit http://localhost:8001/docs. -??? tip "Use `pyenv` to manage Python installations" +Once the containers are started, you can run tests with `docker exec -it openml-python-rest-api python -m pytest -m "not php_api" tests`. +For migration testing, which compares output of the Python-based REST API with the old PHP-based one, also start the PHP server (`docker compose --profile "php" --profile "python" up -d`) and include tests with the `php_api` marker/fixture: `docker exec -it openml-python-rest-api python -m pytest tests`. - We recommend using [`pyenv`](https://github.com/pyenv/pyenv) if you are working with - multiple local Python versions. After following the installation instructions for - `pyenv` check that you can execute it: +!!! note - ```text - > pyenv local - 3.12 - ``` + The PHP REST API needs Elasticsearch. In some cases, it also needs the ES indices to be built. + The current set up does not automatically build ES indices, because that takes a long time. + When we start testing more upload functionality, for which the PHP API needs built indices, we'll work on an ES image with prebuilt indices. - If `pyenv` can't be found, please make sure to update the terminal environment - (either by `reset`ing it, or by closing and opening the terminal). If you get the message - `pyenv: no local version configured for this directory` first clone the repository - as described below and try again from the root of the cloned repository. - - You can then install the Python version this project uses with: - `cat .python-version | pyenv install` - - -## Local Installation - -These instructions assume [Python 3.12](https://www.python.org/downloads/) -and [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) are already installed. - -!!! info "You may need to install Python3 and MySQL development headers." - - It may be necessary to first install additional headers before proceeding with a - local installation of the `mysqlclient` dependency. They are documented under - ["Installation"](https://github.com/PyMySQL/mysqlclient#linux) of the `mysqlclient` - documentation. - -=== "For Users" - - If you don't plan to make code changes, you can install directly from Github. - We recommend to install the OpenML server and its dependencies into a new virtual - environment. - ```bash title="Installing the project into a new virtual environment" - python -m venv venv - source venv/bin/activate - - python -m pip install git+https://github.com/openml/server-api.git - ``` - If you do plan to make code changes, we recommend you follow the instructions - under the "For Contributors" tab, even if you do not plan to contribute your - changes back into the project. - - -=== "For Contributors" - - If you plan to make changes to this project, it will be useful to install - the project from a cloned fork. To fork the project, go to our - [project page](https://github.com/openml/server-api) and click "fork". - This makes a copy of the repository under your own Github account. - You can then clone your own fork (replace `USER_NAME` with your Github username): - - ```bash title="Cloning your fork" - git clone https://github.com/USER_NAME/server-api.git - cd server-api - ``` - - Then we can install the project into a new virtual environment in edit mode: - - ```bash title="Installing the project into a new virtual environment" - python -m venv venv - source venv/bin/activate - - python -m pip install -e ".[dev,docs]" - ``` - Note that this also installs optional dependencies for development and documentation - tools. We require this for contributors, but we also highly recommend it anyone - that plans to make code changes. - -## Setting up a Database Server -Depending on your use of the server, there are multiple ways to set up your own -OpenML database. To simply connect to an existing database, see -[configuring the REST API Server](#configuring-the-rest-api-server) below. - - -### Setting up a new database -This sets up an entirely empty database with the expected OpenML tables in place. -This is intended for new deployments of OpenML, for example to host a private OpenML -server. - -!!! Failure "" - - Instructions are incomplete. See [issue#78](https://github.com/openml/server-api/issues/78). - -### Setting up a test database - -We provide a prebuilt docker image that already contains test data. - -=== "Docker Compose" - To start the database through `docker compose`, run: - - ```bash - docker compose up database - ``` - - which starts a database. - -=== "Docker Run" - - To start a test database as stand-alone container, run: - - ```bash - docker run --rm -e MYSQL_ROOT_PASSWORD=ok -p 3306:3306 -d --name openml-test-database openml/test-database:latest - ``` - - You may opt to add the container to a network instead, to make it reachable - from other docker containers: - - ```bash - docker network create openml - docker run --rm -e MYSQL_ROOT_PASSWORD=ok -p 3306:3306 -d --name openml-test-database --network openml openml/test-database:latest - ``` - -The container may take a minute to initialise, but afterwards you can connect to it. -Either from a local `mysql` client at `127.0.0.1:3306` or from a docker container -on the same network. For example: - -```bash -docker run --network NETWORK --rm -it mysql mysql -hopenml-test-database -uroot -pok -``` -where `NETWORK` is `openml` when using `docker run` when following the example, -and `NETWORK` is `server-api_default` if you used `docker compose` (specifically, -it is `DIRECTORY_NAME` + `_default`, so if you renamed the `server-api` directory to -something else, the network name reflects that). - -## Configuring the REST API Server - -The REST API is configured through a [TOML](https://toml.io) file. - -!!! Failure "" - - Instructions are incomplete. Please have patience while we are adding more documentation. +Information for a production deployment will follow, in a nutshell you need to configure the REST API to connect to a persistent database, +which can be the one defined in `docker-compose.yaml` if has an appropriately mounted volume. diff --git a/docs/migration.md b/docs/migration.md index 78f8761..8a69b9f 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -13,6 +13,9 @@ There may be undocumented changes, especially in edge cases which may not have o As the PHP API was underspecified, the re-implementation is based on a mix of reading old code and probing the API. If there is a behavioral change which was not documented but affects you, please [open a bug report](https://github.com/openml/server-api/issues/new?assignees=&labels=bug%2C+triage&projects=&template=bug-report.md&title=). +It is possible this migration guide is out of sync for endpoints not yet deployed to production (currently that includes them all). +Before an endpoint is deployed to production we will ensure that the documentation is up-to-date to the best of our knowledge. + ## All Endpoints The following changes affect all endpoints.