Skip to content

docs: improve HealthyCode deployment docs - #27

Open
keshav-space wants to merge 3 commits into
mainfrom
improve-deployment-docs
Open

keshav-space wants to merge 3 commits into
mainfrom
improve-deployment-docs

Conversation

@keshav-space

Copy link
Copy Markdown
Member

No description provided.

Signed-off-by: Keshav Priyadarshi <git@keshav.space>
Signed-off-by: Keshav Priyadarshi <git@keshav.space>
Signed-off-by: Keshav Priyadarshi <git@keshav.space>

@ziadhany ziadhany left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes:

  • Section: Set the max_map_count
    /etc/sysctl.conf depends on the distro. We should mention that the user should create a config file if it does not exist.
~$ ls /etc/sysctl.conf
ls: cannot access '/etc/sysctl.conf': No such file or directory
  • Section: Add PurlDB base URL and API key to ScanCode.io
    What is the <username>? Do you mean the system username?

  • What are the default ports for the services, and what is the default configuration for deploying them locally? I think we should have a default deployment configuration for both PurlDB and ScanCode.io to make sure users can deploy faster and then tweak the ports and services as needed.

SCIO docker.env example: (users only need to update SECRET_KEY and PURLDB_API_KEY):

POSTGRES_DB=scancodeio
POSTGRES_USER=scancodeio
POSTGRES_PASSWORD=scancodeio
POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8

SCANCODEIO_DB_HOST=db
SCANCODEIO_RQ_REDIS_HOST=redis
SCANCODEIO_ASYNC=True
SCANCODEIO_WORKSPACE_LOCATION=/var/scancodeio/workspace/
GRIMOIRELAB_METRICS_EXECUTABLE=/opt/healthycode/.venv/bin/grimoirelab-metrics

SECRET_KEY="AJmckthcifSC8eegdsfsdgdsfhdfshdjhhdkhghgfjfjF"
PURLDB_URL=http://localhost
PURLDB_API_KEY="...."

GRIMOIRELAB_URL="http://grimoirelab-nginx:8000/"
GRIMOIRELAB_OPENSEARCH_URL="https://grimoirelab-opensearch:9200/"
GRIMOIRELAB_ECOSYSTEM="npmlargerecosystem"
GRIMOIRELAB_PROJECT="npmmostused"
GRIMOIRELAB_OPENSEARCH_INDEX="events"

# Populate this using value from GRIMOIRELAB_SUPERUSER_USERNAME
# and GRIMOIRELAB_SUPERUSER_PASSWORD set earlier in
# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
GRIMOIRELAB_USERNAME="admin"
GRIMOIRELAB_PASSWORD="admin"

# Populate this using value from GRIMOIRELAB_ARCHIVIST_STORAGE_USERNAME
# and GRIMOIRELAB_ARCHIVIST_STORAGE_PASSWORD set earlier in
# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
GRIMOIRELAB_OPENSEARCH_USERNAME="admin"
GRIMOIRELAB_OPENSEARCH_PASSWORD="GrimoireLab.1"

ALLOWED_HOSTS=localhost
CSRF_TRUSTED_ORIGINS=http://localhost:8443

NGINX_PUBLISHED_HTTP_PORT=8081
NGINX_PUBLISHED_HTTPS_PORT=8443

PURLDB docker.env example: (users only need to update SECRET_KEY):

POSTGRES_DB=packagedb
POSTGRES_USER=packagedb
POSTGRES_PASSWORD=packagedb
POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8

PACKAGEDB_DB_HOST=db
PURLDB_REDIS_HOST=redis
PURLDB_ASYNC=True

SECRET_KEY="AJmckthcifSC8eegdsfdhsdfhejtrsdgdsfhdfshdjhhdkhghgfjfjF"
ALLOWED_HOSTS=localhost
SITE_URL=localhost
PURLDB_PUBLIC_SETUP=True
REST_API_ENDPOINTS=scan_queue,health,packages

Comment thread deploy/README.md
Comment on lines +181 to +182
sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
create-scan-queue-worker-user scancode-worker --generate-api-key <username>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the correct command, right ?

Suggested change
sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
create-scan-queue-worker-user scancode-worker --generate-api-key <username>
sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
create-scan-queue-worker-user <username> --generate-api-key

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
create-scan-queue-worker-user scancode-worker --generate-api-key <username>
sudo docker compose -f /opt/purldb/docker-compose.yml exec web purldb \
create-scan-queue-worker-user --generate-api-key <username>

@keshav-space

Copy link
Copy Markdown
Member Author

@ziadhany

SCIO docker.env example: (users only need to update SECRET_KEY and PURLDB_API_KEY):

POSTGRES_DB=scancodeio
POSTGRES_USER=scancodeio
POSTGRES_PASSWORD=scancodeio
POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8

SCANCODEIO_DB_HOST=db
SCANCODEIO_RQ_REDIS_HOST=redis
SCANCODEIO_ASYNC=True
SCANCODEIO_WORKSPACE_LOCATION=/var/scancodeio/workspace/
GRIMOIRELAB_METRICS_EXECUTABLE=/opt/healthycode/.venv/bin/grimoirelab-metrics

SECRET_KEY="AJmckthcifSC8eegdsfsdgdsfhdfshdjhhdkhghgfjfjF"
PURLDB_URL=http://localhost
PURLDB_API_KEY="...."

GRIMOIRELAB_URL="http://grimoirelab-nginx:8000/"
GRIMOIRELAB_OPENSEARCH_URL="https://grimoirelab-opensearch:9200/"
GRIMOIRELAB_ECOSYSTEM="npmlargerecosystem"
GRIMOIRELAB_PROJECT="npmmostused"
GRIMOIRELAB_OPENSEARCH_INDEX="events"

# Populate this using value from GRIMOIRELAB_SUPERUSER_USERNAME
# and GRIMOIRELAB_SUPERUSER_PASSWORD set earlier in
# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
GRIMOIRELAB_USERNAME="admin"
GRIMOIRELAB_PASSWORD="admin"

# Populate this using value from GRIMOIRELAB_ARCHIVIST_STORAGE_USERNAME
# and GRIMOIRELAB_ARCHIVIST_STORAGE_PASSWORD set earlier in
# /opt/healthycode/deploy/docker-compose/docker-compose-grimoirelab2.yml
GRIMOIRELAB_OPENSEARCH_USERNAME="admin"
GRIMOIRELAB_OPENSEARCH_PASSWORD="GrimoireLab.1"

ALLOWED_HOSTS=localhost
CSRF_TRUSTED_ORIGINS=http://localhost:8443

NGINX_PUBLISHED_HTTP_PORT=8081
NGINX_PUBLISHED_HTTPS_PORT=8443

This HealthyCode deployment docs do not ask/require edit to docker.env it should be left as is, all the changes are to be
made in .env file. SECRET_KEY is auto generated using make envfile command as insturcted here

make envfile


PURLDB docker.env example: (users only need to update SECRET_KEY):

POSTGRES_DB=packagedb
POSTGRES_USER=packagedb
POSTGRES_PASSWORD=packagedb
POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8

PACKAGEDB_DB_HOST=db
PURLDB_REDIS_HOST=redis
PURLDB_ASYNC=True

SECRET_KEY="AJmckthcifSC8eegdsfdhsdfhejtrsdgdsfhdfshdjhhdkhghgfjfjF"
ALLOWED_HOSTS=localhost
SITE_URL=localhost
PURLDB_PUBLIC_SETUP=True
REST_API_ENDPOINTS=scan_queue,health,packages

Once again docker.env for purldb should not be edited please add changes to .env as instructed in deployment docs.
Please take look at steps below: (make envfile -> config required env variables in .env file)

## Installation of PurlDB and ScanCode.io
The first step is to install PurlDB and ScanCode.io using their standard
documentation.
### Install and setup PurlDB
```
git clone https://github.com/aboutcode-org/purldb.git/ /opt/purldb/ && cd /opt/purldb/
make envfile
```
Open `/opt/purldb/.env` in a text editor and populate with this config:
```
ALLOWED_HOSTS=<your host>
SITE_URL=https://<your host>/
PURLDB_PUBLIC_SETUP=True
REST_API_ENDPOINTS=scan_queue,health,packages
```

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants