Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 48 additions & 10 deletions .github/actions/provision-meos/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,22 +138,23 @@ runs:
bash "$GITHUB_ACTION_PATH/apt-get.sh" --purge remove postgresql* -y || true
sudo rm -rf /var/lib/postgresql/ /etc/postgresql/ /var/log/postgresql/ || true

# The two caches of the build path. The packages: libgdal-dev depends on the -dev
# packages of its drivers (mariadb, poppler, spatialite, xerces) and postgresql-server-dev-17
# on clang-19 and llvm-19, about 200 MB a runner downloads on every run, so the .deb files
# are kept in a runner-writable directory keyed on the runner image and the package list;
# a new image restores the previous one's files and downloads only what changed. The
# derivation: the libmeos prefix and the catalog are a function of the MobilityDB commit,
# the MEOS-API files that derive them, the families and the runner image, so a run on the
# same four restores them instead of building. A cross-repo action has no .git, so the
# MEOS-API side of the key is a hash of its files' contents.
# The three caches of the build path. The packages: postgresql-server-dev-17 depends on
# clang-19 and llvm-19, which a runner would download on every run, so the .deb files are
# kept in a runner-writable directory keyed on the runner image and the package list; a new
# image restores the previous one's files and downloads only what changed. GDAL: libmeos
# links the GDAL build-gdal.sh builds from its release source without GEOS, a function of the
# runner image and of that script, so a run on the same two restores it. The derivation: the
# libmeos prefix and the catalog are a function of the MobilityDB commit, the MEOS-API files
# that derive them, the families and the runner image, so a run on the same four restores them
# instead of building. A cross-repo action has no .git, so the MEOS-API side of each key is a
# hash of its files' contents.
- name: Key the provision caches
id: keys
if: inputs.build-libmeos == 'true'
shell: bash
env:
PACKAGES: >-
libproj-dev libjson-c-dev libh3-dev libgdal-dev libxml2-dev
libproj-dev libjson-c-dev libh3-dev libsqlite3-dev zlib1g-dev libxml2-dev
autoconf automake libtool pkg-config postgresql-17 postgresql-server-dev-17
FAMILIES: ${{ inputs.families }}
run: |
Expand All @@ -165,10 +166,12 @@ runs:
| sort -z | xargs -0 sha256sum | sha256sum | cut -c1-16)"
pkg_hash="$(echo "$PACKAGES" | tr -s ' \n' '\n\n' | sort | sha256sum | cut -c1-16)"
fam_hash="$(echo "$FAMILIES" | sha256sum | cut -c1-16)"
gdal_hash="$(sha256sum "$GITHUB_ACTION_PATH/build-gdal.sh" | cut -c1-16)"
{
echo "packages=$PACKAGES"
echo "deb-prefix=provision-meos-debs-${ImageOS:-unknown}-"
echo "deb-key=provision-meos-debs-$image-$pkg_hash"
echo "gdal-key=provision-meos-gdal-$image-$gdal_hash"
echo "build-key=provision-meos-build-$image-$mdb_sha-$api_hash-$fam_hash"
} >> "$GITHUB_OUTPUT"

Expand Down Expand Up @@ -205,6 +208,28 @@ runs:
sudo chmod -R a+rX "$APT_ARCHIVES"
sudo rm -f "$APT_ARCHIVES/lock"

- name: Restore the GDAL build
id: gdal-cache
if: inputs.build-libmeos == 'true'
uses: actions/cache@v4
with:
path: ${{ runner.temp }}/gdal-prefix
key: ${{ steps.keys.outputs.gdal-key }}

- name: Build GDAL without GEOS
if: inputs.build-libmeos == 'true' && steps.gdal-cache.outputs.cache-hit != 'true'
shell: bash
run: bash "$GITHUB_ACTION_PATH/build-gdal.sh" "$RUNNER_TEMP/gdal-prefix"

# Installed on a cache hit of the libmeos prefix too, as libmeos.so links it. MobilityDB's
# CMake finds it under /usr/local, the GDAL every consumer then builds and runs against.
- name: Install GDAL
if: inputs.build-libmeos == 'true'
shell: bash
run: |
sudo cp -a "$RUNNER_TEMP/gdal-prefix/." /usr/local/
sudo ldconfig

# Build path: provision-meos.sh builds libmeos, stages a clean MEOS-only
# header prefix ($RUNNER_TEMP/meos-headers) that the parse reads, and installs
# the runtime library under /usr/local for the consumer builds. It derives the
Expand Down Expand Up @@ -247,3 +272,16 @@ runs:
echo "libmeos-prefix=/usr/local"
} >> "$GITHUB_OUTPUT"
echo "Restored libmeos and the catalog ($(wc -c < "$REPO_ROOT/output/meos-idl.json") bytes)" >&2

# MEOS calls no GEOS function, so no GEOS library loads with libmeos: the step fails where
# one does, naming the library that brings it.
- name: Check that no GEOS loads with libmeos
if: inputs.build-libmeos == 'true'
shell: bash
run: |
deps="$(ldd /usr/local/lib/libmeos.so)"
echo "$deps" | grep -E 'libgdal|libproj|libmeos' >&2 || true
if echo "$deps" | grep -i 'libgeos'; then
echo "::error::libmeos loads GEOS: $(echo "$deps" | grep -i libgeos | awk '{print $1}' | tr '\n' ' ')"
exit 1
fi
54 changes: 54 additions & 0 deletions .github/actions/provision-meos/build-gdal.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
# build-gdal.sh — build the GDAL libmeos links, from its release source, without GEOS, into a
# prefix the provision-meos action caches and installs under /usr/local. MEOS calls no GEOS
# function, and the GDAL a distribution packages (libgdal-dev on Ubuntu) links GEOS, so a
# libmeos linked against it loads GEOS with it; this GDAL loads none.
#
# The configuration is the one MEOS.js builds its GDAL with: every optional driver off, the
# libraries GDAL carries internally used in place of external ones, and only PROJ, SQLite and
# zlib taken from the system, which libmeos already needs. MEOS reads raster bands through the
# generic GDAL API and the built-in drivers (MEM, VRT, GTiff), so the core is the whole
# requirement. The libraries are shared, as the hosted bindings link libmeos.so.
#
# The release is pinned by its SHA-256, so the build is the same on every runner and a changed
# file fails the step rather than building something else.
#
# Usage:
# build-gdal.sh <prefix>
#
# Environment:
# CMAKE_BUILD_PARALLEL_LEVEL build jobs (default: the number of processors)
set -euo pipefail

PREFIX="${1:?usage: build-gdal.sh <prefix>}"
VERSION=3.12.0
SHA256=44c95baabe6c1a047c1ebe5043e38dd73e4936bc4c481db14efbfd03342eab73
URL="https://github.com/OSGeo/gdal/releases/download/v$VERSION/gdal-$VERSION.tar.gz"

WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT

curl -fsSL --retry 3 --max-time 600 -o "$WORK/gdal.tar.gz" "$URL"
echo "$SHA256 $WORK/gdal.tar.gz" | sha256sum -c -
tar xzf "$WORK/gdal.tar.gz" -C "$WORK"

cmake -S "$WORK/gdal-$VERSION" -B "$WORK/build" \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_APPS=OFF \
-DBUILD_TESTING=OFF \
-DBUILD_DOCS=OFF \
-DBUILD_PYTHON_BINDINGS=OFF \
-DBUILD_JAVA_BINDINGS=OFF \
-DBUILD_CSHARP_BINDINGS=OFF \
-DGDAL_BUILD_OPTIONAL_DRIVERS=OFF \
-DOGR_BUILD_OPTIONAL_DRIVERS=OFF \
-DGDAL_USE_EXTERNAL_LIBS=OFF \
-DGDAL_USE_INTERNAL_LIBS=ON \
-DGDAL_USE_PROJ=ON \
-DGDAL_USE_GEOS=OFF \
-DGDAL_USE_SQLITE3=ON \
-DGDAL_USE_ZLIB=ON
cmake --build "$WORK/build" -j "${CMAKE_BUILD_PARALLEL_LEVEL:-$(nproc)}"
cmake --install "$WORK/build"
Loading