Dev containers: build any port in a codespace

.devcontainer/Dockerfile

@@ -0,0 +1,15 @@
+# The image every dev container here builds from: Ubuntu, and the Debian
+# packages for one port, which install.sh --deps installs from the same list it
+# prints for a machine of your own. Nothing else is baked in: build_mp.py
+# fetches the port's toolchain (ESP-IDF, emsdk, the pico-sdk, SDL2 for MinGW)
+# on its first build, into deps/, which each config keeps in a volume.
+FROM mcr.microsoft.com/devcontainers/base:ubuntu-24.04
+
+ARG PORT
+ARG INSTALL_SH=https://pydevices.github.io/install.sh
+
+# Downloaded first, not piped: a failed download must fail the image.
+RUN test -n "$PORT" \
+    && curl -fsSL --retry 3 -o /tmp/install.sh "$INSTALL_SH" \
+    && sh /tmp/install.sh --deps "$PORT" \
+    && rm -rf /tmp/install.sh /var/lib/apt/lists/*

.devcontainer/esp32/devcontainer.json

@@ -0,0 +1,22 @@
+{
+  "name": "esp32: ESP32-S3, ESP32-P4 and the other ESP chips",
+  "build": {
+    "dockerfile": "../Dockerfile",
+    "context": "..",
+    "args": {
+      "PORT": "esp32"
+    }
+  },
+  "mounts": [
+    "source=pydevices-deps-${devcontainerId},target=${containerWorkspaceFolder}/deps,type=volume"
+  ],
+  "containerEnv": {
+    "IDF_TOOLS_PATH": "${containerWorkspaceFolder}/deps/espressif-tools"
+  },
+  "hostRequirements": {
+    "cpus": 4,
+    "storage": "32gb"
+  },
+  "onCreateCommand": "sh .devcontainer/on-create.sh && sh .devcontainer/esp32/prefetch.sh",
+  "postCreateCommand": "echo 'To build: ./build_mp.py --port esp32 --board ESP32_GENERIC_S3 --variant SPIRAM_OCT --flash 8MB --modules all'"
+}

.devcontainer/esp32/prefetch.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+# The esp32 container fetches, when it's created, what build_mp.py would fetch
+# on its first esp32 build: MicroPython, the modules, and ESP-IDF with its
+# compilers (about 4 GB, into the deps/ volume). In a Codespaces prebuild that
+# happens once, ahead of time, so your first build starts compiling.
+set -eu
+cd "$(dirname "$0")/../.."
+python3 -c 'import build_mp as b; b.ensure_micropython(None); b.ensure_modules(None); b.ensure_dep("esp-idf", None)'

.devcontainer/on-create.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+# Every config's first step when its container is created. deps/ is a Docker
+# volume, so a rebuilt container keeps the toolchains it fetched; Docker makes
+# a new volume root's, and build_mp.py writes to it as you.
+set -eu
+cd "$(dirname "$0")/.."
+sudo chown "$(id -u):$(id -g)" deps

.devcontainer/rp2/devcontainer.json

@@ -0,0 +1,15 @@
+{
+  "name": "rp2: Raspberry Pi Pico and Pico 2",
+  "build": {
+    "dockerfile": "../Dockerfile",
+    "context": "..",
+    "args": {
+      "PORT": "rp2"
+    }
+  },
+  "mounts": [
+    "source=pydevices-deps-${devcontainerId},target=${containerWorkspaceFolder}/deps,type=volume"
+  ],
+  "onCreateCommand": "sh .devcontainer/on-create.sh",
+  "postCreateCommand": "echo 'To build: ./build_mp.py --port rp2 --board RPI_PICO2 --modules all'"
+}

.devcontainer/unix/devcontainer.json

@@ -0,0 +1,15 @@
+{
+  "name": "unix: MicroPython for Linux, with a desktop display",
+  "build": {
+    "dockerfile": "../Dockerfile",
+    "context": "..",
+    "args": {
+      "PORT": "unix"
+    }
+  },
+  "mounts": [
+    "source=pydevices-deps-${devcontainerId},target=${containerWorkspaceFolder}/deps,type=volume"
+  ],
+  "onCreateCommand": "sh .devcontainer/on-create.sh",
+  "postCreateCommand": "echo 'To build: ./build_mp.py --port unix --variant pydevices --modules all'"
+}

.devcontainer/webassembly/devcontainer.json

@@ -0,0 +1,24 @@
+{
+  "name": "webassembly: MicroPython in the browser, opened in workbench",
+  "build": {
+    "dockerfile": "../Dockerfile",
+    "context": "..",
+    "args": {
+      "PORT": "webassembly"
+    }
+  },
+  "mounts": [
+    "source=pydevices-deps-${devcontainerId},target=${containerWorkspaceFolder}/deps,type=volume"
+  ],
+  "onCreateCommand": "sh .devcontainer/on-create.sh",
+  "forwardPorts": [
+    8000
+  ],
+  "portsAttributes": {
+    "8000": {
+      "label": "workbench",
+      "onAutoForward": "openBrowser"
+    }
+  },
+  "postCreateCommand": "echo 'To build: ./build_mp.py --port webassembly --variant pydevices --modules all, then scripts/serve_workbench.sh'"
+}

.devcontainer/windows/devcontainer.json

@@ -0,0 +1,15 @@
+{
+  "name": "windows: micropython.exe, cross-built from Linux",
+  "build": {
+    "dockerfile": "../Dockerfile",
+    "context": "..",
+    "args": {
+      "PORT": "windows"
+    }
+  },
+  "mounts": [
+    "source=pydevices-deps-${devcontainerId},target=${containerWorkspaceFolder}/deps,type=volume"
+  ],
+  "onCreateCommand": "sh .devcontainer/on-create.sh",
+  "postCreateCommand": "echo 'To build: ./build_mp.py --port windows --variant pydevices --modules all'"
+}

.github/workflows/devcontainers.yml

@@ -0,0 +1,94 @@
+# Each dev container in .devcontainer/ builds its port from nothing, as a
+# codespace does: the image is made from the Dockerfile (install.sh --deps for
+# that port, on a clean Ubuntu), the container is created with its deps/ volume
+# empty, and build_mp.py fetches MicroPython, the modules and the port's
+# toolchain before it builds. One job per config, one build each.
+#
+# It runs when a config changes, and weekly, which also catches a change to
+# install.sh's package list or to a pin that stops a port building from a
+# clean machine.
+name: Dev containers
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - '.devcontainer/**'
+      - 'scripts/serve_workbench.sh'
+      - '.github/workflows/devcontainers.yml'
+  pull_request:
+    paths:
+      - '.devcontainer/**'
+      - 'scripts/serve_workbench.sh'
+      - '.github/workflows/devcontainers.yml'
+  schedule:
+    - cron: '41 4 * * 3'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: devcontainers-${{ github.event_name == 'pull_request' && github.ref || github.run_id }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  build:
+    name: ${{ matrix.port }}
+    runs-on: ubuntu-latest
+    timeout-minutes: ${{ matrix.port == 'esp32' && 150 || 75 }}
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - port: unix
+            run: |
+              ./build_mp.py --port unix --variant pydevices --modules all
+              builds/unix/pydevices/micropython -c "import sys; print(sys.implementation); import displaydev; print('displaydev imports')"
+          - port: webassembly
+            run: |
+              ./build_mp.py --port webassembly --variant pydevices --modules all
+              b=builds/webassembly/pydevices
+              printf 'import sys\nprint(sys.implementation)\nimport displaydev\nprint("displaydev imports")\n' > /tmp/check.py
+              node $b/micropython.mjs /tmp/check.py
+              # workbench, with this build in it, on the port the config forwards
+              WORKBENCH_PORT=8000 scripts/serve_workbench.sh < /dev/null > /tmp/serve.log 2>&1 &
+              srv=$!
+              for i in $(seq 60); do curl -fs -o /dev/null http://127.0.0.1:8000/ && break; sleep 2; done
+              cat /tmp/serve.log
+              curl -fsS http://127.0.0.1:8000/index.html | grep -qi '<html'
+              curl -fsS -o /tmp/served.wasm http://127.0.0.1:8000/assets/pydevices/micropython.wasm
+              cmp /tmp/served.wasm $b/micropython.wasm
+              echo "workbench serves this build's micropython.wasm"
+              kill $srv
+          - port: rp2
+            run: |
+              ./build_mp.py --port rp2 --board RPI_PICO2 --modules all
+              ls -l builds/rp2/RPI_PICO2/default/firmware.uf2
+          - port: windows
+            run: |
+              ./build_mp.py --port windows --variant pydevices --modules all
+              ls -l builds/windows/pydevices/micropython.exe
+          - port: esp32
+            run: |
+              ./build_mp.py --port esp32 --board ESP32_GENERIC_S3 --variant SPIRAM_OCT --flash 8MB --modules all
+              ls -l builds/esp32/ESP32_GENERIC_S3/SPIRAM_OCT/firmware.bin
+    steps:
+      - uses: actions/checkout@v7
+
+      # ESP-IDF's compilers for every chip, its clone and an S3 build with
+      # every module come to about 7 GB, beside the image.
+      - name: Room for ESP-IDF
+        if: matrix.port == 'esp32'
+        run: |
+          sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
+          df -h / | tail -1
+
+      - name: Build the ${{ matrix.port }} container and build its port in it
+        uses: devcontainers/ci@v0.3
+        with:
+          configFile: .devcontainer/${{ matrix.port }}/devcontainer.json
+          push: never
+          runCmd: |
+            set -e
+            ${{ matrix.run }}

README.md

@@ -1,5 +1,7 @@
 # micropython-pydevices
 
+[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/PyDevices/micropython-pydevices)
+
 Build MicroPython firmware with PyDevices modules in it, for any port, with
 one command:
 
@@ -29,6 +31,10 @@ It names the Debian packages each port still needs, with the `apt-get` line
 that installs them; [What each port needs](docs/newcomers.md#what-each-port-needs)
 is the same list.
 
+Or build in a codespace, with nothing to install: the badge above opens one,
+and [In a codespace](docs/newcomers.md#in-a-codespace) says how to pick your
+port and get the file out.
+
 On Windows you can build the windows port natively from PowerShell, with
 `python build_mp.py --port windows --variant pydevices --modules all`, once
 MSYS2 is installed: [On Windows](docs/newcomers.md#on-windows) has the

ROADMAP.md

@@ -4,6 +4,10 @@ Work here mostly follows bug reports and requests, so if something is broken
 or missing for you, open an
 [issue](https://github.com/PyDevices/micropython-pydevices/issues).
 
+- **Your own firmware from the firmware page.** A build you made, in a
+  codespace or anywhere else, goes onto your board from the browser with the
+  firmware page's "Install from a file", as the released images do.
+
 More modules in CircuitPython-compatible builds
 ([what builds today](docs/build-plan.md#which-modules-build-on-circuitpython)):
 

docs/newcomers.md

@@ -34,6 +34,42 @@ The desktop `pydevices` variants freeze a board config that imports
 repository whose `lib/` holds it) or `all`; without it `import board_config`
 fails.
 
+## In a codespace
+
+You can build without installing anything, in a GitHub codespace or in a dev
+container on your own machine. Each port has its own container, holding only
+that port's packages: unix, webassembly, rp2, windows and esp32.
+
+On GitHub, open the repository's **Code** menu, then **Codespaces**, **...**,
+**New with options**, and choose your port under "Dev container
+configuration". With VS Code and Docker on your own machine, open your clone
+and run **Dev Containers: Reopen in Container**, which asks for the port.
+
+When the container is ready, its terminal prints the build line for its port,
+and you build as you would anywhere else:
+
+```bash
+./build_mp.py --port rp2 --board RPI_PICO2 --modules all
+```
+
+The first build fetches the port's toolchain into `deps/`, a volume that
+outlives the container, so rebuilding the container doesn't fetch it again.
+The esp32 container fetches ESP-IDF and its compilers (about 4 GB) while it's
+being created, so your first build starts compiling; it asks for a 4-core
+machine.
+
+To flash, take the file to the machine your board is plugged into. In the
+Explorer, open `builds/` down to your build, right-click `firmware.bin` (or
+`firmware.uf2` on rp2) and choose **Download**. Then write it with
+`mpftp firmware flash --artifact firmware.bin` (`--uf2` for a `.uf2`), or with
+esptool as in [Flash it](#flash-it).
+
+The webassembly container opens your build in workbench, the browser IDE.
+Build, then run `scripts/serve_workbench.sh`: it downloads the newest
+workbench release, puts your `micropython.wasm` in it and serves it on port
+8000, which the container forwards and your browser opens. Click the
+simulator button (the display icon) and the REPL that starts is your build.
+
 ## What each port needs
 
 [`build_mp.py`](../build_mp.py) fetches the big pieces itself: MicroPython, the modules,
@@ -44,7 +80,7 @@ fetch are your distribution's packages. On Debian or Ubuntu (WSL included):
 |---|---|
 | every port | `git make cmake python3 gcc g++` |
 | unix | `pkg-config libsdl2-dev libffi-dev` (the desktop display) |
-| esp32 | `ninja-build python3-venv` |
+| esp32 | `ninja-build python3-venv libusb-1.0-0` (ESP-IDF's installer checks the openocd it fetches, which needs libusb) |
 | rp2 | `gcc-arm-none-eabi libnewlib-arm-none-eabi` |
 | windows, cross-built from Linux | `gcc-mingw-w64-x86-64 binutils-mingw-w64-x86-64 libtool libltdl-dev autoconf automake` (the binutils package has `windres`) |
 | webassembly | `nodejs` |
@@ -53,15 +89,17 @@ All of it in one line:
 
 ```bash
 sudo apt-get update && sudo apt-get install -y git make cmake ninja-build \
-    python3 python3-venv python3-pip gcc g++ pkg-config libffi-dev libsdl2-dev \
-    gcc-mingw-w64-x86-64 binutils-mingw-w64-x86-64 libtool libltdl-dev \
-    autoconf automake \
+    python3 python3-venv python3-pip libusb-1.0-0 gcc g++ pkg-config libffi-dev \
+    libsdl2-dev gcc-mingw-w64-x86-64 binutils-mingw-w64-x86-64 libtool \
+    libltdl-dev autoconf automake \
     gcc-arm-none-eabi libnewlib-arm-none-eabi nodejs
 ```
 
 The installer prints only what your machine is missing, one `apt-get install`
 line per port and marked with it, so you can leave out the ports you won't
-build. It also says which ports build now. Other distributions carry the same
+build. It also says which ports build now, and it can install a port's
+packages for you: `curl -fsSL https://pydevices.github.io/install.sh | sh -s
+-- --deps rp2` (or `all`). Other distributions carry the same
 tools under their own names; a build for a port that's still missing
 something stops and says so.