Merge branch 'master' into no-envars

0x2b3bfa0 · web-flow · commit 09d7a236f312 · 2022-11-01T19:56:37.000+01:00
diff --git a/README.md b/README.md
@@ -81,14 +81,14 @@ jobs:
   run:
     runs-on: ubuntu-latest
     # optionally use a convenient Ubuntu LTS + DVC + CML image
-    # container: docker://ghcr.io/iterative/cml:0-dvc2-base1
+    # container: ghcr.io/iterative/cml:0-dvc2-base1
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       # may need to setup NodeJS & Python3 on e.g. self-hosted
-      # - uses: actions/setup-node@v2
+      # - uses: actions/setup-node@v3
       #   with:
       #     node-version: '16'
-      # - uses: actions/setup-python@v2
+      # - uses: actions/setup-python@v4
       #   with:
       #     python-version: '3.x'
       - uses: iterative/setup-cml@v1
@@ -103,17 +103,17 @@ jobs:
         run: |
           # Post reports as comments in GitHub PRs
           cat results.txt >> report.md
-          cml send-comment report.md
+          cml comment create report.md
 ```
 
 ## Usage
 
 We helpfully provide CML and other useful libraries pre-installed on our
 [custom Docker images](https://github.com/iterative/cml/blob/master/Dockerfile).
 In the above example, uncommenting the field
-`container: docker://ghcr.io/iterative/cml:0-dvc2-base1`) will make the runner
-pull the CML Docker image. The image already has NodeJS, Python 3, DVC and CML
-set up on an Ubuntu LTS base for convenience.
+`container: ghcr.io/iterative/cml:0-dvc2-base1`) will make the runner pull the
+CML Docker image. The image already has NodeJS, Python 3, DVC and CML set up on
+an Ubuntu LTS base for convenience.
 
 ### CML Functions
 
@@ -124,18 +124,17 @@ report.
 Below is a table of CML functions for writing markdown reports and delivering
 those reports to your CI system.
 
-| Function                | Description                                                      | Example Inputs                                              |
-| ----------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------- |
-| `cml runner`            | Launch a runner locally or hosted by a cloud provider            | See [Arguments](https://github.com/iterative/cml#arguments) |
-| `cml publish`           | Publicly host an image for displaying in a CML report            | `<path to image> --title <image title> --md`                |
-| `cml send-comment`      | Return CML report as a comment in your GitLab/GitHub workflow    | `<path to report> --head-sha <sha>`                         |
-| `cml send-github-check` | Return CML report as a check in GitHub                           | `<path to report> --head-sha <sha>`                         |
-| `cml pr`                | Commit the given files to a new branch and create a pull request | `<path>...`                                                 |
-| `cml tensorboard-dev`   | Return a link to a Tensorboard.dev page                          | `--logdir <path to logs> --title <experiment title> --md`   |
+| Function                  | Description                                                      | Example Inputs                                              |
+| ------------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------- |
+| `cml runner launch`       | Launch a runner locally or hosted by a cloud provider            | See [Arguments](https://github.com/iterative/cml#arguments) |
+| `cml comment create`      | Return CML report as a comment in your GitLab/GitHub workflow    | `<path to report> --head-sha <sha>`                         |
+| `cml check create`        | Return CML report as a check in GitHub                           | `<path to report> --head-sha <sha>`                         |
+| `cml pr create`           | Commit the given files to a new branch and create a pull request | `<path>...`                                                 |
+| `cml tensorboard connect` | Return a link to a Tensorboard.dev page                          | `--logdir <path to logs> --title <experiment title> --md`   |
 
 #### CML Reports
 
-The `cml send-comment` command can be used to post reports. CML reports are
+The `cml comment create` command can be used to post reports. CML reports are
 written in markdown ([GitHub](https://github.github.com/gfm),
 [GitLab](https://docs.gitlab.com/ee/user/markdown.html), or
 [Bitbucket](https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html)
@@ -153,11 +152,12 @@ cat results.txt >> report.md
 
 :framed_picture: **Images** Display images using the markdown or HTML. Note that
 if an image is an output of your ML workflow (i.e., it is produced by your
-workflow), you will need to use the `cml publish` function to include it a CML
-report. For example, if `graph.png` is output by `python train.py`, run:
+workflow), it can be uploaded and included automaticlly to your CML report. For
+example, if `graph.png` is output by `python train.py`, run:
 
 ```bash
-cml publish graph.png --md >> report.md
+echo "![](./graph.png)" >> report.md
+cml comment create report.md
 ```
 
 ### Getting Started
@@ -189,8 +189,8 @@ jobs:
   run:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v2
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
       - uses: iterative/setup-cml@v1
       - name: Train model
         env:
@@ -200,8 +200,8 @@ jobs:
           python train.py
 
           cat metrics.txt >> report.md
-          cml publish plot.png --md >> report.md
-          cml send-comment report.md
+          echo "![](./plot.png)" >> report.md
+          cml comment create report.md
 ```
 
 3. In your text editor of choice, edit line 16 of `train.py` to `depth = 5`.
@@ -253,9 +253,9 @@ on: [push]
 jobs:
   run:
     runs-on: ubuntu-latest
-    container: docker://ghcr.io/iterative/cml:0-dvc2-base1
+    container: ghcr.io/iterative/cml:0-dvc2-base1
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Train model
         env:
           REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -278,16 +278,16 @@ jobs:
           echo "## Plots" >> report.md
           echo "### Class confusions" >> report.md
           dvc plots diff --target classes.csv --template confusion -x actual -y predicted --show-vega master > vega.json
-          vl2png vega.json -s 1.5 > plot.png
-          cml publish --md plot.png >> report.md
+          vl2png vega.json -s 1.5 > confusion_plot.png
+          echo "![](./confusion_plot.png)" >> report.md
 
           # Publish regularization function diff
           echo "### Effects of regularization" >> report.md
           dvc plots diff --target estimators.csv -x Regularization --show-vega master > vega.json
           vl2png vega.json -s 1.5 > plot.png
-          cml publish --md plot.png >> report.md
+          echo "![](./plot.png)" >> report.md
 
-          cml send-comment report.md
+          cml comment create report.md
 ```
 
 > :warning: If you're using DVC with cloud storage, take note of environment
@@ -423,14 +423,14 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: iterative/setup-cml@v1
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Deploy runner on EC2
         env:
           REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
           AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
         run: |
-          cml runner \
+          cml runner launch \
             --cloud=aws \
             --cloud-region=us-west \
             --cloud-type=g4dn.xlarge \
@@ -440,10 +440,10 @@ jobs:
     runs-on: [self-hosted, cml-gpu]
     timeout-minutes: 50400 # 35 days
     container:
-      image: docker://iterativeai/cml:0-dvc2-base1-gpu
+      image: ghcr.io/iterative/cml:0-dvc2-base1-gpu
       options: --gpus all
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Train model
         env:
           REPO_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
@@ -452,7 +452,7 @@ jobs:
           python train.py
 
           cat metrics.txt > report.md
-          cml send-comment report.md
+          cml comment create report.md
 ```
 
 In the workflow above, the `deploy-runner` step launches an EC2 `g4dn.xlarge`
@@ -466,72 +466,83 @@ newly-launched instance. See [Environment Variables] below for details on the
 
 #### Docker Images
 
-The CML Docker image (`docker://iterativeai/cml`) comes loaded with Python,
-CUDA, `git`, `node` and other essentials for full-stack data science. Different
-versions of these essentials are available from different `iterativeai/cml`
+The CML Docker image (`ghcr.io/iterative/cml` or `iterativeai/cml`) comes loaded
+with Python, CUDA, `git`, `node` and other essentials for full-stack data
+science. Different versions of these essentials are available from different
 image tags. The tag convention is `{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`:
 
 | `{BASE_VER}` | Software included (`-gpu`)                    |
 | ------------ | --------------------------------------------- |
 | 0            | Ubuntu 18.04, Python 2.7 (CUDA 10.1, CuDNN 7) |
 | 1            | Ubuntu 20.04, Python 3.8 (CUDA 11.2, CuDNN 8) |
 
-For example, `docker://iterativeai/cml:0-dvc2-base1-gpu`, or
-`docker://ghcr.io/iterative/cml:0-dvc2-base1`.
+For example, `iterativeai/cml:0-dvc2-base1-gpu`, or
+`ghcr.io/iterative/cml:0-dvc2-base1`.
 
 #### Arguments
 
-The `cml runner` function accepts the following arguments:
+The `cml runner launch` function accepts the following arguments:
 
 ```
---help                      Show help                                [boolean]
---version                   Show version number                      [boolean]
---log                       Maximum log level
-                 [choices: "error", "warn", "info", "debug"] [default: "info"]
---labels                    One or more user-defined labels for this runner
-                            (delimited with commas)           [default: "cml"]
---idle-timeout              Seconds to wait for jobs before shutting down. Set
-                            to -1 to disable timeout            [default: 300]
---name                      Name displayed in the repository once registered
-                            cml-{ID}
---no-retry                  Do not restart workflow terminated due to instance
-                            disposal or GitHub Actions timeout
-                                                    [boolean] [default: false]
---single                    Exit after running a single job
-                                                    [boolean] [default: false]
---reuse                     Don't launch a new runner if an existing one has
-                            the same name or overlapping labels
-                                                    [boolean] [default: false]
---driver                    Platform where the repository is hosted. If not
-                            specified, it will be inferred from the
-                            environment          [choices: "github", "gitlab"]
---repo                      Repository to be used for registering the runner.
-                            If not specified, it will be inferred from the
-                            environment
---token                     Personal access token to register a self-hosted
-                            runner on the repository. If not specified, it
-                            will be inferred from the environment
-                                                            [default: "infer"]
---cloud                     Cloud to deploy the runner
-                                [choices: "aws", "azure", "gcp", "kubernetes"]
---cloud-region              Region where the instance is deployed. Choices:
-                            [us-east, us-west, eu-west, eu-north]. Also
-                            accepts native cloud regions  [default: "us-west"]
---cloud-type                Instance type. Choices: [m, l, xl]. Also supports
-                            native types like i.e. t2.micro
---cloud-gpu                 GPU type.
-                                    [choices: "nogpu", "k80", "v100", "tesla"]
---cloud-hdd-size            HDD size in GB
---cloud-ssh-private         Custom private RSA SSH key. If not provided an
-                            automatically generated throwaway key will be used
-                                                                 [default: ""]
---cloud-spot                Request a spot instance                  [boolean]
---cloud-spot-price          Maximum spot instance bidding price in USD.
-                            Defaults to the current spot bidding price
-                                                               [default: "-1"]
---cloud-startup-script      Run the provided Base64-encoded Linux shell script
-                            during the instance initialization   [default: ""]
---cloud-aws-security-group  Specifies the security group in AWS  [default: ""]
+  --labels                                  One or more user-defined labels for
+                                            this runner (delimited with commas)
+                                                       [string] [default: "cml"]
+  --idle-timeout                            Time to wait for jobs before
+                                            shutting down (e.g. "5min"). Use
+                                            "never" to disable
+                                                 [string] [default: "5 minutes"]
+  --name                                    Name displayed in the repository
+                                            once registered
+                                                    [string] [default: cml-{ID}]
+  --no-retry                                Do not restart workflow terminated
+                                            due to instance disposal or GitHub
+                                            Actions timeout            [boolean]
+  --single                                  Exit after running a single job
+                                                                       [boolean]
+  --reuse                                   Don't launch a new runner if an
+                                            existing one has the same name or
+                                            overlapping labels         [boolean]
+  --reuse-idle                              Creates a new runner only if the
+                                            matching labels don't exist or are
+                                            already busy               [boolean]
+  --docker-volumes                          Docker volumes, only supported in
+                                            GitLab         [array] [default: []]
+  --cloud                                   Cloud to deploy the runner
+                         [string] [choices: "aws", "azure", "gcp", "kubernetes"]
+  --cloud-region                            Region where the instance is
+                                            deployed. Choices: [us-east,
+                                            us-west, eu-west, eu-north]. Also
+                                            accepts native cloud regions
+                                                   [string] [default: "us-west"]
+  --cloud-type                              Instance type. Choices: [m, l, xl].
+                                            Also supports native types like i.e.
+                                            t2.micro                    [string]
+  --cloud-permission-set                    Specifies the instance profile in
+                                            AWS or instance service account in
+                                            GCP           [string] [default: ""]
+  --cloud-metadata                          Key Value pairs to associate
+                                            cml-runner instance on the provider
+                                            i.e. tags/labels "key=value"
+                                                           [array] [default: []]
+  --cloud-gpu                               GPU type. Choices: k80, v100, or
+                                            native types e.g. nvidia-tesla-t4
+                                                                        [string]
+  --cloud-hdd-size                          HDD size in GB              [number]
+  --cloud-ssh-private                       Custom private RSA SSH key. If not
+                                            provided an automatically generated
+                                            throwaway key will be used  [string]
+  --cloud-spot                              Request a spot instance    [boolean]
+  --cloud-spot-price                        Maximum spot instance bidding price
+                                            in USD. Defaults to the current spot
+                                            bidding price [number] [default: -1]
+  --cloud-startup-script                    Run the provided Base64-encoded
+                                            Linux shell script during the
+                                            instance initialization     [string]
+  --cloud-aws-security-group                Specifies the security group in AWS
+                                                          [string] [default: ""]
+  --cloud-aws-subnet,                       Specifies the subnet to use within
+  --cloud-aws-subnet-id                     AWS           [string] [default: ""]
+
 ```
 
 #### Environment Variables
@@ -556,12 +567,13 @@ CML support proxy via known environment variables `http_proxy` and
 
 #### On-premise (Local) Runners
 
-This means using on-premise machines as self-hosted runners. The `cml runner`
-function is used to set up a local self-hosted runner. On a local machine or
-on-premise GPU cluster, [install CML as a package](#local-package) and then run:
+This means using on-premise machines as self-hosted runners. The
+`cml runner launch` function is used to set up a local self-hosted runner. On a
+local machine or on-premise GPU cluster,
+[install CML as a package](#local-package) and then run:
 
 ```bash
-cml runner \
+cml runner launch \
   --repo=$your_project_repository_url \
   --token=$PERSONAL_ACCESS_TOKEN \
   --labels="local,runner" \
@@ -577,9 +589,13 @@ pre-installed in a custom Docker image pulled by a CI runner. You can also
 install CML as a package:
 
 ```bash
-npm i -g @dvcorg/cml
+npm install --location=global @dvcorg/cml
 ```
 
+You can use `cml` without node by downloading the correct standalone binary for
+your system from the asset section of the
+[releases](https://github.com/iterative/cml/releases).
+
 You may need to install additional dependencies to use DVC plots and Vega-Lite
 CLI commands:
 
@@ -599,15 +615,15 @@ CML and Vega-Lite package installation require the NodeJS package manager
   use a set up action to install NodeJS:
 
 ```bash
-uses: actions/setup-node@v2
+uses: actions/setup-node@v3
   with:
     node-version: '16'
 ```
 
 - **GitLab**: Requires direct installation.
 
 ```bash
-curl -sL https://deb.nodesource.com/setup_12.x | bash
+curl -sL https://deb.nodesource.com/setup_16.x | bash
 apt-get update
 apt-get install -y nodejs
 ```
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "@dvcorg/cml",
-  "version": "0.18.7",
+  "version": "0.18.8",
   "description": "<p align=\"center\"> <img src=\"https://static.iterative.ai/img/cml/title_strip_trim.png\" width=400> </p>",
   "author": {
     "name": "Iterative Inc",

Original file line number	Diff line number	Diff line change
`@@ -1,6 +1,6 @@`
`1`	`1`	`{`
`2`	`2`	`"name": "@dvcorg/cml",`
`3`		`- "version": "0.18.7",`
	`3`	`+ "version": "0.18.8",`
`4`	`4`	`"description": "<p align=\"center\"> <img src=\"https://static.iterative.ai/img/cml/title_strip_trim.png\" width=400> </p>",`
`5`	`5`	`"author": {`
`6`	`6`	`"name": "Iterative Inc",`