Add examples

Author: VeckoTheGecko

intermediate/cataloging.ipynb

@@ -40,20 +40,86 @@
     "\n",
     "Intake was originally developed within Anaconda by the team behind Dask.\n",
     "\n",
-    "### Example using intake v2\n"
+    "### Resources\n",
+    "\n",
+    "Project Pythia has a [cookbook on Intake](https://projectpythia.org/intake-cookbook/notebooks/creating-catalogs/).\n",
+    "\n",
+    "### Example code\n",
+    "\n",
+    "Here we work based off [an Intake tutorial on easy.gems!](https://easy.gems.dkrz.de/Processing/intake.html) from the DKRZ (German Climate Supercomputing Center). Note that the tutorial is based on v1 of Intake (which also uses `intake-xarray`).\n"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "526792bd",
+   "id": "1",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "# Browse an existing catalogue\n",
+    "\n",
+    "import intake\n",
+    "\n",
+    "cat = intake.open_catalog(\"https://data.nextgems-h2020.eu/online.yaml\")\n",
+    "# TIP: Its just a YAML file - open it in your browser and you'll see!\n",
+    "\n",
+    "# view the available subcatalogues or datasets\n",
+    "list(cat)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Access catalogue elements using `[\"attr\"]` or `.attr` notation\n",
+    "# NOTE: The `.attr` notation only works if the entry is a valid Python attribute name - which the entry might not be\n",
+    "\n",
+    "# Let's see what the `tutorial` item is...\n",
+    "cat[\"tutorial\"]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# it's a subcatalogue!\n",
+    "\n",
+    "print(list(cat[\"tutorial\"]))\n",
+    "\n",
+    "# Let's look at an entry within...\n",
+    "cat['tutorial']['ICON.native.2d_PT6H_inst']"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Let's open it as an xarray dataset\n",
+    "\n",
+    "ds = cat['tutorial']['ICON.native.2d_PT6H_inst'].to_dask()\n",
+    "ds"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5",
+   "metadata": {},
+   "source": [
+    "That's it for the example! Feel free to poke around further to see what other datasets are available in this catalogue, or to see which other catalogues are available online."
+   ]
   },
   {
    "cell_type": "markdown",
-   "id": "d6767fe7",
+   "id": "6",
    "metadata": {},
    "source": [
     "## STAC\n",
@@ -62,9 +128,9 @@
     "> The STAC specification is a common language to describe geospatial information, so it can more easily be worked with, indexed, and discovered.\n",
     "> -stacspec.org\n",
     "\n",
-    "Let's set the scene a bit: During sattelite data processing, many image files (predominantly TIFF) get generated. Each of these files can have different spatial and temporal extents, and over time these individual files can number in the millions, containing terrabytes of data in total. When doing analyses on these files, the first thing one needs to know is whether the file is even relevant to the analysis at hand (i.e., at least \"what is the spatial and temporal extent of the file?\"), however downloading the file just to check this metadata within is very wasteful. This is the context from which the STAC project was born.\n",
+    "Let's set the scene a bit: During satellite data processing, many image files (predominantly TIFF) get generated. Each of these files can have different spatial and temporal extents, and over time these individual files can number in the millions, containing terrabytes of data in total. When doing analyses on these files, the first thing one needs to know is whether the file is even relevant to the analysis at hand (i.e., at least \"what is the spatial and temporal extent of the file?\"), however downloading the file just to check this metadata within is very wasteful. This is the context from which the STAC project was born.\n",
     "\n",
-    "The goal of the STAC project is to make geospatial data more accesible by providing a specification to describe assets and their metadata. Adding this metadata alongside the asset (or lifting it out above the asset itself) allows:\n",
+    "The goal of the STAC project is to make geospatial data more accessible by providing a specification to describe assets and their metadata. Adding this metadata alongside the asset (or lifting it out above the asset itself) allows:\n",
     "\n",
     "- The combining of assets into larger collections representing a larger dataset (e.g., a dataset with multiple time slices)\n",
     "- For more efficient workflows, as the whole asset doesn't need to be downloaded to check the metadata.\n",
@@ -83,20 +149,13 @@
     "- https://stacspec.org/en\n",
     "- https://stacindex.org/\n",
     "\n",
-    "### Example using odc-stac\n"
+    "### Example using odc-stac\n",
+    "\n"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f6a76a8e",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
   {
    "cell_type": "markdown",
-   "id": "2",
+   "id": "7",
    "metadata": {},
    "source": [
     "## Commercial offering - Arraylake\n",
@@ -120,7 +179,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "06af9324",
+   "id": "8",
    "metadata": {},
    "source": []
   }

pixi.lock

@@ -105,3 +105,5 @@ flox = ">=0.10.4,<0.11"
 numbagg = ">=0.9.0,<0.10"
 rich = ">=14.0.0,<15"
 jupyterlab_vim = ">=4.1.4,<5"
+intake = "<2"
+intake-xarray = "<2"