Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

# Normalize text files in the repository
* text=auto
.gitattributes text eol=lf

# Source code
*.py text eol=lf
Expand All @@ -15,6 +16,7 @@
*.yaml text eol=lf
*.toml text eol=lf
*.json text eol=lf
*.geojson text eol=lf

# Documentation
*.md text eol=lf
Expand Down
94 changes: 78 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,49 @@ A spatial operation requires geometries to use a compatible spatial reference sy

---

### LAB 02 - Spatial Exploratory Data Analysis

[Open the recruiter-facing notebook](labs/lab02_spatial_eda/02_spatial_eda.ipynb)

**Problem:** an apparent spatial hotspot can reflect uneven sampling intensity rather than a strong underlying spatial signal.

The walkthrough demonstrates:

- synthetic spatial sampling with regional and campaign components
- projected analysis in WGS 84 / UTM Zone 19S (`EPSG:32719`)
- nearest-neighbour sampling-density diagnostics
- regular-grid aggregation
- separation of sampling density from underlying synthetic signal
- grid-resolution sensitivity using persisted Monte Carlo results
- geographic context with documented Natural Earth provenance
- reusable spatial utilities
- automated tests

Key numerical evidence:

| Metric | Result |
| --- | ---: |
| Observations | 800 |
| Regional mean NN distance | 1807.35 m |
| Campaign mean NN distance | 502.73 m |
| Campaign / regional NN ratio | 0.278 |
| Primary descriptive grid | 10 km |
| Densest cell | `c2_r3` |
| Highest observed-signal cell | `c6_r6` |
| Highest true-signal cell | `c6_r6` |

The densest cell differs from both signal hotspots, while the highest observed-signal cell agrees with the highest true-signal cell in this synthetic realization.

The 10 km grid is used only as the primary descriptive resolution for this experiment. It is not presented as a universal optimum.

**Core lesson:**

```text
sampling density is not underlying spatial signal
```

---

## Why this matters for GeoAI

Spatial errors can silently propagate into features such as:
Expand Down Expand Up @@ -102,34 +145,51 @@ technical interpretation

The current public portfolio includes:

- 2 recruiter-facing geospatial notebooks
- reusable CRS utilities under `src/geoai/`
- 3 recruiter-facing geospatial notebooks
- reusable CRS and spatial-analysis utilities under `src/geoai/`
- automated pytest validation
- 15 passing CRS-related tests
- 24 passing automated tests
- persisted synthetic Monte Carlo result artifacts for LAB 02
- documented provenance for the LAB 02 geographic context layer
- reproducible Conda environment definition
- pre-commit quality controls
- Gitleaks secret scanning
- synthetic and redistribution-safe examples
- synthetic and redistribution-safe analytical examples

## Repository structure

```text
geoai-mineral-portfolio/
|
|-- labs/
| `-- lab01_geospatial_foundations/
| |-- 01a_coordinate_foundations.ipynb
| |-- 01b_cross_zone_crs.ipynb
| `-- README.md
| |-- lab01_geospatial_foundations/
| | |-- 01a_coordinate_foundations.ipynb
| | |-- 01b_cross_zone_crs.ipynb
| | `-- README.md
| |
| `-- lab02_spatial_eda/
| |-- 02_spatial_eda.ipynb
| |-- 02_spatial_eda.py
| |-- README.md
| |-- data/
| | `-- context/
| | |-- chile_context.geojson
| | `-- SOURCE.md
| `-- results/
| |-- experiment_metadata.json
| |-- monte_carlo_by_realization.csv
| `-- resolution_summary.csv
|
|-- src/
| `-- geoai/
| |-- __init__.py
| `-- crs.py
| |-- crs.py
| `-- spatial.py
|
|-- tests/
| |-- test_crs.py
| `-- test_cross_zone_crs.py
| |-- test_cross_zone_crs.py
| `-- test_spatial.py
|
|-- environment.yml
|-- pyproject.toml
Expand Down Expand Up @@ -202,10 +262,10 @@ python -m pytest -q
Current validated result:

```text
15 passed
24 passed
```

The reusable CRS logic is tested independently from the explanatory notebooks.
Reusable CRS and spatial-analysis logic is tested independently from the explanatory notebooks.

## Data strategy

Expand All @@ -218,14 +278,16 @@ This approach keeps the experiments:
- independent of proprietary exploration datasets
- focused on the spatial concept being demonstrated

LAB 02 additionally uses a Natural Earth country boundary only for geographic context. Its source, version, license status, and integrity hashes are documented alongside the local context file. The analytical experiment itself remains entirely synthetic.

Future external datasets will require explicit review of source, license, attribution, CRS, resolution, redistribution rights, and known limitations.

## Current roadmap

| Lab | Topic | Status |
| --- | --- | --- |
| LAB 01 | Geospatial Python Foundations | In progress |
| LAB 02 | Spatial Exploratory Data Analysis | Planned |
| LAB 01 | Geospatial Python Foundations | Complete |
| LAB 02 | Spatial Exploratory Data Analysis | Complete |
| LAB 03 | Spatial Cross-Validation | Planned |
| LAB 04 | Geochemical CoDA | Planned |
| LAB 05 | Mineral Prospectivity Mapping | Planned |
Expand All @@ -235,9 +297,9 @@ Future external datasets will require explicit review of source, license, attrib
| LAB 09 | Deep Learning for GeoAI | Planned |
| LAB 10 | Graph & Multimodal GeoAI | Planned |

LAB 01A and LAB 01B are completed and validated.
LAB 01 and LAB 02 are completed and validated.

Additional LAB 01 work will progressively cover geometry validation, spatial operations, raster foundations, and raster/vector integration.
LAB 03 will extend the portfolio from exploratory spatial reasoning into spatial cross-validation, with explicit attention to spatial leakage and the limitations of conventional random train/test splits.

## Technical principles

Expand Down
1,589 changes: 1,589 additions & 0 deletions labs/lab02_spatial_eda/02_spatial_eda.ipynb

Large diffs are not rendered by default.

221 changes: 221 additions & 0 deletions labs/lab02_spatial_eda/02_spatial_eda.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
from __future__ import annotations

from geoai.spatial import (
assign_regular_grid,
generate_synthetic_sampling,
nearest_neighbor_distances,
require_metric_projected_crs,
summarize_grid,
)


PRIMARY_CELL_SIZE_M = 10000.0

GRID_ORIGIN = (
350000.0,
7100000.0,
)


def main() -> None:
samples = generate_synthetic_sampling()

crs = require_metric_projected_crs(samples)

regional = samples[
samples["sampling_component"] == "regional"
].copy()

campaign = samples[
samples["sampling_component"] == "campaign"
].copy()

regional_nn = nearest_neighbor_distances(regional)
campaign_nn = nearest_neighbor_distances(campaign)

regional_mean_nn = float(regional_nn.mean())
campaign_mean_nn = float(campaign_nn.mean())

nn_ratio = campaign_mean_nn / regional_mean_nn

gridded = assign_regular_grid(
samples,
cell_size_m=PRIMARY_CELL_SIZE_M,
origin=GRID_ORIGIN,
)

observed_summary = summarize_grid(
gridded,
signal_column="signal",
)

true_summary = summarize_grid(
gridded,
signal_column="signal_true",
)

densest_cell = observed_summary.loc[
observed_summary["observation_count"].idxmax()
]

highest_observed_signal_cell = observed_summary.loc[
observed_summary["mean_signal"].idxmax()
]

highest_true_signal_cell = true_summary.loc[
true_summary["mean_signal"].idxmax()
]

observations_preserved = int(
observed_summary["observation_count"].sum()
)

densest_differs_from_observed = (
densest_cell["grid_id"]
!= highest_observed_signal_cell["grid_id"]
)

densest_differs_from_true = (
densest_cell["grid_id"]
!= highest_true_signal_cell["grid_id"]
)

observed_agrees_with_true = (
highest_observed_signal_cell["grid_id"]
== highest_true_signal_cell["grid_id"]
)

print("=" * 72)
print("LAB 02 - SPATIAL EXPLORATORY DATA ANALYSIS")
print("=" * 72)

print("\nDATASET")
print(f"Observations: {len(samples)}")
print(f"CRS: {crs.to_string()}")

print("\nSAMPLING COMPONENTS")
print(f"Regional observations: {len(regional)}")
print(f"Campaign observations: {len(campaign)}")

print("\nNEAREST-NEIGHBOUR DISTANCE")
print(
"Regional mean NN distance (m): "
f"{regional_mean_nn:.2f}"
)
print(
"Campaign mean NN distance (m): "
f"{campaign_mean_nn:.2f}"
)
print(
"Campaign/regional NN ratio: "
f"{nn_ratio:.3f}"
)
print(
"Campaign more densely sampled: "
f"{campaign_mean_nn < regional_mean_nn}"
)

print("\nREGULAR GRID")
print(
"Cell size (m): "
f"{PRIMARY_CELL_SIZE_M:.0f}"
)
print(
"Grid origin: "
f"{GRID_ORIGIN}"
)
print(
"Populated cells: "
f"{len(observed_summary)}"
)
print(
"Observations preserved: "
f"{observations_preserved}"
)

print("\nDENSEST CELL")
print(
"Grid ID: "
f"{densest_cell['grid_id']}"
)
print(
"Observation count: "
f"{int(densest_cell['observation_count'])}"
)
print(
"Mean observed signal: "
f"{float(densest_cell['mean_signal']):.2f}"
)

print("\nHIGHEST OBSERVED-SIGNAL CELL")
print(
"Grid ID: "
f"{highest_observed_signal_cell['grid_id']}"
)
print(
"Observation count: "
f"{int(highest_observed_signal_cell['observation_count'])}"
)
print(
"Mean observed signal: "
f"{float(highest_observed_signal_cell['mean_signal']):.2f}"
)

print("\nHIGHEST TRUE-SIGNAL CELL")
print(
"Grid ID: "
f"{highest_true_signal_cell['grid_id']}"
)
print(
"Observation count: "
f"{int(highest_true_signal_cell['observation_count'])}"
)
print(
"Mean true signal: "
f"{float(highest_true_signal_cell['mean_signal']):.2f}"
)

print("\nKEY RESULTS")
print(
"Densest cell differs from highest observed-signal cell: "
f"{densest_differs_from_observed}"
)
print(
"Densest cell differs from highest true-signal cell: "
f"{densest_differs_from_true}"
)
print(
"Highest observed-signal cell agrees with true-signal cell: "
f"{observed_agrees_with_true}"
)

print("\nINTERPRETATION")
print(
"The synthetic sampling campaign produces a clear "
"sampling-density hotspot."
)
print(
"The densest grid cell is separated from the strongest "
"underlying synthetic signal."
)
print(
"A 10 km grid is used as the primary descriptive resolution "
"for this experiment."
)
print(
"Monte Carlo sensitivity analysis showed that this resolution "
"provides a useful compromise between hotspot stability, "
"local sample support, and preservation of spatial signal."
)
print(
"The selected resolution is specific to this synthetic "
"experiment and is not presented as a universal optimum."
)
print(
"Sampling density must therefore be distinguished from "
"underlying spatial signal before predictive modelling."
)


if __name__ == "__main__":
main()
Loading
Loading