Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 121 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,24 @@

IPP is a location-based, on-chain-verifiable clinical-records app for Chilean
doctors and medical staff (Spanish UI). It is built as a **prototype /
demonstrator** of a reusable template - GPS + location-based AR capture on iOS,
backed by a Bun service on Neon Postgres, with each record's hash anchored to
demonstrator** of a reusable template - GPS + AR on iOS (location-based AR at
capture, plus a camera-based ARKit/RealityKit scene), backed by a Bun service on
Neon Postgres, with each record's hash anchored to
**Cardano** (through the **EffectStream** packages) so the data
can be cryptographically verified later.

The app pairs **GPS** (where each patient lives) with **location-based AR**
(live population context augmenting what the clinician sees as they work) and a
(live population context augmenting what the clinician sees as they work),
**camera-based AR** (an ARKit/RealityKit scene that anchors a podium to the real
world and renders the geolocated records as a living data map on the floor -
see [Tiro al Trofeo](#tiro-al-trofeo---camera-based-ar-mini-game)) and a
**gamified** contribution layer that makes the dataset grow.

## What this is

- **An iOS app** (SwiftUI) for capturing a ~70-question women's-health intake
form across four sections.
form across four sections, plus a camera-based AR mini-game on the
leaderboard (ARKit/RealityKit).
- **A web dashboard** (Vite + React + Leaflet) for population maps, filters,
feedback, and on-chain verification - also embedded inside the iOS app.
- **A Bun + Fastify backend** on Neon Postgres, with a swappable chain adapter.
Expand All @@ -27,7 +32,7 @@ The app pairs **GPS** (where each patient lives) with **location-based AR**
GPS is the backbone: every patient has an address that geocodes to a
latitude/longitude. That location unlocks **augmented reality anchored to
place** - augmenting what the clinician sees about the physical world in front
of them, in two places:
of them, in three places:

1. **At capture.** As you enter a value, the field shows the population context
for it - the **local** (the patient's own locality), **país** (country), and
Expand All @@ -36,11 +41,17 @@ of them, in two places:
2. **On the map.** Doctors draw **notes and named areas** over the filtered
population layer, turning patterns into plans - e.g. *"many patients in this
zone need X, assign a specialist and schedule exams here."*

> This is **location-based AR**: the augmentation is anchored to physical place
> through GPS rather than to a camera feed. The reality being augmented is the
> clinician's view of the patient and population in front of them, keyed to
> where the patient actually lives.
3. **Through the camera.** The
[Tiro al Trofeo](#tiro-al-trofeo---camera-based-ar-mini-game) scene uses
ARKit/RealityKit world tracking to anchor a virtual podium onto the real
surface in front of the user and projects the anonymized record locations as
a living data map on the actual floor around it.

> The first two are **location-based AR** - the augmentation is anchored to
> physical place through GPS rather than to a camera feed, and the reality being
> augmented is the clinician's view of the patient and population in front of
> them, keyed to where the patient actually lives. The third is **camera-based
> AR** consuming the same GPS data through the phone's camera.

**Why we prioritized it:** women's-health and pelvic-floor risk cluster
geographically. Location-aware context at the point of capture (and on the map)
Expand Down Expand Up @@ -77,7 +88,9 @@ size the radius of affectation, and act locally.
│ ├── Models/ # Patient, FormSchema, FieldStats, ...
│ ├── Services/ # APIPatientStore, SessionService, Wallet, SchemaService, AppEnvironment
│ ├── Components/ # AddressPicker (CoreLocation), MultiSelectAutocomplete
│ ├── Game/ # Tiro al Trofeo - ARKit/RealityKit podium mini-game
│ └── Views/ # Home, DynamicForm (StatCaption), Leaderboard, WebDashboard, Theme
├── ios/IPPTests/ # unit tests - game logic, geometry, backend locator (no backend needed)
├── web/src/ # Vite + React dashboard
│ └── components/ # MapView, MapFilters, AnnotationsLayer, DrawingController, Feedback, PinVerify
└── cardano/ # EffectStream workspace - local Cardano devnet + sync
Expand Down Expand Up @@ -112,7 +125,9 @@ areas** ([AnnotationsLayer](web/src/components/AnnotationsLayer.tsx),
[DrawingController](web/src/components/DrawingController.tsx),
[AnnotationsList](web/src/components/AnnotationsList.tsx)) to mark interventions -
the manual planning layer that turns a cluster into "assign a specialist here."
(Again: location-based AR anchored through GPS, not a camera overlay.)
(Again: this layer is anchored through GPS, not through the camera - the one
camera-based piece in IPP is the
[Tiro al Trofeo](#tiro-al-trofeo---camera-based-ar-mini-game) mini-game.)

### Engine

Expand Down Expand Up @@ -147,6 +162,48 @@ Search actions are logged via `POST /api/v1/events`; field/record counts are
computed from the stored data. Each account also has a Cardano wallet address
shown in the app, tying the contributor to an on-chain identity.

### Tiro al Trofeo - camera-based AR mini-game

The leaderboard has a **"Jugar"** entry point that opens **Tiro al Trofeo**, a
single-player ARKit/RealityKit mini-game built around the podium the ranking
already draws in gold/silver/bronze
([ios/IPP/Game/](ios/IPP/Game/), opened from
[LeaderboardView.swift](ios/IPP/Views/LeaderboardView.swift)).

> **Video:** <https://www.youtube.com/watch?v=7qFVhvGDVyk> - a full round on
> device: place the podium → flick → make → summary.

- **Place it.** Scan a desk or the floor, tap a detected horizontal plane, and a
procedural podium appears - three steps in the leaderboard's exact medal
colours plus a trophy cup. No 3D asset files: everything is generated in code.
- **Play it.** Flick upward from anywhere on the screen; the ball launches from
where your finger started, aimed where the phone points. Touching the cup
scores **+1**, landing inside scores **+10**, and the cup hops to a different
step after every make. Rounds are 60 s with a countdown, an end-of-round
summary, and a **device-local** best score.
- **Watch it.** The steps slowly breathe, the top-3 places float above them as
labels, and a field of dots on the floor around the podium maps where the
app's records are, pulsing so it reads as live data.
- **The game awards no leaderboard points and writes nothing.** It never touches
the ranking, never posts an event, and its only persistence is one integer in
`UserDefaults`. The game module itself makes **zero** network requests.
- **Offline behaviour.** The one thing that is live is the floor map: the app
layer (not the game) reads the public, anonymized `GET /api/v1/map-pins` and
hands the game plain coordinates. With the backend up the caption reads
**"Datos en vivo · N ubicaciones"**; with it down or unreachable the app
substitutes a synthetic offline sample and the caption reads **"Datos de
ejemplo · N ubicaciones"**. The game is fully playable either way.
- **Permissions.** The camera is requested the first time you open the game -
never at app launch - and denying it shows a Spanish explanation with a
shortcut to Ajustes. On devices without ARKit world tracking (and in the
Simulator) the "Jugar" row is disabled with a label saying why; the rest of
the app is unaffected.

Game logic that does not need a camera - toss physics, rounds, best score,
podium and floor-map geometry - is covered by unit tests in
[ios/IPPTests/](ios/IPPTests/) (`xcodebuild test`, see
[iOS app](#ios-app)).

## Cardano anchor

The [`cardano/`](cardano/) workspace runs a local Cardano devnet on the
Expand Down Expand Up @@ -297,6 +354,32 @@ curl http://localhost:3334/health

Schema is created on startup (idempotent `CREATE TABLE IF NOT EXISTS`).

#### Running against a local Postgres instead of Neon

The backend targets Neon, and three things bite if you point it at a local
database instead. None of them is fixed in code (a fix would touch the
production path); they are written down here so the next person does not
rediscover them:

- **Postgres must serve TLS.** [backend/src/db.ts](backend/src/db.ts) passes
`ssl: "require"` as a postgres.js *client option*, which overrides any
`sslmode` in `DATABASE_URL`; unlike `prefer`, `require` never falls back to
plaintext, so a stock `docker run postgres:16` fails the handshake. A
self-signed certificate is enough (`require` does not verify the chain):
generate `server.crt`/`server.key`, copy them into the container's `PGDATA`
(owner `postgres`, key mode `600`), `ALTER SYSTEM SET ssl = on`, and restart
the container.
- **`scripts/seed-cities.ts` no longer works.** It POSTs `/api/v1/patients`
with only a `Content-Type` header, but that route is behind `requireDoctor`
and every row comes back `401 missing auth headers`. The script predates the
signed-request auth ([backend/src/auth.ts](backend/src/auth.ts)).
- **`scripts/seed-year.ts` needs a column the schema no longer creates.** Its
INSERT still lists the legacy plaintext `passcode`, while `initSchema` now
creates only `passcode_hash`. One
`ALTER TABLE patients ADD COLUMN IF NOT EXISTS passcode TEXT` before seeding
makes it run; it writes straight to Postgres, so it needs no auth and is the
richer data set anyway.

### Cardano devnet (for `CHAIN=cardano`)

```bash
Expand Down Expand Up @@ -324,12 +407,28 @@ brew install xcodegen # one-time
cd ios
xcodegen generate
open IPP.xcodeproj # pick an iPhone simulator, ⌘R

# unit tests (no backend needed - every suite is offline and deterministic)
xcodebuild test -scheme IPP -destination 'platform=iOS Simulator,name=iPhone 17 Pro'
```

The app talks to `http://localhost:3334` and embeds the web dashboard at
`http://localhost:5174` (both set via `Info.plist`: `BackendURL`, `WebURL`).
Demo logins: `user01`…`user10` / `pass01`…`pass10`.

**Finding the backend from a real iPhone.** `localhost` is the phone itself, so
a device build cannot use `BackendURL` as-is. At launch the app asks each host
in the `Info.plist` array **`BackendCandidates`** for `/health` (1.5 s each, in
order) and points every client - login, patients, field stats, schema,
leaderboard, map pins - at the first one that answers, falling back to
`BackendURL` if none does
([ios/IPP/Services/BackendLocator.swift](ios/IPP/Services/BackendLocator.swift)).
To run against your own Mac, put its LAN address in that array - a plist edit,
no Swift change. In the Simulator the list is skipped and `BackendURL` is used
directly. Requests wait for that resolution, so nothing can leave with a
half-resolved URL; when nothing is reachable the first request pays the probe
timeout once and then the app behaves offline as before.

## Doctor authentication

Doctor-scope endpoints require a **signed request**: the client signs
Expand All @@ -343,7 +442,11 @@ deterministically from the account seed, sending `X-IPP-PubKey` /

## Demo - video & screenshots

> A screen recording of the end-to-end flow (capture → location-based AR stats → save →
> **Camera-AR demo (published):** <https://www.youtube.com/watch?v=7qFVhvGDVyk> -
> a single on-device take: leaderboard → Jugar → plane detection → podium
> anchored on the real table → floor data map → physics toss gameplay.
>
> A screen recording of the end-to-end clinical flow (capture → location-based AR stats → save →
> on-chain verify → population map → gamified leaderboard) and screenshots will
> be added here / in the [EffectStream blog post](https://effectstream.github.io/docs/blog/ipp-clinical-records-cardano).

Expand All @@ -362,9 +465,12 @@ is the long-form write-up of the engineering and use-cases.

## What's intentionally not done

- **AR is location-based, not camera-based.** There is no ARKit/RealityKit
overlay; the augmentation is anchored to place through GPS - the
location-aware stats and map planning layer.
- **The clinical AR is location-based, not camera-based.** Everything that
augments the *work* - the local/país/mundo stat lines and the map planning
layer - is anchored to place through GPS, not to a camera feed. The only
camera-based AR in IPP is the [Tiro al Trofeo](#tiro-al-trofeo---camera-based-ar-mini-game)
mini-game on the leaderboard, which is a game and nothing else: it reads no
clinical record, awards no points and writes nothing.
- **Demo accounts ship fixed seeds** - fine for a demo, but a real deployment
needs per-user generated keys (see Roadmap).
- **No smart-contract token mint** - the chain layer is metadata anchoring only.
Expand Down
71 changes: 71 additions & 0 deletions ios/IPP/Game/ARSupport.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import ARKit
import AVFoundation
import Foundation

/// Capability and camera-permission gate for the "Tiro al Trofeo" AR mini-game.
///
/// Nothing here starts a camera on its own:
/// - `isWorldTrackingSupported` and `cameraPermission` only *read* state iOS
/// already knows, so they are safe to call from the leaderboard while
/// deciding whether to show the entry point.
/// - `requestCameraAccess()` is the only call that can raise the system prompt,
/// and it is invoked exclusively from `TrophyTossView` — i.e. when the player
/// opens the game, never at app launch (FR-010).
///
/// This type is offline by construction: it performs no networking (FR-008).
enum ARSupport {

// MARK: - Device capability

/// `true` when the device can run ARKit world tracking.
///
/// Returns `false` in the iOS Simulator and on hardware without
/// world-tracking support, which is what hides/disables the game entry
/// point (FR-001).
static var isWorldTrackingSupported: Bool {
ARWorldTrackingConfiguration.isSupported
}

/// Short Spanish explanation shown next to the disabled entry point.
static var unsupportedMessage: String {
#if targetEnvironment(simulator)
return "El mini-juego usa la cámara: solo funciona en un iPhone real."
#else
return "Este iPhone no admite el seguimiento de realidad aumentada."
#endif
}

// MARK: - Camera permission

/// The camera answer the player has already given, if any.
enum CameraPermission: Equatable {
/// The player has not been asked yet — asking is up to the game view.
case notDetermined
case granted
case denied
/// Blocked by the device itself (parental controls, MDM profile).
case restricted
}

/// Current camera authorization, read without touching the camera.
static var cameraPermission: CameraPermission {
switch AVCaptureDevice.authorizationStatus(for: .video) {
case .authorized: return .granted
case .denied: return .denied
case .restricted: return .restricted
case .notDetermined: return .notDetermined
@unknown default: return .denied
}
}

/// Raises the system camera prompt when the answer is still
/// `.notDetermined`, and reports the resulting permission.
///
/// Call this **only** when the game view appears (FR-010).
@discardableResult
static func requestCameraAccess() async -> CameraPermission {
guard cameraPermission == .notDetermined else { return cameraPermission }
_ = await AVCaptureDevice.requestAccess(for: .video)
return cameraPermission
}
}
52 changes: 52 additions & 0 deletions ios/IPP/Game/BestScoreStore.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import Foundation

/// The one thing "Tiro al Trofeo" remembers between launches: a single integer
/// best score (FR-008, US2).
///
/// That is the whole persistence story for the game — no round history, no
/// player profile, nothing that leaves the device, and nothing that touches the
/// app's own data or the leaderboard. The spec is explicit that a device-local
/// best score is the *only* persistence allowed, so this type is deliberately
/// small enough to audit at a glance.
///
/// `UserDefaults` is injectable so tests can run against their own suite
/// instead of the app's, and so two instances can be pointed at the same suite
/// to prove the value really survives (which is the part that matters — the
/// store holds no cached copy, every read goes to disk).
struct BestScoreStore {

/// Key under which the best score lives. Namespaced so it cannot collide
/// with anything the rest of the app stores.
static let defaultKey = "com.nonturing.ipp.trophyToss.bestScore"

private let defaults: UserDefaults
private let key: String

init(defaults: UserDefaults = .standard, key: String = BestScoreStore.defaultKey) {
self.defaults = defaults
self.key = key
}

/// The stored best, or 0 when nothing has been stored yet — `integer(forKey:)`
/// already returns 0 for a missing key, and the `max` guards against a value
/// some other writer left negative.
var best: Int { max(defaults.integer(forKey: key), 0) }

/// Records the score of a finished round.
///
/// - Returns: `true` only when this round beat the stored best, which is
/// also the cue the summary uses to say "¡Nuevo récord!". Equal scores do
/// not count as a new record, and a zero-point round never writes at all.
@discardableResult
func submit(_ score: Int) -> Bool {
guard score > 0, score > best else { return false }
defaults.set(score, forKey: key)
return true
}

/// Forgets the best score. Not reachable from the UI; it exists so tests
/// (and a future debug menu) can start from a clean slate.
func clear() {
defaults.removeObject(forKey: key)
}
}
Loading