EmotionAdapt AI

A real-time, emotion-aware web application that dynamically adapts its entire UI — typography, color palette, layout, and animations — based on the user's facial expression, powered by face-api.js and React 19.

---

Table of Contents

• Overview
• Features
• Tech Stack
• Architecture
• Emotion Themes
• Getting Started
• Project Structure
• How It Works
• Fallback & Accessibility
• Scripts
• License

---

Overview

EmotionAdapt AI uses your device's webcam to continuously detect facial expressions and map the dominant emotion to a distinct visual theme. Every aspect of the interface — background gradients, font families, font sizes, spacing, border radius, particle colors, and layout columns — shifts fluidly in response to how you feel. No backend required; all ML inference runs entirely client-side via WebAssembly.

---

Features

• 🎭 Real-time emotion detection at configurable inference rates (400 ms default, 1000 ms on low-end devices)
• 🎨 7 adaptive themes — one for each detected emotion (happy, sad, angry, fearful, surprised, disgusted, neutral)
• 🖋️ Adaptive typography — font family, size, weight, letter-spacing, and line-height all change per emotion
• 🌈 Adaptive backgrounds — gradient, particle, and shadow colors shift per emotion
• 📐 Adaptive layouts — column count and spacing reconfigures with each emotional state
• 🔀 Emotion smoothing — exponential moving average prevents flickering between transient expressions
• 🕹️ Manual override HUD — force any emotion via the in-app HUD panel, even without a camera
• 🚫 No-face fallback — automatically reverts to neutral after 5 seconds of no detected face
• ⚙️ Low-end device detection — reduces inference rate automatically on hardware with fewer than 4 CPU cores
• 🔐 Consent gate — explicit user consent before webcam access is requested
• 🏭 100% client-side — all face detection models run in the browser via WebAssembly; no data leaves the device

---

Tech Stack

Layer	Technology
UI Framework	React 19
Language	TypeScript 5.9
Build Tool	Vite 7
Face Detection	face-api.js 0.22 (TensorFlow.js + WASM)
Animations	Framer Motion 12
Styling	Vanilla CSS with CSS custom properties
Linting	ESLint 9 + typescript-eslint

---

Architecture

src/
├── core/
│   ├── EmotionEngine.ts        # face-api.js wrapper — runs expression detection
│   ├── EmotionSmoother.ts      # EMA-based smoothing of raw expression scores
│   └── WebcamController.ts     # MediaDevices API abstraction with error codes
│
├── context/
│   └── EmotionContext.tsx      # Global state: dominant emotion, scores, theme, detection loop
│
├── themes/
│   ├── types.ts                # ThemeConfig interface
│   ├── themeMap.ts             # Maps emotion string → ThemeConfig
│   ├── neutral.theme.ts
│   ├── happy.theme.ts
│   ├── sad.theme.ts
│   ├── angry.theme.ts
│   ├── fearful.theme.ts
│   ├── surprised.theme.ts
│   └── disgusted.theme.ts
│
├── components/
│   ├── ConsentGate/            # Webcam permission consent screen
│   ├── ModelLoader/            # face-api.js model download progress UI
│   ├── AdaptiveShell/          # Root wrapper that injects CSS variables from theme
│   ├── AdaptiveBackground/     # Animated gradient + particle canvas
│   ├── AdaptiveLayout/         # CSS grid that reflows columns per emotion
│   ├── AdaptiveTypography/     # Typography container driven by theme tokens
│   ├── TopNav/                 # Navigation bar with live emotion indicator
│   ├── EmotionHUD/             # Floating panel with scores + manual override controls
│   └── DemoContent/            # Sample content for showcasing theme transitions
│
└── App.tsx                     # App state machine: consent → loading → running

---

Emotion Themes

Each detected emotion maps to a fully specified ThemeConfig object with the following tokens:

Token	Description
bg / surface	Background and card surface colors
primary / accent	Brand and interactive accent colors
text	Body text color
fontFamily	Font stack (e.g., serif for sad, monospace for fearful)
fontSize	Base font size
fontWeight	Body font weight
letterSpacing / lineHeight	Typographic rhythm
borderRadius	Corner rounding (sharp for angry, pill for happy)
spacing	Layout density (compact / normal / relaxed)
layoutColumns	Number of content columns
shadowColor / particleColor	Atmospheric visual effects
gradientStart / gradientEnd	Background gradient stops
transitionDuration	Theme cross-fade speed in milliseconds

---

Getting Started

Prerequisites

• Node.js ≥ 18
• npm ≥ 9
• A device with a webcam (or use the manual HUD override)

Installation

# 1. Clone the repository
git clone https://github.com/your-username/emotion-adapt-ai.git
cd emotion-adapt-ai

# 2. Install dependencies
npm install

# 3. Start the development server
npm run dev

Open http://localhost:5173 in your browser.

Note: face-api.js model weights are loaded from the /public/models directory at runtime. Ensure these files are present before starting the app.

---

How It Works

1. Consent Gate — The user grants explicit webcam permission before any camera access is attempted.
2. Model Loading — face-api.js loads the tiny face detector and expression recognition models from /public/models.
3. Detection Loop — EmotionEngine.ts runs faceapi.detectSingleFace().withFaceExpressions() on the live video feed at the configured interval.
4. Smoothing — Raw expression scores are passed through EmotionSmoother, which applies an exponential moving average to stabilize noisy frame-by-frame predictions.
5. Dominant Emotion — The emotion with the highest smoothed score is selected as the dominant emotion.
6. Theme Injection — EmotionContext calls getTheme(dominantEmotion) and updates global state. AdaptiveShell reads the new ThemeConfig and writes CSS custom properties onto the root element.
7. Framer Motion — All theme transitions are animated with Framer Motion for smooth cross-fades and layout shifts.

---

Fallback & Accessibility

• No camera available — The app displays a warning banner and falls back to manual emotion override via the HUD panel.
• No face detected — After 5 seconds (configurable via noFaceThreshold), the UI resets to the neutral theme.
• Low-end hardware — Devices with fewer than 4 logical CPU cores automatically run detection at 1000 ms intervals instead of 400 ms.
• Manual override — The EmotionHUD component provides buttons to force any emotion at any time, regardless of camera state.

---

Scripts

Command	Description
npm run dev	Start Vite development server with HMR
npm run build	Type-check with tsc and produce a production bundle
npm run preview	Serve the production bundle locally
npm run lint	Run ESLint across the entire source tree

---

License

This project is licensed under the MIT License.

---

Built with ❤️ using React, face-api.js, and Framer Motion