An interactive AI-powered physics tutoring application built with Next.js that provides real-time audio/video conversations with Teacher Be, an AI tutor specialized in physics education.
Note: This codebase is originally from better-ed using Gemini that has been retrofitted for Gemma. Some remnants of Gemini may exist, but is not used.
- Real-time AI Conversations: Interactive audio/video conversations with Teacher Be using Gemma 3n AI model
- Multimodal Interaction:
- Real-time audio processing with speech detection
- Video capture and display
- Image capture during conversations
- Audio worklet processing for low-latency performance
- WebSocket Integration: Real-time communication with Gemma AI for instant responses
- Firebase Integration: Automatic conversation logging and storage
- Responsive Design: Modern UI with Tailwind CSS and Radix UI components
- Next.js 15: React framework with App Router
- TypeScript: Type-safe development
- Tailwind CSS: Utility-first styling
- Radix UI: Accessible component primitives
- Gemma 3n AI Model: Google's multimodal AI for real-time conversations
- WebSocket: Real-time bidirectional communication
- Audio Processing: Custom audio worklets for low-latency processing
- Speech Recognition: Real-time transcription and response generation
- Firebase Firestore: Conversation logging and data persistence
- Firebase App: Backend infrastructure
- Web Audio API: Real-time audio processing and playback
- MediaDevices API: Camera and microphone access
- Canvas API: Image capture and processing
- Node.js 18+ installed
- Modern browser with WebRTC support
- Microphone and camera access
- Environment variables configured (see Configuration section)
- Clone the repository
git clone <repository-url>
cd be-mini- Install dependencies
npm install
# or
yarn install-
Configure environment variables Create a
.env.localfile in the root directory: -
Run the development server
npm run dev
# or
yarn dev- Open your browser Navigate to http://localhost:3000 to start using the app.
- Next.js 15.1.7: React framework with App Router
- React 19: UI library
- TypeScript 5: Type safety
- Tailwind CSS 3.4.1: Utility-first CSS framework
- Radix UI: Accessible component primitives
- Lucide React: Icon library
- Class Variance Authority: Component variant management
- Google Generative AI: Gemma 3n model integration
- WebSocket: Real-time communication
- Web Audio API: Audio processing and playback
- Firebase 11.4.0: Backend services
- Firestore: NoSQL database
- LameJS: Audio encoding/decoding
- JS-Base64: Base64 encoding utilities
The app requires several environment variables to function properly:
- Modern browser with WebRTC support
- Microphone and camera permissions
- HTTPS connection (required for media access in production)
be-mini/
├── app/ # Next.js App Router
│ ├── components/ # React components
│ │ ├── CameraPreview/ # Camera and audio processing
│ │ └── ...
│ ├── hooks/ # Custom React hooks
│ ├── services/ # AI and WebSocket services
│ ├── types/ # TypeScript type definitions
│ └── utils/ # Utility functions
├── components/ # Shared UI components
├── lib/ # Library utilities
├── public/ # Static assets
│ ├── avatars/ # AI and user avatars
│ └── worklets/ # Audio processing worklets
└── ...
npm run build
npm startEnsure all environment variables are properly configured in your deployment platform.
- Fork the repository
- Create a feature branch
- Make your changes
- Test thoroughly
- Submit a pull request
This project is licensed under the MIT License.
For support or questions, please open an issue in the repository or contact the development team.
Better Ed: Education Reimagined - Making learning interactive and engaging through AI-powered conversations.