🤖 RAG Chatbot with Claude AI

A production-grade Retrieval Augmented Generation (RAG) chatbot built with FastAPI, ChromaDB, and Claude AI. This system answers questions based on your custom knowledge base with lightning-fast responses (1-3 seconds).

🌟 Features

• ⚡ Lightning Fast: 1-3 second responses using Claude Haiku
• 🎯 Accurate Answers: RAG architecture prevents hallucinations
• 📚 Custom Knowledge Base: Add your own documents
• 🐳 Fully Dockerized: One-command deployment
• 🔍 Semantic Search: ChromaDB vector database
• 🎨 Beautiful UI: Real-time chat interface
• 💰 Cost Efficient: ~$0.0003 per query

🏗️ Architecture

┌─────────────────────────────────────────────────────┐
│                  User Interface                      │
│              (React/HTML Frontend)                   │
└────────────────────┬────────────────────────────────┘
                     │
┌────────────────────▼────────────────────────────────┐
│              FastAPI Backend                         │
│         (RAG Logic & Orchestration)                  │
└─────┬──────────────────────────┬────────────────────┘
      │                          │
      ▼                          ▼
┌─────────────┐          ┌──────────────────┐
│  ChromaDB   │          │   Claude Haiku   │
│ (Vector DB) │          │   (Anthropic)    │
└─────────────┘          └──────────────────┘

How It Works

1. User asks a question → Frontend sends to API
2. API queries ChromaDB → Retrieves relevant document chunks
3. Context + Question sent to Claude → Generates grounded answer
4. Response returned → Displayed in UI

🚀 Quick Start

Prerequisites

• Docker Desktop installed and running
• Anthropic API key (Get one here)
• 8GB+ RAM recommended
• Windows, Mac, or Linux

Installation

1. Clone the repository

   git clone <your-repo-url>
   cd rag-chatbot-lab

2. Set up environment variables

   Create a .env file in the project root:

   ANTHROPIC_API_KEY=sk-ant-api-YOUR-KEY-HERE

3. Add your documents

   Place .txt files in the data/documents/ folder:

   data/
   └── documents/
       ├── sample.txt
       ├── your-doc-1.txt
       └── your-doc-2.txt

4. Start the application

   docker-compose up -d

5. Wait for services to initialize (~30 seconds)

6. Open your browser

   Navigate to: http://localhost:3000

That's it! Start chatting with your AI assistant! 🎉

📁 Project Structure

rag-chatbot-lab/
├── api/                        # FastAPI backend
│   ├── Dockerfile
│   ├── main.py                 # Core application logic
│   └── requirements.txt        # Python dependencies
├── frontend/                   # Chat interface
│   └── index.html             # Single-page application
├── data/
│   └── documents/             # Your knowledge base (add .txt files here)
│       └── sample.txt
├── docker-compose.yml         # Container orchestration
├── .env                       # Environment variables (create this)
└── README.md                  # You are here

🔧 Configuration

Environment Variables

Variable	Description	Required	Default
ANTHROPIC_API_KEY	Claude API key	Yes	-
CHROMA_HOST	ChromaDB hostname	No	chromadb
CHROMA_PORT	ChromaDB port	No	8000

Model Configuration

By default, the system uses Claude Haiku for optimal speed and cost.

To change models, edit api/main.py (line ~180):

# Available models:
model="claude-3-haiku-20240307",     # Fast & cheap (current)
model="claude-3-sonnet-20240229",    # Balanced quality/speed
model="claude-3-opus-20240229",      # Highest quality

Customizing Response Length

Edit api/main.py (line ~180):

max_tokens=500,   # Default: medium-length responses
max_tokens=200,   # Short, concise answers
max_tokens=1000,  # Detailed, comprehensive answers

📊 API Endpoints

Health Check

GET http://localhost:8080/health

Returns system status (API, Claude, ChromaDB).

Chat

POST http://localhost:8080/chat
Content-Type: application/json

{
  "message": "What is this lab about?"
}

Ingest Document

POST http://localhost:8080/ingest
Content-Type: application/json

{
  "text": "Your document content here...",
  "metadata": {
    "source": "my-doc",
    "type": "guide"
  }
}

List Collections

GET http://localhost:8080/collections

🧪 Testing

Test the API directly

# Health check
curl http://localhost:8080/health

# Chat test
curl -X POST http://localhost:8080/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"Hello, what can you help me with?"}'

Test in browser

1. Open http://localhost:3000
2. Type a question: "What is this innovation lab about?"
3. Press Send
4. You should get a response in 1-3 seconds ⚡

📝 Adding Documents

Method 1: File Upload (Restart Required)

1. Add .txt files to data/documents/
2. Restart the API:

   docker-compose restart api

3. Documents are automatically loaded on startup

Method 2: API Upload (No Restart)

curl -X POST http://localhost:8080/ingest \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Your content here...",
    "metadata": {"source": "api-upload"}
  }'

Supported Content

• Plain text (.txt) files
• UTF-8 encoding
• Any length (automatically chunked)
• Technical docs, guides, notes, articles, etc.

🐛 Troubleshooting

Issue: "Cannot connect to Docker daemon"

Solution: Make sure Docker Desktop is running.

# Check Docker status
docker --version
docker ps

Issue: API returns blank responses

Solution: Check if API key is set correctly.

# Verify API key in container
docker exec rag-api printenv | findstr ANTHROPIC_API_KEY

# If blank, check .env file exists and restart
docker-compose down
docker-compose up -d

Issue: ChromaDB not connecting

Solution: Wait longer for startup, or restart ChromaDB.

# Restart ChromaDB
docker-compose restart chromadb

# Wait 20 seconds
docker-compose restart api

Issue: Slow responses (>10 seconds)

Solution: You might be using the wrong model.

• Verify you're using claude-3-haiku-20240307
• Check api/main.py line ~180
• Rebuild if changed: docker-compose build api

View Logs

# View all logs
docker-compose logs

# Follow API logs
docker-compose logs -f api

# Follow ChromaDB logs
docker-compose logs -f chromadb

🔄 Maintenance

Update Dependencies

# Rebuild with latest packages
docker-compose build --no-cache api
docker-compose up -d

Clear Data & Start Fresh

# Stop and remove all data
docker-compose down -v

# Start fresh
docker-compose up -d

Backup Your Knowledge Base

# Backup documents
cp -r data/documents /path/to/backup/

# Backup ChromaDB data
docker cp chromadb:/chroma/chroma ./chroma-backup

💰 Cost Analysis

Claude Haiku Pricing:

• Input: $0.25 per 1M tokens
• Output: $1.25 per 1M tokens

Average Query Cost:

• ~500 input tokens (context + question)
• ~200 output tokens (answer)
• Cost per query: ~$0.0003

Example Usage:

• 1,000 queries: ~$0.30
• 10,000 queries: ~$3.00
• 100,000 queries: ~$30.00

Free Tier: Anthropic provides $5 in free credits (~16,000 queries)

📈 Performance Metrics

Metric	Value
Response Time	1-3 seconds
Accuracy	High (RAG-grounded)
Concurrent Users	10-50 (single instance)
Document Limit	Unlimited (chunked)
Memory Usage	~2GB (all services)

🛠️ Tech Stack

• Backend: Python 3.11, FastAPI
• AI Model: Claude Haiku (Anthropic)
• Vector DB: ChromaDB 0.4.24
• Frontend: HTML/JavaScript (vanilla)
• Containerization: Docker & Docker Compose
• Web Server: Nginx (Alpine)

📚 Key Dependencies

fastapi==0.109.0          # Modern Python web framework
anthropic==0.18.1         # Claude API client
chromadb==0.4.24          # Vector database
uvicorn==0.27.0           # ASGI server
numpy==1.26.4             # Numerical computing

🔐 Security Notes

• ⚠️ API key stored in .env (add to .gitignore)
• ⚠️ CORS enabled for development (restrict in production)
• ⚠️ No authentication (add before production deployment)
• ✅ API keys never logged or exposed in responses

🚀 Production Deployment

For production use, consider adding:

• User authentication (JWT, OAuth)
• Rate limiting (Redis)
• HTTPS/SSL (Let's Encrypt)
• Monitoring (Prometheus/Grafana)
• Auto-scaling (Kubernetes)
• Backup strategy (automated)
• Error tracking (Sentry)
• Analytics (Mixpanel, Amplitude)

🤝 Contributing

This is a personal innovation lab project, but suggestions are welcome!

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

📄 License

MIT License - feel free to use for personal or commercial projects.

🙏 Acknowledgments

• Anthropic for Claude AI
• ChromaDB team for the excellent vector database
• FastAPI community for the amazing framework
• Everyone who said "you can't do it" (thanks for the motivation 😉)

📞 Support

Issues? Check the Troubleshooting section first.

Questions? Open an issue or contact via LinkedIn.

---

🎯 Quick Command Reference

# Start all services
docker-compose up -d

# Stop all services
docker-compose down

# View logs
docker-compose logs -f api

# Restart API only
docker-compose restart api

# Rebuild after code changes
docker-compose build api
docker-compose up -d

# Complete reset
docker-compose down -v
docker-compose up -d

# Check service status
docker-compose ps

# Access API container shell
docker exec -it rag-api bash

---

Built with ❤️ and 20+ years of IT experience

2026 Innovation Lab Project

---

🌟 Star This Repo!

If you found this useful, give it a ⭐ on GitHub!

---