Medical Q&A Assistant

A ReAct (Reasoning and Acting) AI agent designed to provide reliable, source-backed medical information. The agent uses a combination of local vector search (FAISS) and external APIs (OpenFDA) to answer health-related queries safely.

⚠️ DISCLAIMER This project is for informational and educational purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of your physician or other qualified health provider with any questions you may have regarding a medical condition.

Architecture

flowchart TD
    User([User Query]) --> API[FastAPI Endpoint]
    API --> PreFilter{Emergency<br>Keywords?}
    
    PreFilter -- Yes --> EmergTool[emergency_check Tool]
    EmergTool --> Output
    
    PreFilter -- No --> ReAct[ReAct Core Loop]
    
    subgraph LLM Backend
        ReAct --> PrimaryLLM[Primary LLM Provider]
        PrimaryLLM -- Fallback --> BackupLLM[Secondary LLM Provider]
        BackupLLM -- Fallback --> TertiaryLLM[Tertiary LLM Provider]
    end
    
    subgraph Tools
        ReAct <--> FAISS[(FAISS Index<br>~25ms Retrieval)]
        ReAct <--> FDA[OpenFDA API]
        ReAct <--> Symptom[(Symptom DB)]
    end
    
    ReAct --> Output([Final Answer +<br>Disclaimer])

Loading

The assistant uses a standard ReAct loop backed by a multi-provider cascading failover architecture. It interfaces with four specialized tools:

1. search_medical_docs: Uses a local FAISS vector index to retrieve chunks of MedlinePlus medical articles, which are then re-ranked using a Cross-Encoder for high precision.
2. check_drug_interactions: Queries the live OpenFDA API to check for contraindications and interactions between medications.
3. lookup_symptoms: Maps described symptoms to potential conditions using a curated local JSON database.
4. emergency_check: A deterministic safety pre-filter that triggers an immediate warning if emergency keywords (e.g., "chest pain", "can't breathe") are detected in the query, bypassing LLM routing.

Tech Stack

• Backend: Python, FastAPI
• AI/Agent: Groq API / Google Gemini API
• Embeddings/Search: sentence-transformers (pritamdeka/S-PubMedBert-MS-MARCO), cross-encoder/ms-marco-MiniLM-L-6-v2, faiss-cpu
• Frontend: Vanilla HTML/JS/CSS with Server-Sent Events (SSE) for streaming thoughts and actions.

Setup & Installation

1. Clone the repository (if not already done).

2. Install dependencies:

   pip install -r requirements.txt

3. Configure Environment Variables: Copy .env.example to .env and add your API keys:

   cp .env.example .env
   # Open .env and add your GROQ_API_KEY or GEMINI_API_KEY

4. Build the Local Medical Knowledge Base: The agent relies on a local FAISS index for document retrieval. Generate it by running the data pipeline scripts in order:

   # 1. Fetch raw data from MedlinePlus
   python scripts/fetch_medlineplus.py
   
   # 2. Chunk the documents
   python scripts/chunk_documents.py
   
   # 3. Embed chunks and build the FAISS index
   python scripts/build_index.py
   
   # 4. Ingest Kaggle Disease-Symptom dataset
   python scripts/ingest_curated_dataset.py
   
   # 5. Build the Symptom FAISS index
   python scripts/build_symptom_index.py

How to Run

1. Start the API Server:

   python main.py

   The server will start on http://0.0.0.0:8000.

2. Open the Frontend: Simply open index.html in your web browser.

3. Ask Questions: Try queries like:

   • "What are the symptoms of diabetes?"
   • "Is it safe to take ibuprofen with metformin?"
   • "I have a headache, nausea, and light sensitivity."
   • "I am experiencing severe chest pain." (Tests the emergency safety tool)

Evaluation

A comprehensive 20-case automated evaluation suite validates the agent's tool-routing accuracy, safety guardrail adherence, and disclaimer compliance. It also utilizes an LLM-as-a-judge approach to score Faithfulness (ensuring the agent does not hallucinate beyond the retrieved context).

Make sure the server is running, then run the evaluation:

python evaluation/run_evaluation.py --verbose --record

Metrics Tracked:

• Faithfulness: % of answers strictly supported by retrieved context
• Safety Guardrails: % adherence to emergency and injection bypass protocols
• Disclaimer Compliance: % of responses including required medical disclaimers
• Tool Routing: % accuracy of tool selection for various query types

Results are saved to evaluation/regression_history.json to monitor improvements or regressions over time.