Skip to content

Add integration tests and CI workflow#11

Draft
Copilot wants to merge 2 commits into
mainfrom
copilot/add-unit-tests-ci
Draft

Add integration tests and CI workflow#11
Copilot wants to merge 2 commits into
mainfrom
copilot/add-unit-tests-ci

Conversation

Copy link
Copy Markdown

Copilot AI commented Feb 26, 2026

No CI existed to validate the RAG pipeline end-to-end across code check-ins. This adds a full integration test suite wired to GitHub Actions, covering Cosmos DB retrieval, Azure OpenAI endpoints, and the decomposed RAG pipeline.

New files

  • tests/test_questions.json – Sample Mars-themed questions driving the RAG pipeline tests

  • tests/conftest.py – Module-scoped fixtures (event_loop, config, llm_client, retriever); all tests auto-skip when config.yaml is absent

  • tests/test_integration.py – 15 integration tests across 5 classes:

    Class Coverage
    TestEmbeddingEndpoint Non-empty float vectors, correct dimension
    TestLLMEndpoint Non-empty, coherent LLM response
    TestVectorSearch Non-null results per configured source
    TestFullTextSearch Non-null results for sources with fulltext_k > 0
    TestRAGPipeline initial_answer, rounds (reasoning traces), non-empty final_answer for all test questions
  • scripts/generate_config.py – Writes config.yaml from config.yaml.example by substituting environment variables sourced from GitHub Secrets; avoids committing credentials

  • .github/workflows/ci.yml – Runs on every PR and push to main: installs deps, generates config.yaml from secrets, runs pytest tests/ -v --tb=short; GITHUB_TOKEN scoped to contents: read

Required GitHub Secrets

Secret Purpose
COSMOS_URI / COSMOS_KEY Cosmos DB account
COSMOS_DATABASE_NAME Database (default: divdet)
COSMOS_SOURCE1_CONTAINER / COSMOS_SOURCE2_CONTAINER Container names
LLM_ENDPOINT / LLM_API_KEY / LLM_MODEL Azure OpenAI LLM
EMBED_ENDPOINT / EMBED_API_KEY / EMBED_MODEL / EMBED_DIMENSIONS Azure OpenAI embeddings
Original prompt

This section details on the original issue you should resolve

<issue_title>Add unit tests + CI</issue_title>
<issue_description>We need to add unit tests to ensure this code is working end to end between code check ins. The unit tests should be run against every PR.

Guidelines for copilot

They tests should utilize the following assets:

  • 2 small prebuilt Cosmos DB collections under 1 account with full text and vector index. We will store its details and credentials in github secrets.
  • A seperate Azure OpenAI LLM and inference endpoint with moderate rate limit. The details and credentials will be in github secrets.
  • Th CI test should securely generate config.yaml from config.yaml.example by inserting GH secrets (if this is advisable, if not advise on best practice)
  • Use sample questions stored in a test_file in the repo

The tests should check that:

  • vector search and full text search to both cosmos DB accounts shoudl work and give non-null results
  • that the LLM and embedding endpoints are functional.
  • python rag_divdet.py with the test questions should generate reasoning traces and final answers </issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: harsha-simhadri <5590673+harsha-simhadri@users.noreply.github.com>
Copilot AI changed the title [WIP] Add unit tests and CI configuration Add integration tests and CI workflow Feb 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add unit tests + CI

2 participants