Skip to content

Add Search API with OpenSearch Domain and VPC Resources#1219

Merged
isabeleliassen merged 137 commits intocsg-org:mainfrom
InspiringApps:feat/add-vpc
Dec 29, 2025
Merged

Add Search API with OpenSearch Domain and VPC Resources#1219
isabeleliassen merged 137 commits intocsg-org:mainfrom
InspiringApps:feat/add-vpc

Conversation

@landonshumway-ia
Copy link
Copy Markdown
Collaborator

@landonshumway-ia landonshumway-ia commented Nov 20, 2025

To support improved search capability, this project will be leveraging AWS OpenSearch Service. Provider data from our DynamoDB table will be indexed within an OpenSearch Domain (Cluster), which will then be queryable by staff users through the Compact Connect UI. The OpenSearch resources will be deployed within a Virtual Private Cloud (VPC) to provide a layer of network security. This PR adds a new VPC CDK stack which deploys the VPC and the necessary subnets, VPC endpoints (to allow lambdas to log to CloudWatch and query DynamoDB), and security groups which will be used by the OpenSearch domain and lambdas.

It also adds a SearchPersistentStack which holds the OpenSearch domain, associated lambda resources, and event bridge pipeline for ingesting record updates from DynamoDB into the OpenSearch domain.

Finally, it adds a new SearchAPIStack which holds a new API Gateway instance to allow the frontend to query the OpenSearch domain with authentication and security controls in place.

For more information about the search functionality, review the README documentation updates.

Requirements List

  • Backwards compatible change ✅ this new stack does not change existing APIs
  • NOTE: This will require redeployment of our bootstrap stack to add permissions to our pipeline roles to deploy VPC and OpenSearch resources

Description List

  • Added VpcStack in CDK with VPC resources
  • Added SearchPersistentStack in CDK with OpenSearch Domain and related lambdas
  • Added SearchAPIStack in CDK with Search API Gateway resources
  • Added CDK tests to verify configuration settings

Closes #1206 #1207 #1208 #1209 #1210

Summary by CodeRabbit

  • New Features

    • VPC-backed search platform: private VPC, encrypted OpenSearch domain, index manager, paginated provider search API, CSV privilege export with presigned URLs, resumable initial population, queued real‑time ingest pipeline with DLQ and monitoring.
  • Documentation

    • Added Search OpenAPI spec and Swagger UI; design docs and onboarding guidance updated (removed beta references).
  • Tests

    • Extensive new unit and integration tests covering VPC, OpenSearch, indexing, ingest, search, and export flows.
  • Chores

    • Dependency/version updates and CLI/tools extended to support Search workflows.

✏️ Tip: You can customize this high-level summary in your review settings.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

improved search/report BE - Add VPC stack with needed VPC configuration

4 participants