Adopt Aspire workflows and parallelize integration tests by ejsmith · Pull Request #2205 · exceptionless/Exceptionless

ejsmith · 2026-04-17T22:11:15Z

Summary

Adopt Aspire-first local and CI workflows, parallelize backend integration tests, and add a sample event generator for new accounts.

What Changed

Test infrastructure

share Aspire-backed integration test infrastructure across the test process instead of restarting it per class
allocate one AppScope slice per integration test fixture using test, test-1, and so on (the first fixture keeps the plain test scope so single-test runs stay easy to inspect)
reset Elasticsearch, queues, storage, and cache within each fixture scope rather than behind a global lock
make TestServer readiness tracking safe for multiple concurrent hosts without retaining disposed servers
harden baseline and fixture-file tests for parallel execution and line-ending differences

Aspire and local dev

update the AppHost, devcontainer, tasks, and docs to use aspire run as the default local startup path
wire the legacy Angular and new Svelte UIs into Aspire with developer-cert support and a shared API_HTTPS/API_HTTP proxy convention
replace ASPNETCORE_URLS env wiring in connect.js / vite.config.ts with the values Aspire injects

CI/CD workflow

consolidate gating logic into version job outputs (should_publish, is_prod_deploy, is_dev_deploy) so publish/deploy jobs are skipped cleanly on forks and PRs
keep the all-in-one exceptionless Docker image tag-only; standard PRs build api, job, and app
simplify dev environment retention from hours to days and run scheduled stop checks at 5am America/Chicago

Frontend

consolidate ClientApp dependencies into devDependencies (SvelteKit static SPA — nothing is consumed at runtime)
simplify vite.config.ts and fix dev:api to use API_HTTPS instead of the no-op ASPNETCORE_URLS
minor footer/layout polish

Sample data

new admin endpoint plus GenerateSampleEventsWorkItem / RandomEventGenerator to seed realistic events for fresh accounts

Docs

trim and correct README, AGENTS, and repo skill guidance to match the current Angular/Svelte split and Aspire startup flow

Verification

dotnet test
dotnet test -- --filter-class Exceptionless.Tests.Controllers.OpenApiControllerTests
dotnet test -- --filter-class Exceptionless.Tests.Controllers.AuthControllerTests
verified aspire run from the repo root starts the app

Notes

the legacy Angular app in src/Exceptionless.Web/ClientApp.angular still powers the main site while the Svelte 5 app in src/Exceptionless.Web/ClientApp remains under development

Copilot

Pull request overview

This PR enables bounded class-level parallelism for the backend integration test suite by scoping test data per fixture (AppScope) while sharing the Aspire-managed infrastructure across the overall test process.

Changes:

Enable xUnit parallel execution with a fixed MaxParallelThreads value.
Introduce per-fixture AppScope slices (test, test-1, …) while sharing a single Aspire-backed Elasticsearch instance.
Update test helpers/baselines to behave correctly under parallel execution (server readiness tracking, file sharing, OpenAPI baseline comparison).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/Exceptionless.Tests/Utility/StackData.cs	Allow reading stack JSON files without exclusive locks (parallel-safe file access).
tests/Exceptionless.Tests/Properties/PropertyInfo.cs	Enable bounded xUnit class-level parallelism.
tests/Exceptionless.Tests/IntegrationTestsBase.cs	Remove global reset lock; reset data per fixture scope and clear additional queues.
tests/Exceptionless.Tests/Extensions/TestServerExtensions.cs	Make readiness wait tracking per-`TestServer` instead of global.
tests/Exceptionless.Tests/Controllers/OpenApiControllerTests.cs	Compare OpenAPI baseline structurally and attempt to normalize line endings.
tests/Exceptionless.Tests/AppWebHostFactory.cs	Share Aspire infrastructure across the run and allocate/reuse `AppScope` slices per fixture.
AGENTS.md	Document new parallel integration test behavior and `AppScope` slicing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ejsmith · 2026-04-18T22:12:52Z

Follow-up on the review feedback:

Addressed in 9c7c6cb2b:

tests/Exceptionless.Tests/Controllers/OpenApiControllerTests.cs: normalized both physical line endings and escaped \\r\\n sequences so the OpenAPI baseline comparison is stable across formatting differences.
tests/Exceptionless.Tests/Extensions/TestServerExtensions.cs: switched readiness tracking to ConditionalWeakTable<TestServer, object> so disposed servers are not retained.

Verification:

dotnet test --project ".\\tests\\Exceptionless.Tests\\Exceptionless.Tests.csproj" --configuration Release -- --filter-class Exceptionless.Tests.Controllers.OpenApiControllerTests
dotnet test --project ".\\tests\\Exceptionless.Tests\\Exceptionless.Tests.csproj" --no-restore --no-build --configuration Release -- --filter-class Exceptionless.Tests.Controllers.AuthControllerTests

The matching OpenAPI and TestServer review threads are resolved.

Still not addressed:

the shared Aspire application teardown comment in AppWebHostFactory
the global Log.DefaultLogLevel mutation comment in IntegrationTestsBase
the remaining static-analysis suggestions in AppWebHostFactory around static state allocation and empty catch blocks

Those are still open on purpose rather than being silently ignored.

niemyjski · 2026-04-19T03:08:00Z

+            .ReplaceLineEndings("\n")
+            .Replace("\\r\\n", "\\n");
+        actualJson = actualJson
+            .ReplaceLineEndings("\n")
+            .Replace("\\r\\n", "\\n");


is this really needed? should we be using Environment.NewLine so it's less magic strings?

Removed the ReplaceLineEndings calls and added tests/Exceptionless.Tests/Controllers/Data/openapi.json text eol=lf to .gitattributes so the baseline is always LF on disk. Kept the \r\n → \n replace because those are escape sequences embedded inside JSON string values (XML doc comments processed on Windows get serialized as "...\r\n..."), not on-disk line endings — Environment.NewLine would be CRLF on Windows runners and break the comparison there.

niemyjski · 2026-04-19T03:08:16Z

    {
        var startupContext = server.Services.GetService<StartupActionsContext>();
-        var maxWaitTime = !_alreadyWaited ? TimeSpan.FromSeconds(30) : TimeSpan.FromSeconds(2);
+        var maxWaitTime = !s_waitedServers.TryGetValue(server, out _) ? TimeSpan.FromSeconds(30) : TimeSpan.FromSeconds(2);


really wish we got rid of the s_

Done — dropped the ConditionalWeakTable cache and both static fields entirely. The cache only narrowed the failure timeout (30s → 2s) on warm servers; on the success path the first /ready call returns immediately anyway, so the optimization wasn't pulling its weight. Now there's just a single 30s timeout (already covers cold starts and slow CI).

niemyjski · 2026-04-19T03:08:45Z

 public class AppWebHostFactory : WebApplicationFactory<Startup>, IAsyncLifetime
 {
-    private DistributedApplication? _app;
+    private static readonly Uri s_elasticsearchUri = new("http://127.0.0.1:9200");


why hard code this, can we also get rid of s_

Removed the hardcoded URL and the s_ field. The Elasticsearch URL is now resolved from the running Aspire app via elasticsearch.Resource.GetConnectionStringAsync() after app.StartAsync(), then passed into WaitForElasticsearchAsync as a parameter. Single source of truth — the port (9200) is declared once on AddElasticsearch(...) and Aspire owns the rest.

niemyjski · 2026-04-19T03:09:13Z

 public class AppWebHostFactory : WebApplicationFactory<Startup>, IAsyncLifetime
 {
-    private DistributedApplication? _app;
+    private static readonly Uri s_elasticsearchUri = new("http://127.0.0.1:9200");


also shouldn't this come from configuration?

The connection string is now derived from the Aspire ElasticsearchResource itself (see reply above), so it's no longer hardcoded — the test AppHost is the configuration. I'd rather not pull it from appsettings here because then we'd have two places declaring the test ES endpoint that could drift; AddElasticsearch("Elasticsearch", port: 9200) is the single declaration and GetConnectionStringAsync reads it back.

niemyjski · 2026-04-19T03:11:09Z

+FROM build AS build-node
+RUN apt-get update -yq \
+    && apt-get install -yq curl ca-certificates \
+    && curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
+    && apt-get install -yq nodejs \
+    && rm -rf /var/lib/apt/lists/*


I never understood why we didn't do this first, so this layer could be cached more.

Yeah, agreed — the reorder lets Docker cache the dotnet restore layer across source-only changes, which should noticeably speed up CI builds when only .cs files change.

…test-slices

Co-authored-by: Copilot <copilot@github.com>

+    public AppWebHostFactory()
+    {
+        if (!s_pool.TryDequeue(out var instanceId))
+            instanceId = Interlocked.Increment(ref s_counter);


+            catch (HttpRequestException)
+            {
+            }


+            catch (TaskCanceledException)
+            {
+            }


Co-authored-by: Copilot <copilot@github.com>

…/exceptionless/Exceptionless into parallel-integration-test-slices

github-actions · 2026-04-26T04:05:30Z

Package	Line Rate	Branch Rate	Complexity	Health
Exceptionless.Core	65%	59%	7704	➖
Exceptionless.Insulation	25%	23%	203	❌
Exceptionless.Web	57%	43%	3651	➖
Exceptionless.AppHost	19%	9%	84	❌
Summary	60% (11924 / 19794)	53% (5956 / 11192)	11642	➖

Enable parallel integration test slices

b9cc7aa

Copilot AI review requested due to automatic review settings April 17, 2026 22:11

Copilot started reviewing on behalf of ejsmith April 17, 2026 22:11 View session

github-code-quality Bot found potential problems Apr 17, 2026

View reviewed changes

Comment thread tests/Exceptionless.Tests/AppWebHostFactory.cs Fixed

Copilot AI reviewed Apr 17, 2026

View reviewed changes

ejsmith added 13 commits April 17, 2026 21:00

Fix Slack service test infrastructure

8c4808e

Optimize CI docker build path

3b56afc

Fix CI solution selection

46ea4c1

Remove Docker-only solution

a73c3d6

Fix app image publish cache reuse

c8325f2

Fix Docker compose readiness checks

bc96123

Use Aspire CLI for CI services

45d2e8e

Use Aspire CLI for local and CI infrastructure

b5bbc43

Add explicit publish timing job

eb55f4e

Run docker publish in pull requests

374df03

Install Aspire extension in devcontainers

5a7517f

Remove redundant Aspire startup from test-api

6174d9a

Wait for Elasticsearch in test app host

9d48fbb

github-code-quality Bot found potential problems Apr 18, 2026

View reviewed changes

Comment thread tests/Exceptionless.Tests/AppWebHostFactory.cs Fixed

Comment thread tests/Exceptionless.Tests/AppWebHostFactory.cs Fixed

Comment thread tests/Exceptionless.Tests/AppWebHostFactory.cs Fixed

ejsmith added 11 commits April 18, 2026 01:11

Stage docker publish images for pull requests

1f80320

Gate docker publish to deploy branches

a2dc84e

Trigger CI after DEV_DEPLOY_BRANCH update

4198d45

Build all-in-one image only for tags

977149e

Scope test-api to backend test project

c78dd4a

Reduce dev environment workflow noise

cf18f2f

Schedule dev stop at 5am central

c922cf9

Trim onboarding docs

701c88e

Prefer aspire run in docs

6420a90

Remove API-only docs section

9a1dd21

Remove redundant startup warning

bc9d4a7

ejsmith added 2 commits April 18, 2026 16:41

Clarify Angular and Svelte UI docs

292f192

Fix test baseline normalization and server tracking

9c7c6cb

ejsmith requested a review from niemyjski April 18, 2026 22:09

ejsmith changed the title ~~Enable parallel integration test slices~~ Adopt Aspire workflows and parallelize integration tests Apr 18, 2026

niemyjski reviewed Apr 19, 2026

View reviewed changes

ejsmith and others added 7 commits April 24, 2026 22:53

Merge remote-tracking branch 'origin/main' into parallel-integration-…

ba20162

…test-slices

Some updates

13ce8ea

Add sample events

2c7f582

Fix vite logging and update footer

37722eb

Co-authored-by: Copilot <copilot@github.com>

Fix lint

3588e66

Feedback

d9571db

Merge branch 'main' into parallel-integration-test-slices

620b423

github-code-quality Bot found potential problems Apr 26, 2026

View reviewed changes

ejsmith and others added 2 commits April 25, 2026 22:59

Change ports to a 71XX scheme, fix devcontainers

3fde83a

Co-authored-by: Copilot <copilot@github.com>

Merge branch 'parallel-integration-test-slices' of https://github.com…

72281f6

…/exceptionless/Exceptionless into parallel-integration-test-slices

Uh oh!

Conversation

ejsmith commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Changed

Test infrastructure

Aspire and local dev

CI/CD workflow

Frontend

Sample data

Docs

Verification

Notes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ejsmith commented Apr 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

ejsmith commented Apr 17, 2026 •

edited

Loading