Adopt Aspire workflows and parallelize integration tests by ejsmith · Pull Request #2205 · exceptionless/Exceptionless

ejsmith · 2026-04-17T22:11:15Z

Summary

Adopt Aspire-first local and CI workflows while enabling bounded parallel execution for backend integration tests.

What Changed

share Aspire-backed integration test infrastructure across the test process instead of restarting it per class
allocate one AppScope slice per integration test fixture using test, test-1, and so on
reset Elasticsearch, queues, storage, and cache within each fixture scope rather than behind a global lock
make TestServer readiness tracking safe for multiple concurrent hosts without retaining disposed servers
harden baseline and fixture-file tests for parallel execution and line-ending differences
update the AppHost, devcontainer, tasks, and docs to use aspire run as the default local startup path
tighten Docker build and publish behavior, including keeping the all-in-one exceptionless image tag-only
simplify dev environment start/stop retention from hours to days and run scheduled stop checks at 5am America/Chicago
trim and correct README, AGENTS, and repo skill guidance so they match the current Angular/Svelte split and Aspire startup flow

Verification

dotnet test --project "tests\Exceptionless.Tests\Exceptionless.Tests.csproj"
dotnet test --project ".\tests\Exceptionless.Tests\Exceptionless.Tests.csproj" --configuration Release -- --filter-class Exceptionless.Tests.Controllers.OpenApiControllerTests
dotnet test --project ".\tests\Exceptionless.Tests\Exceptionless.Tests.csproj" --no-restore --no-build --configuration Release -- --filter-class Exceptionless.Tests.Controllers.AuthControllerTests
verified aspire run from the repo root starts the app

Notes

the first integration test fixture keeps the plain test scope so single-test runs are easy to inspect
the legacy Angular app in src/Exceptionless.Web/ClientApp.angular still powers the main site while the Svelte 5 app in src/Exceptionless.Web/ClientApp remains under development

Copilot

Pull request overview

This PR enables bounded class-level parallelism for the backend integration test suite by scoping test data per fixture (AppScope) while sharing the Aspire-managed infrastructure across the overall test process.

Changes:

Enable xUnit parallel execution with a fixed MaxParallelThreads value.
Introduce per-fixture AppScope slices (test, test-1, …) while sharing a single Aspire-backed Elasticsearch instance.
Update test helpers/baselines to behave correctly under parallel execution (server readiness tracking, file sharing, OpenAPI baseline comparison).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/Exceptionless.Tests/Utility/StackData.cs	Allow reading stack JSON files without exclusive locks (parallel-safe file access).
tests/Exceptionless.Tests/Properties/PropertyInfo.cs	Enable bounded xUnit class-level parallelism.
tests/Exceptionless.Tests/IntegrationTestsBase.cs	Remove global reset lock; reset data per fixture scope and clear additional queues.
tests/Exceptionless.Tests/Extensions/TestServerExtensions.cs	Make readiness wait tracking per-`TestServer` instead of global.
tests/Exceptionless.Tests/Controllers/OpenApiControllerTests.cs	Compare OpenAPI baseline structurally and attempt to normalize line endings.
tests/Exceptionless.Tests/AppWebHostFactory.cs	Share Aspire infrastructure across the run and allocate/reuse `AppScope` slices per fixture.
AGENTS.md	Document new parallel integration test behavior and `AppScope` slicing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-17T22:15:02Z

@@ -26,9 +46,11 @@ public async ValueTask InitializeAsync()
            .WithContainerName("Exceptionless-Elasticsearch-Test")
            .WithLifetime(ContainerLifetime.Persistent);

-        _app = builder.Build();
+        var app = builder.Build();

-        await _app.StartAsync();
+        await app.StartAsync();
+
+        return app;
    }



The shared Aspire DistributedApplication is started via the static s_sharedApplication but there is no corresponding disposal/shutdown path at the end of the test run. This can leave background resources running longer than necessary and can cause hangs or port contention in some runners/IDE scenarios. Consider adding a single assembly-/collection-level fixture to dispose the shared application once after all tests complete (while still avoiding per-class restarts).

I have not added a one-time teardown for the shared Aspire application in this PR. The current change keeps the shared app alive for the duration of the test process to avoid per-class restarts, so I’m leaving this thread open rather than claiming it is addressed.

Copilot · 2026-04-17T22:15:03Z

+        var oldLoggingLevel = Log.DefaultLogLevel;
+        Log.DefaultLogLevel = LogLevel.Warning;
+


ResetDataAsync temporarily changes Log.DefaultLogLevel, which is a static/global setting. With class-level parallelism enabled, concurrent fixtures can interleave these assignments and restores, leading to the wrong log level being applied in other tests (and potentially affecting log-based assertions). Consider avoiding global log-level mutation here (e.g., adjust only specific categories) or guard the change/restore with a dedicated lock around just the log-level section.

I have not changed the Log.DefaultLogLevel mutation path yet. This thread is still open intentionally because the global log-level race concern is valid and still needs a separate follow-up decision.

ejsmith · 2026-04-18T22:15:27Z

+    public AppWebHostFactory()
+    {
+        if (!s_pool.TryDequeue(out var instanceId))
+            instanceId = Interlocked.Increment(ref s_counter);


I did not take this refactor-only suggestion in this PR. The constructor logic is still the same and the thread is left open intentionally rather than being resolved as addressed.

ejsmith · 2026-04-18T22:15:27Z

+            catch (HttpRequestException)
+            {
+            }


I did not change this retry loop yet. I’m leaving the thread open intentionally rather than marking it addressed, although I agree that attaching the last exception would improve timeout diagnostics.

ejsmith · 2026-04-18T22:15:27Z

+            catch (TaskCanceledException)
+            {
+            }


I did not apply this comment-only suppression change in this PR. This thread is still open intentionally so it is clear the empty-catch suggestion has not been taken yet.

github-actions · 2026-04-18T22:10:50Z

Package	Line Rate	Branch Rate	Complexity	Health
Exceptionless.Core	66%	60%	7629	➖
Exceptionless.Insulation	25%	23%	203	❌
Exceptionless.Web	57%	43%	3643	➖
Exceptionless.AppHost	24%	11%	69	❌
Summary	61% (11920 / 19472)	54% (5961 / 11098)	11544	➖

ejsmith · 2026-04-18T22:12:52Z

Follow-up on the review feedback:

Addressed in 9c7c6cb2b:

tests/Exceptionless.Tests/Controllers/OpenApiControllerTests.cs: normalized both physical line endings and escaped \\r\\n sequences so the OpenAPI baseline comparison is stable across formatting differences.
tests/Exceptionless.Tests/Extensions/TestServerExtensions.cs: switched readiness tracking to ConditionalWeakTable<TestServer, object> so disposed servers are not retained.

Verification:

dotnet test --project ".\\tests\\Exceptionless.Tests\\Exceptionless.Tests.csproj" --configuration Release -- --filter-class Exceptionless.Tests.Controllers.OpenApiControllerTests
dotnet test --project ".\\tests\\Exceptionless.Tests\\Exceptionless.Tests.csproj" --no-restore --no-build --configuration Release -- --filter-class Exceptionless.Tests.Controllers.AuthControllerTests

The matching OpenAPI and TestServer review threads are resolved.

Still not addressed:

the shared Aspire application teardown comment in AppWebHostFactory
the global Log.DefaultLogLevel mutation comment in IntegrationTestsBase
the remaining static-analysis suggestions in AppWebHostFactory around static state allocation and empty catch blocks

Those are still open on purpose rather than being silently ignored.

niemyjski · 2026-04-19T03:08:00Z

+            .ReplaceLineEndings("\n")
+            .Replace("\\r\\n", "\\n");
+        actualJson = actualJson
+            .ReplaceLineEndings("\n")
+            .Replace("\\r\\n", "\\n");


is this really needed? should we be using Environment.NewLine so it's less magic strings?

niemyjski · 2026-04-19T03:08:16Z

    {
        var startupContext = server.Services.GetService<StartupActionsContext>();
-        var maxWaitTime = !_alreadyWaited ? TimeSpan.FromSeconds(30) : TimeSpan.FromSeconds(2);
+        var maxWaitTime = !s_waitedServers.TryGetValue(server, out _) ? TimeSpan.FromSeconds(30) : TimeSpan.FromSeconds(2);


really wish we got rid of the s_

niemyjski · 2026-04-19T03:08:45Z

 public class AppWebHostFactory : WebApplicationFactory<Startup>, IAsyncLifetime
 {
-    private DistributedApplication? _app;
+    private static readonly Uri s_elasticsearchUri = new("http://127.0.0.1:9200");


why hard code this, can we also get rid of s_

niemyjski · 2026-04-19T03:09:13Z

 public class AppWebHostFactory : WebApplicationFactory<Startup>, IAsyncLifetime
 {
-    private DistributedApplication? _app;
+    private static readonly Uri s_elasticsearchUri = new("http://127.0.0.1:9200");


also shouldn't this come from configuration?

niemyjski · 2026-04-19T03:11:09Z

+FROM build AS build-node
+RUN apt-get update -yq \
+    && apt-get install -yq curl ca-certificates \
+    && curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
+    && apt-get install -yq nodejs \
+    && rm -rf /var/lib/apt/lists/*


I never understood why we didn't do this first, so this layer could be cached more.

Enable parallel integration test slices

b9cc7aa

Copilot AI review requested due to automatic review settings April 17, 2026 22:11

Copilot started reviewing on behalf of ejsmith April 17, 2026 22:11 View session

github-code-quality bot found potential problems Apr 17, 2026

View reviewed changes

Comment thread tests/Exceptionless.Tests/AppWebHostFactory.cs Fixed

Copilot AI reviewed Apr 17, 2026

View reviewed changes

ejsmith added 13 commits April 17, 2026 21:00

Fix Slack service test infrastructure

8c4808e

Optimize CI docker build path

3b56afc

Fix CI solution selection

46ea4c1

Remove Docker-only solution

a73c3d6

Fix app image publish cache reuse

c8325f2

Fix Docker compose readiness checks

bc96123

Use Aspire CLI for CI services

45d2e8e

Use Aspire CLI for local and CI infrastructure

b5bbc43

Add explicit publish timing job

eb55f4e

Run docker publish in pull requests

374df03

Install Aspire extension in devcontainers

5a7517f

Remove redundant Aspire startup from test-api

6174d9a

Wait for Elasticsearch in test app host

9d48fbb

github-code-quality bot found potential problems Apr 18, 2026

View reviewed changes

ejsmith added 11 commits April 18, 2026 01:11

Stage docker publish images for pull requests

1f80320

Gate docker publish to deploy branches

a2dc84e

Trigger CI after DEV_DEPLOY_BRANCH update

4198d45

Build all-in-one image only for tags

977149e

Scope test-api to backend test project

c78dd4a

Reduce dev environment workflow noise

cf18f2f

Schedule dev stop at 5am central

c922cf9

Trim onboarding docs

701c88e

Prefer aspire run in docs

6420a90

Remove API-only docs section

9a1dd21

Remove redundant startup warning

bc9d4a7

ejsmith added 2 commits April 18, 2026 16:41

Clarify Angular and Svelte UI docs

292f192

Fix test baseline normalization and server tracking

9c7c6cb

ejsmith requested a review from niemyjski April 18, 2026 22:09

ejsmith changed the title ~~Enable parallel integration test slices~~ Adopt Aspire workflows and parallelize integration tests Apr 18, 2026

niemyjski reviewed Apr 19, 2026

View reviewed changes

		var oldLoggingLevel = Log.DefaultLogLevel;
		Log.DefaultLogLevel = LogLevel.Warning;

Uh oh!

Conversation

ejsmith commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Changed

Verification

Notes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 18, 2026

Uh oh!

ejsmith commented Apr 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

ejsmith commented Apr 17, 2026 •

edited

Loading