Skip to content

Fix Ollama AppHostTests race: wait for healthy, not just Running - #1606

Open
afscrome wants to merge 2 commits into
mainfrom
afscrome-fix-ollama-health-race
Open

afscrome wants to merge 2 commits into
mainfrom
afscrome-fix-ollama-health-race

Conversation

@afscrome

Copy link
Copy Markdown
Contributor

The race condition

AppHostTests.OllamaResourcesStartAndRespondOk waited for the ollama/ollama2 resources to reach the Running state, then immediately issued an HTTP GET / against them:

await Task.WhenAll([
    .. ollamaResources.Select(o => rns.WaitForResourceAsync(o.Name, KnownResourceStates.Running))
]).WaitAsync(TimeSpan.FromMinutes(5));

foreach (var ollama in ollamaResources)
{
    using var httpClient = fixture.CreateHttpClient(ollama.Name);
    var response = await httpClient.GetAsync("/");
    Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}

Running only means the container/executable process has started — it does not mean the Ollama server inside is actually listening and accepting HTTP connections yet. On a slow or contended CI runner, the GET / can race ahead of the server actually being ready, causing intermittent connection refused/non-200 failures. This matches the resource-health guidance in OllamaResourcesListAvailableModels in the same file, which already correctly uses WaitForResourceHealthyAsync instead.

The fix

Use ResourceNotificationService.WaitForResourceHealthyAsync instead of WaitForResourceAsync(..., KnownResourceStates.Running), mirroring the pattern already used elsewhere in the same test class.

Why no change was needed to OllamaModelResource/AddModel

I initially explored adding an explicit wait dependency (WaitFor/WaitAnnotation) from OllamaModelResource onto its parent Ollama resource, and separately looked at catching the pre-download 404 in OllamaModelHealthCheck. Neither is necessary:

  • OllamaModelResource already implements IResourceWithParent<IOllamaResource>, and it isn't a container/executable/project/parameter/connection-string resource. Aspire's ApplicationOrchestrator.ResourceHasOwnLifetime/SetChildResourceAsync explicitly treats such "no owned lifetime" IResourceWithParent resources by propagating the parent's lifecycle state to them automatically, so no extra wiring is required or appropriate.
  • A pre-download 404 from OllamaSharp.ShowModelAsync inside OllamaModelHealthCheck is already converted by Microsoft.Extensions.Diagnostics.HealthChecks.DefaultHealthCheckService into an Unhealthy result internally (it just logs a warning first), so the standard Aspire health-check retry/WaitForHealthy polling already handles it correctly today.

Validation

  • dotnet build on the hosting project and test project: succeeds.
  • Non-Docker unit tests in CommunityToolkit.Aspire.Hosting.Ollama.Tests (--filter-not-class "*AppHostTests*"): 69/69 passed.
  • [RequiresDocker] AppHostTests (the class containing the fixed test): I was not able to get a clean run in this sandboxtests-app-hosts/Ollama.AppHost starts one resource via AddOllamaLocal (a local executable resource requiring the ollama CLI binary on PATH), which is not installed in this environment, so that resource fails to start regardless of this change. This is a pre-existing environment limitation, not a regression. The change itself is a one-line swap to an existing, already-used API (WaitForResourceHealthyAsync), so risk is low.
  • TypeScriptAppHostTests was not run locally (requires the aspire CLI + Node/npm driving a real multi-container/model-pull scenario; out of scope for this fix and not verified here).

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

OllamaResourcesStartAndRespondOk waited for the ollama/ollama2 resources
to reach the "Running" state before issuing an HTTP GET against them.
"Running" only means the container/executable process has started; it
does not mean the Ollama server inside is actually accepting
connections yet. This is a genuine race: on a slow/contended CI runner
the request can be issued before the server is listening, causing
intermittent connection-refused failures.

OllamaModelResource does not need any additional wait wiring for this:
it implements IResourceWithParent<IOllamaResource> and has no
"resource with its own lifetime" markers (it isn't a container,
executable, project, etc.), so Aspire's ApplicationOrchestrator already
propagates the parent's state/lifecycle to it automatically
(see ResourceHasOwnLifetime/SetChildResourceAsync). No custom
WaitAnnotation or WaitFor() is required or appropriate here.

Fix: use ResourceNotificationService.WaitForResourceHealthyAsync
instead of WaitForResourceAsync(..., KnownResourceStates.Running), the
same pattern already used correctly by OllamaResourcesListAvailableModels.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 22, 2026 16:52
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/CommunityToolkit/Aspire/main/eng/scripts/dogfood-pr.sh | bash -s -- 1606

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/CommunityToolkit/Aspire/main/eng/scripts/dogfood-pr.ps1) } 1606"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The readiness race is addressed with a focused, low-risk change.

Review effort: Lite
Findings: None

What changed in this PR

Fixes an Ollama AppHost test race by waiting for resource health before sending HTTP requests.

Changes:

  • Replaces the Running state wait with WaitForResourceHealthyAsync.
  • Aligns the test with the existing health-readiness pattern.
File Description
tests/​CommunityToolkit.Aspire.Hosting.Ollama.Tests/​AppHostTests.cs Waits for Ollama health before testing HTTP availability.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@afscrome
afscrome deployed to azure-artifacts September 22, 2026 16:59 — with GitHub Actions Active
@afscrome
afscrome enabled auto-merge (squash) September 22, 2026 19:46
@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Branch Rate Complexity Health
CommunityToolkit.Aspire.Hosting.Ollama 77% 67% 265
Summary 77% (640 / 829) 67% (164 / 246) 265

@afscrome
afscrome deployed to azure-artifacts September 23, 2026 18:03 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
azure-artifacts 8c065d2b Deployed Sep 23, 2026 by afscrome via publish-azure-artifacts #3703
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants