Skip to content

[tests] Replace make-based packaged macOS test runner with C# script#25749

Draft
rolfbjarne wants to merge 6 commits into
mainfrom
dev/rolf/improved-macos-test-runner
Draft

[tests] Replace make-based packaged macOS test runner with C# script#25749
rolfbjarne wants to merge 6 commits into
mainfrom
dev/rolf/improved-macos-test-runner

Conversation

@rolfbjarne

@rolfbjarne rolfbjarne commented Jun 19, 2026

Copy link
Copy Markdown
Member

Replace the Makefile-based test runner (packaged-macos-tests.mk + run-with-timeout + mac-test-report-generator) with a single C# script project (scripts/run-packaged-macos-tests) that:

  • Executes pre-built macOS and Mac Catalyst test apps with timeout and launch retry logic
  • Generates an HTML report with per-platform/architecture breakdown tables showing output links and [FAIL] lines inline
  • Generates TestSummary.md for GitHub PR comments
  • Handles ARM64/x64 architecture detection and skipping
  • Collects and links crash reports

Simplifies run_mac_tests.ps1 from ~235 lines to ~150 lines by delegating all test execution and reporting to the new C# tool.

🤖 Pull request created by Copilot

rolfbjarne and others added 5 commits June 19, 2026 13:03
@rolfbjarne
Replace packaged macOS test runner with C# script
ea79841 
Replace the Makefile-based test runner (packaged-macos-tests.mk + run-with-timeout
+ mac-test-report-generator) with a single C# script project
(scripts/run-packaged-macos-tests) that:

- Executes pre-built macOS and Mac Catalyst test apps with timeout and
  launch retry logic (previously in run-with-timeout)
- Generates an HTML report with per-platform/architecture breakdown tables
  (previously in mac-test-report-generator)
- Generates TestSummary.md for GitHub PR comments
- Handles ARM64/x64 architecture detection and skipping
- Reports crash reports

Simplify run_mac_tests.ps1 to delegate all test execution and reporting
to the new C# tool, and update package-mac-tests.sh to include it in the
test package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rolfbjarne
Fix launch arguments: use for Mac Catalyst only, not macOS
63a7071 
The --autostart --autoexit arguments are only passed to Mac Catalyst
test apps, matching the behavior in packaged-macos-tests.mk.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rolfbjarne
Remove stale sentinel file
c4c874a 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rolfbjarne
Merge stdout+stderr into single output file, improve HTML report
ff58ccc 
- Combine stdout and stderr into a single output file per test config
- Add output file link directly in the table's Output column
- Show [FAIL] lines immediately after each failing test row in the
  table instead of in a separate list after the table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rolfbjarne
Show entire [FAIL] line including the [FAIL] prefix
1af29c6 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

✅ [PR Build #f7220b4] Build passed (Build packages) ✅

Pipeline on Agent
Hash: f7220b4438eed86cd475a7db87d5046c76c170f9 [PR build]

@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

✅ [PR Build #f7220b4] Build passed (Build macOS tests) ✅

Pipeline on Agent
Hash: f7220b4438eed86cd475a7db87d5046c76c170f9 [PR build]

@vs-mobiletools-engineering-service2

This comment has been minimized.

Sign in to view
@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

🔥 [CI Build #f7220b4] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

5 tests crashed, 19 tests failed, 176 tests passed.

Failures

❌ linker tests (tvOS)

15 tests failed, 0 tests passed.

Failed tests

  • dont link/tvOS - simulator/Debug: LaunchTimedOut
  • dont link/tvOS - simulator/Release: LaunchTimedOut
  • dont link/tvOS - simulator/Debug (PrepareAssemblies, MonoVM, Dynamic Registrar): LaunchTimedOut
  • dont link/tvOS - simulator/Debug (PrepareAssemblies, MonoVM, Managed Static Registrar): LaunchTimedOut
  • dont link/tvOS - simulator/Release (PrepareAssemblies, MonoVM, Dynamic Registrar): LaunchTimedOut
  • dont link/tvOS - simulator/Release (PrepareAssemblies, MonoVM, Managed Static Registrar): LaunchTimedOut
  • link sdk/tvOS - simulator/Debug: LaunchTimedOut
  • link sdk/tvOS - simulator/Release: LaunchTimedOut
  • link all/tvOS - simulator/Debug: LaunchTimedOut
  • link all/tvOS - simulator/Release: LaunchTimedOut
  • link all/tvOS - simulator/Debug (don't bundle original resources): LaunchTimedOut
  • trimmode copy/tvOS - simulator/Debug: LaunchTimedOut
  • trimmode copy/tvOS - simulator/Release: LaunchTimedOut
  • trimmode link/tvOS - simulator/Debug: LaunchTimedOut
  • trimmode link/tvOS - simulator/Release: LaunchTimedOut

Html Report (VSDrops) Download

❌ monotouch tests (macOS)

3 tests failed, 21 tests passed.

Failed tests

  • monotouch-test/macOS/Release (trimmable static registrar): TimedOut (Execution timed out after 1200 seconds.
    Test run crashed)
  • monotouch-test/macOS/Release (NativeAOT, ARM64): TimedOut (Execution timed out after 1200 seconds.
    Test run crashed)
  • monotouch-test/macOS/Release (static registrar): Failed (Test run failed.
    Tests run: 3717 Passed: 3609 Inconclusive: 4 Failed: 1 Ignored: 107)

Html Report (VSDrops) Download

❌ windows tests

1 tests failed, 2 tests passed.

Failed tests

  • BGen tests/GeneratorTests.ErrorTests.BI1117_methodinternal(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.BMac_Smoke(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GHIssue9065_Sealed: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NoAsyncWarningCS0219(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug16036: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.INativeObjectsInBlocks(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.HyphenInName: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GeneratedAttributeOnPropertyAccessors: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.AvailabilityAttributes(tvOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Issue19612: Failed: CSC exit code

  • BGen tests/GeneratorTests.ErrorTests.GH6863_method(iOS): Failed: The error 'BI1072' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.XmlDocs(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1060(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BindAsNoRefParam(iOS): Failed: The error 'BI1080' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug40282: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.Bug57094Test(iOS): Failed: The error 'BI1014' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.BMac_With_Hyphen_In_Name(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GHIssue5444: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GH5416_method(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.SnippetAttributesOptimizable: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug24078: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.FieldEnumTests(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.WarnAsError(macOSMobile): Failed: The error 'BI1117' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.DesignatedInitializer(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ProtocolTests.Members(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NSCopyingNullability(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NoAvailabilityForAccessors(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1037(macOSMobile): Failed: The error 'BI1037' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug39614(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.BindAsTests: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.UnderlyingFieldType(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.StrongDictsNativeEnums: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.InternalDelegate: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NativeEnum(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ObsoletedOSPlatform(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1046(iOS): Failed: The error 'BI1046' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Issue3875("issue3875B.cs","BProtocol"): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1064(iOS): Failed: The error 'BI1064' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.GHIssue6626: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug18035(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug46292(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ProtocolTests.Members(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ProtocolBindProperty: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.IgnoreUnavailableProtocol: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NewerAvailabilityInInlinedProtocol(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.PropertyRedefinition: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GHIssue3869: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug27430: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1066(iOS): Failed: The error 'BI1066' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BI1040(macOSMobile): Failed: The error 'BI1040' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.NoWarn(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ProtocolDuplicateAbstract(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ReleaseAttribute(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1112_Bug37527_WrongProperty(iOS): Failed: The error 'BI1112' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BindAsNullableNSValueArrayError(iOS): Failed: The error 'BI1048' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.DelegateWithINativeObjectReturnType: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GH5416_setter(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.BackingFieldType(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GHIssue7304: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1062_NoAsyncMethodOutHandlerTest(iOS): Failed: The error 'BI1062' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.INativeObjectArraysInBlocks(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.PreviewAPIs(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.SimulatorAvailabilityAttributes(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.PreviewAPIs(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ForcedType: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1050_protocol(iOS): Failed: The error 'BI1050' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.XmlDocs(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.RefOutParameters: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1059(iOS): Failed: The error 'BI1084' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.AbstractTypeTest: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ArrayFromHandleBug: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BindAsNullableNSNumberArrayError(iOS): Failed: The error 'BI1048' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug36457: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.DisposeAttributeOptimizable: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.GH6863_property(iOS): Failed: The error 'BI1071' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BI1067_1070(iOS): Failed: The error 'BI1067' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug27428: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.AvailabilityAttributes(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug15283: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.AttributesFromInlinedProtocols(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1041(macOSMobile): Failed: The error 'BI1041' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.StrongDictionaryErrors(MacCatalyst): Failed: The error 'BI1121' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.SimulatorAvailabilityAttributes(tvOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug31788(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1117_classinternal(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NSApplicationPublicEnsureMethods(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.SmartEnumWithFramework(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Forum54078: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.PreviewAPIs(tvOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BindAsNoMultidimensionalArrays(iOS): Failed: The error 'BI1048' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BI1062_NoAsyncMethodRefHandlerTest(iOS): Failed: The error 'BI1062' was not found in the output.

  • BGen tests/GeneratorTests.BGen.ResponseFile(iOS): Failed: BGen failed with exit code 1: response file

  • BGen tests/GeneratorTests.ErrorTests.BI1049(iOS): Failed: The error 'BI1049' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.MissingExportOnProperty(iOS): Failed: The error 'BI1018' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.ErrorDomain(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.BackingFieldType(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.BothProtectedAndInternal(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.PropertyRedefinitionMac(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1039(macOSMobile): Failed: The error 'BI1039' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.DelegatesWithNullableReturnType(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.VirtualWrap(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug15307: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1038(macOSMobile): Failed: The error 'BI1038' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.PreviewAPIs(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.VSTS970507: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Issue3875("issue3875.cs","api0__Issue3875_AProtocol"): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.StackOverflow20696157: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1113_Bug37527_MissingProperty(iOS): Failed: The error 'BI1113' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.MissingExportOnProperty(macOSMobile): Failed: The error 'BI1018' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.SimulatorAvailabilityAttributes_NotEmittedForMacPlatforms(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Desk79124: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BindAsNullableNSStringArrayError(iOS): Failed: The error 'BI1048' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.GHIssue18645_DuplicatedFiled: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug42742: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug57531: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GHIssue5692: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1050_model(iOS): Failed: The error 'BI1050' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.NFloatType: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BindAsNoOutParam(iOS): Failed: The error 'BI1080' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug35176: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1077(iOS): Failed: The error 'BI1077' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BI1036(iOS): Failed: The error 'BI1036' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Issue3875("issue3875C.cs","api0__Issue3875_AProtocol"): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1062_NoAsyncMethodRefParameterTest(iOS): Failed: The error 'BI1062' was not found in the output.

  • BGen tests/GeneratorTests.ErrorTests.BI1062_NoAsyncMethodOutParameterTest(iOS): Failed: The error 'BI1062' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.DiamondProtocol: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1048(iOS): Failed: The error 'BI1048' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.EditorBrowsable(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug34042: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug27986: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.NoAsyncInternalWrapper(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug53076WithModel: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1042(iOS): Failed: The error 'BI1042' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.IsDirectBinding: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ProtocolWithBaseTypeButNoModel(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.DelegateParameterAttributes: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.SimulatorAvailabilityAttributes_NotEmittedForMacPlatforms(MacCatalyst): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.ErrorDomain_NoLibraryName(iOS): Failed: The error 'BI1087' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug17232: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.AvailabilityAttributes(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.NoWarn(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug57870: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GenericStrongDictionary: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.TypesInMultipleNamespaces(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1117(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GenericNSObjectParameter: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.DelegatesWithPointerTypes(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.GeneratedAttributeOnPropertyAccessors2: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug29493: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Desk63279: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.XmlDocs(tvOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.AvailabilityAttributes(macOSMobile): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.Bug43579: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.MultipleApiDefinitions2: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ClassNameCollision(iOS): Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.ReturnRelease: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.DynamicDependencyAttribute: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.CSharp10Syntax: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1065(iOS): Failed: The error 'BI1065' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.StrongDictSupportTemplatedDicts: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.BI1063_NoDoubleWrapTest(iOS): Failed: The error 'BI1063' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.Bug53076: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.ErrorTests.WarnAsError(iOS): Failed: The error 'BI1117' was not found in the output.

  • BGen tests/GeneratorTests.BGenTests.MultipleApiDefinitions1: Failed: BGen failed with exit code 1: build

  • BGen tests/GeneratorTests.BGenTests.XmlDocs(macOSMobile): Failed: BGen failed with exit code 1: build

Html Report (VSDrops) Download

❌ Tests on macOS Monterey (12) tests

🔥 Failed catastrophically on VSTS: test results - mac_monterey (no summary found).

Html Report (VSDrops) Download

❌ Tests on macOS Ventura (13) tests

🔥 Failed catastrophically on VSTS: test results - mac_ventura (no summary found).

Html Report (VSDrops) Download

❌ Tests on macOS Sonoma (14) tests

🔥 Failed catastrophically on VSTS: test results - mac_sonoma (no summary found).

Html Report (VSDrops) Download

❌ Tests on macOS Sequoia (15) tests

🔥 Failed catastrophically on VSTS: test results - mac_sequoia (no summary found).

Html Report (VSDrops) Download

❌ Tests on macOS Tahoe (26) tests

🔥 Failed catastrophically on VSTS: test results - mac_tahoe (no summary found).

Html Report (VSDrops) Download

Successes

✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 6 tests passed. Html Report (VSDrops) Download
✅ linker (iOS): All 15 tests passed. Html Report (VSDrops) Download
✅ linker (MacCatalyst): All 15 tests passed. Html Report (VSDrops) Download
✅ linker (macOS): All 21 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 21 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 24 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 21 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. [Html Report (VSDrops)](https://vsdrop.corp.microsoft.com/file/v1/macios/device-tests/20260619.34/14428075//simulator_testsxtro-1/;/t\n\nThe message from CI is too large for the GitHub comments. You can find the full results here.

@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

🔥 [PR Build #f7220b4] Build failed (Detect API changes) 🔥

Build failed for the job 'Detect API changes' (with job status 'Failed')

Pipeline on Agent
Hash: f7220b4438eed86cd475a7db87d5046c76c170f9 [PR build]

@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

🔥 Unable to find the contents for the comment: D:\a\1\s\change-detection\results\gh-comment.md does not exist :fire

Pipeline on Agent
Hash: f7220b4438eed86cd475a7db87d5046c76c170f9 [PR build]

@vs-mobiletools-engineering-service2

vs-mobiletools-engineering-service2 commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

🔥 [CI Build #f7220b4] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 1 tests failed, 219 tests passed.

Failures

❌ monotouch tests (macOS) [attempt 2]

1 tests failed, 23 tests passed.

Failed tests

  • monotouch-test/macOS/Release (trimmable static registrar, NativeAOT): Failed (Test run crashed (exit code: 139).
    Test run crashed)

Html Report (VSDrops) Download

Successes

✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 5 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ introspection: All 6 tests passed. Html Report (VSDrops) Download
✅ linker (iOS): All 15 tests passed. Html Report (VSDrops) Download
✅ linker (MacCatalyst): All 15 tests passed. Html Report (VSDrops) Download
✅ linker (macOS): All 21 tests passed. Html Report (VSDrops) Download
✅ linker (tvOS): All 15 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 21 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 24 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 21 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Monterey (12): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Ventura (13): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download

Linux Build Verification

✅ Linux build succeeded

Pipeline on Agent
Hash: f7220b4438eed86cd475a7db87d5046c76c170f9 [PR build]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

copilot

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rolfbjarne @vs-mobiletools-engineering-service2