[msbuild/tools] Move post-trimming custom trimmer steps to a post-ILLink MSBuild task.#25742
[msbuild/tools] Move post-trimming custom trimmer steps to a post-ILLink MSBuild task.#25742rolfbjarne wants to merge 20 commits into
Conversation
…embly in the AssemblyPreparerInfo.
There was a problem hiding this comment.
Pull request overview
This PR is a WIP step toward replacing custom ILLink linker steps by moving the “post-trimming” work into an MSBuild task (assembly preparer mode), enabling a trimmer-agnostic pipeline (ILLink and future NativeAOT scenarios).
Changes:
- Introduces an MSBuild-driven post-processing phase for prepared assemblies (
PostProcessAssemblies+_PostprocessAssembliestarget). - Refactors/extends assembly-preparer to load assemblies, run pre/post steps, and save results (new steps + expanded pipeline).
- Adapts several linker steps to run both in the traditional linker pipeline and in assembly-preparer mode; adds a Windows remote test for assembly preparer.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/linker/RemoveUserResourcesSubStep.cs | Updates the step to work in both linker and assembly-preparer pipelines. |
| tools/linker/MonoTouch.Tuner/ListExportedSymbols.cs | Adds ASSEMBLY_PREPARER support for the step. |
| tools/dotnet-linker/Steps/RegistrarStep.cs | Adjusts registrar filtering behavior for assembly-preparer mode. |
| tools/dotnet-linker/Steps/GenerateReferencesStep.cs | Switches to StringUtils.IsNullOrEmpty for consistency. |
| tools/dotnet-linker/Steps/ExtractBindingLibrariesStep.cs | Uses PathUtils.AbsoluteToRelative instead of Path.GetRelativePath. |
| tools/dotnet-linker/LinkerConfiguration.cs | Extends configuration for assembly-preparer mode (AssemblyInfos, etc). |
| tools/common/Application.cs | Adds an explicit IsPostProcessingAssemblies flag for assembly-preparer mode. |
| tools/assembly-preparer/SaveAssembliesStep.cs | New step to write assemblies after processing (and strip crossgen markers). |
| tools/assembly-preparer/LoadAssembliesStep.cs | New step to load assemblies + determine trimming action. |
| tools/assembly-preparer/ComputeMethodOverridesStep.cs | New step to precompute method overrides into annotations. |
| tools/assembly-preparer/CollectFieldsStep.cs | New step to collect exported field symbols (compat for InlineDlfcn). |
| tools/assembly-preparer/AssemblyPreparer.cs | Splits pipeline into Prepare vs PostProcess and runs step lists accordingly. |
| tools/assembly-preparer/assembly-preparer.csproj | Wires additional steps and dependencies into the assembly-preparer tool. |
| tests/dotnet/UnitTests/WindowsTest.cs | Adds a remote Windows test validating assembly-preparer behavior/artifacts. |
| msbuild/Xamarin.Shared/Xamarin.Shared.targets | Adds PostProcessAssemblies and introduces _PostprocessAssemblies target. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/PrepareAssemblies.cs | Adds a post-processing mode invocation path for AssemblyPreparer. |
| dotnet/targets/Xamarin.Shared.Sdk.targets | Avoids running existing custom trimmer steps when post-processing is enabled. |
| @@ -44,6 +54,7 @@ protected override void EndProcess () | |||
| base.EndProcess (); | |||
| } | |||
| <Target | ||
| Name="_PostprocessAssemblies" | ||
| Condition="'$(PrepareAssemblies)' == 'true'" | ||
| Inputs="@(_PreparedAssemblies)" | ||
| Outputs="$(_PrepareAssembliesStampFile)" | ||
| BeforeTargets="_LoadLinkerOutput" | ||
| > | ||
| <PrepareAssemblies | ||
| InputAssemblies="@(_PreparedAssemblies)" | ||
| MakeReproPath="$(_PostprocessAssembliesMakeReproPath)" | ||
| OptionsFile="$(_CustomLinkerOptionsFile)" | ||
| OriginalAssemblies="$(_AssembliesToPrepare)" | ||
| Postprocessing="true" | ||
| TargetFrameworkMoniker="$(_ComputedTargetFrameworkMoniker)" | ||
| > | ||
| <Output TaskParameter="OutputAssemblies" ItemName="_PostProcessedAssemblies" /> | ||
| </PrepareAssemblies> | ||
|
|
||
| <ItemGroup> | ||
| <ResolvedFileToPublish Remove="@(_PreparedAssemblies)" /> | ||
| <ResolvedFileToPublish Include="@(_PostProcessedAssemblies)" /> | ||
| <FileWrites Include="@(_PostProcessedAssemblies)" /> | ||
| </ItemGroup> | ||
| </Target> |
| foreach (var assembly in configuration.AssemblyInfos) { | ||
| var assemblyDefinition = assembly.Assembly; | ||
| if (assemblyDefinition is null) { | ||
| exceptions.Add (ErrorHelper.CreateError (99, $"Assembly definition is null for {assembly.InputPath}")); | ||
| return; | ||
| } |
| try { | ||
| assemblyDefinition.Write (assembly.OutputPath, writerParameters); | ||
| ModuleAttributes m = assemblyDefinition.MainModule.Attributes; | ||
| } catch (Exception e) { |
| @@ -0,0 +1,89 @@ | |||
| // Copilot: add license | |||
| @@ -0,0 +1,124 @@ | |||
| // Copilot: add license | |||
| @@ -0,0 +1,57 @@ | |||
| // Copilot: add license | |||
| @@ -0,0 +1,14 @@ | |||
| // Copilot: add license | |||
| using var memoryStream = new MemoryStream ((int) infoPlistEntry!.Length); | ||
| using var plistStream = infoPlistEntry.Open (); | ||
| plistStream.CopyTo (memoryStream); | ||
|
|
||
| var infoPlist = (PDictionary) PDictionary.FromStream (memoryStream)!; |
| public ITaskItem [] OriginalAssemblies { get; set; } = []; | ||
|
|
||
| public bool PostProcessing { get; set; } |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
🔥 [CI Build #8a3a11e] Test results 🔥Test results❌ Tests failed on VSTS: test results 0 tests crashed, 61 tests failed, 159 tests passed. Failures❌ dotnettests tests (iOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (MacCatalyst)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (macOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (Multiple platforms)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (tvOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ generator tests1 tests failed, 4 tests passed.Failed tests
Html Report (VSDrops) Download ❌ interdependent-binding-projects tests2 tests failed, 2 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (iOS)5 tests failed, 10 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (MacCatalyst)4 tests failed, 11 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (macOS)10 tests failed, 11 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (tvOS)4 tests failed, 11 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (iOS)21 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (MacCatalyst)1 tests failed, 23 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (macOS)2 tests failed, 22 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (tvOS)1 tests failed, 20 tests passed.Failed tests
Html Report (VSDrops) Download ❌ windows tests3 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ xcframework tests2 tests failed, 2 tests passed.Failed tests
Html Report (VSDrops) [Download](https://devdiv.visualstudio.com/DevDiv/_apis/build/builds/14417896/artif\n\nThe message from CI is too large for the GitHub comments. You can find the full results here. |
Five fixes for the assembly-preparer (prepare-assemblies) build path: 1. SaveAssembliesStep: Skip non-CIL assemblies (e.g. NativeLibrary.dll) instead of erroring out. LoadAssembliesStep already handles them by setting OutputPath = InputPath. 2. _PostprocessAssemblies target: Remove Inputs/Outputs attributes since _PrepareAssembliesStampFile was never defined, causing MSBuild to always skip the target. 3. ListExportedSymbols.TryProcessAssembly: Remove base.ProcessAssembly() call which caused infinite recursion through ConfigurationAwareStep's sealed ProcessAssembly → TryProcessAssembly cycle. 4. ListExportedSymbols.TryEndProcess: Guard base.EndProcess() call with #if !ASSEMBLY_PREPARER to prevent the same infinite recursion pattern. 5. FlushOutputForMSBuild: Create ItemsDirectory before writing .items files, since the linker-items directory doesn't exist in the assembly-preparer path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In the old linker path, InlineClassGetHandleStep is conditionally added to the ILLink custom steps only when InlineClassGetHandle is set and not 'disabled'. In the assembly-preparer, it was always included in the Prepare step list. When InlineClassGetHandle is empty (disabled), the step was creating P/Invoke wrappers (e.g. xamarin_Class_GetHandle_XamarinSwiftFunctions_Native) that have no native implementation when the registrar is dynamic, causing undefined symbol errors at native link time. Override ConditionToProcess() to skip the entire step (initialization, per-assembly processing, and end processing) when the mode is Disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move _PostprocessAssemblies target to run before _ComputeStripAssemblyIL (in addition to _LoadLinkerOutput) so that assembly modifications happen before AOT compilation, preventing MVID mismatches at runtime. Add PopulateApplicationAssembliesStep to PostProcess to populate Application.Assemblies, which is needed by ComputeAOTArguments and GatherFrameworksStep (equivalent to LoadNonSkippedAssembliesStep in the ILLink path). Fix Assembly.FullPath setter in ASSEMBLY_PREPARER builds: was throwing InvalidOperationException unconditionally, now correctly computes is_framework_assembly using Configuration.FrameworkAssemblies (same logic as the non-ASSEMBLY_PREPARER path). Fix ComputeAOTArguments to use asm.FullPath for the assembly path instead of IntermediateLinkDir + filename, so it references the correct location (prepared-assemblies/ instead of linked/). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three fixes to ensure AOT compilation uses the correct (postprocessed) assemblies: 1. SaveAssembliesStep: During PostProcess, copy unchanged (Copy/CopyUsed action) assemblies to the output directory instead of leaving OutputPath pointing to linked/. This ensures all assemblies are in the same directory, which is required by the AOT compiler (it copies them to aot-input/ and validates paths match). 2. PopulateApplicationAssembliesStep: Move after SaveAssembliesStep and use AssemblyInfos + OutputPath instead of TryProcessAssembly, so that Application.Assemblies gets the correct postprocessed paths (not the linked/ input paths). 3. AssemblyPreparer: Reorder steps accordingly. This fixes the runtime crash where AOT images were compiled from linked/ assemblies but the bundle contained postprocessed/ versions with different binary layouts (due to Cecil rewriting assemblies that had resources removed by RemoveUserResourcesSubStep). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
✅ [PR Build #723af88] Build passed (Detect API changes) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
✅ [PR Build #723af88] Build passed (Build packages) ✅Pipeline on Agent |
🔥 [PR Build #723af88] Build failed (Build macOS tests) 🔥Build failed for the job 'Build macOS tests' (with job status 'Failed') Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
🔥 [CI Build #723af88] Test results 🔥Test results❌ Tests failed on VSTS: test results 5 tests crashed, 35 tests failed, 160 tests passed. Failures❌ assembly-processing tests1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ cecil tests1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (iOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (MacCatalyst)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (macOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (Multiple platforms)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ dotnettests tests (tvOS)1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ generator tests3 tests failed, 2 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (iOS)2 tests failed, 13 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (MacCatalyst)3 tests failed, 12 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (macOS)9 tests failed, 12 tests passed.Failed tests
Html Report (VSDrops) Download ❌ linker tests (tvOS)2 tests failed, 13 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (iOS)1 tests failed, 20 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (MacCatalyst)1 tests failed, 23 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (macOS)2 tests failed, 22 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (tvOS)1 tests failed, 20 tests passed.Failed tests
Html Report (VSDrops) Download ❌ sharpie tests1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ windows tests2 tests failed, 1 tests passed.Failed tests
Html Report (VSDrops) Download ❌ xtro tests1 tests failed, 0 tests passed.Failed tests
Html Report (VSDrops) Download ❌ Tests on macOS Monterey (12) tests🔥 Failed catastrophically on VSTS: test results - mac_monterey (no summary found). Html Report (VSDrops) Download ❌ Tests on macOS Ventura (13) tests🔥 Failed catastrophically on VSTS: test results - mac_ventura (no summary found). Html Report (VSDrops) Download ❌ Tests on macOS Sonoma (14) tests🔥 Failed catastrophically on VSTS: test results - mac_sonoma (no summary found). Html Report (VSDrops) Download ❌ Tests on macOS Sequoia (15) tests🔥 Failed catastrophically on VSTS: test results - mac_sequoia (no summary found). Html Report (VSDrops) Download ❌ Tests on macOS Tahoe (26) tests🔥 Failed catastrophically on VSTS: test results - mac_tahoe (no summary found). Html Report (VSDrops) Download Successes✅ framework: All 2 tests passed. Html Report (VSDrops) Download macOS testsLinux Build VerificationPipeline on Agent |
WIP WIP WIP.
Contributes towards #17693.