[release/9.0] Fix intermediate artifact upload collisions for NativeAOT and Mono#125564
Open
lewing wants to merge 3 commits intodotnet:release/9.0from
Open
[release/9.0] Fix intermediate artifact upload collisions for NativeAOT and Mono#125564lewing wants to merge 3 commits intodotnet:release/9.0from
lewing wants to merge 3 commits intodotnet:release/9.0from
Conversation
Multiple platform jobs upload to the same IntermediateArtifacts subdirectory (NativeAOTRuntimePacks or MonoRuntimePacks), causing concurrent write conflicts when arch-independent packages like Microsoft.NETCore.App.Ref are uploaded simultaneously by different platform jobs. Fix by making each upload name unique per platform, matching the pattern already used by CoreCLR ($(osGroup)$(osSubgroup)_$(archType)): - NativeAOT: NativeAOTRuntimePacks_$(osGroup)$(osSubgroup)_$(archType) - Mono: MonoRuntimePacks_$(osGroup)$(osSubgroup)_$(archType) - Mono multithread: MonoRuntimePacks_multithread_$(osGroup)$(osSubgroup)_$(archType) - CrossAOT: MonoRuntimePacks_crossaot_$(osGroup)$(osSubgroup)_$(archType) - LLVM AOT: MonoRuntimePacks_llvmaot_$(osGroup)$(osSubgroup)_$(archType) Updated Workloads download patterns to use MonoRuntimePacks* wildcard to match all per-platform subdirectories. Fixes dotnet#125561 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the release/9.0 runtime-official pipeline to prevent intermittent Azure DevOps artifact upload collisions by making NativeAOT and Mono intermediate artifact subdirectories unique per platform (matching the existing CoreCLR approach).
Changes:
- Make NativeAOT intermediate artifact upload subdirectory unique per platform via
$(osGroup)$(osSubgroup)_$(archType). - Make Mono intermediate artifact upload subdirectory unique per platform, including distinct prefixes for multithread/crossaot/llvmaot variants.
- Update the Workloads job’s
DownloadPipelineArtifactpatterns to useMonoRuntimePacks*so it can find packages across the new per-platform subdirectories.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
3 tasks
steveisok
approved these changes
Mar 14, 2026
Member
steveisok
left a comment
steveisok
left a comment
There was a problem hiding this comment.
We get impacted by this problem enough to make it worth a try.
This was referenced Mar 14, 2026
lewing
changed the title
lewing
added a commit
that referenced
this pull request
Mar 15, 2026
… all macOS/MacCatalyst (#125568) ## Summary macOS 15+ returns `EHOSTUNREACH` instead of `EACCES` for UDP broadcast without `SO_BROADCAST` due to Apple's [Local Network privacy feature](https://developer.apple.com/forums/thread/765285). The test expects `SocketError.AccessDenied` but gets `SocketError.HostUnreachable`. The existing `ActiveIssue` (from #113313) only disabled on MacCatalyst x64. The failure affects all macOS 15+ platforms — confirmed on `osx.15.amd64` in both coreclr and Mono legs in [#125564](#125564). ## Change Broadens the `ActiveIssue` from: ```csharp [ActiveIssue("...", typeof(PlatformDetection), nameof(PlatformDetection.IsMacCatalyst), nameof(PlatformDetection.IsX64Process))] ``` to: ```csharp [ActiveIssue("...", TestPlatforms.OSX | TestPlatforms.MacCatalyst)] ``` @ManickaP requested this backport in #114450 (comment). Fixes #114450 on release/9.0. cc @liveans @kotlarmilos @ManickaP Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes intermittent Azure DevOps artifact upload failures in the release/9.0 official runtime pipeline by ensuring NativeAOT and Mono intermediate artifact uploads use per-platform unique subdirectory names, preventing cross-job blob write collisions for arch-independent packages.
Changes:
- Make NativeAOT intermediate artifact upload names unique per platform using
$(osGroup)$(osSubgroup)_$(archType). - Make Mono intermediate artifact upload names unique per platform (including
multithread,crossaot, andllvmaotvariants). - Update the Workloads job download patterns to use
MonoRuntimePacks*wildcards so it can consume all per-platform Mono upload directories.
You can also share your feedback on Copilot code review. Take the survey.
akoeplinger
reviewed
Mar 16, 2026
This was referenced Mar 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
dotnet-runtime-officialpipeline onrelease/9.0has intermittent "File upload failed even after retry" errors caused by multiple platform jobs uploading arch-independent packages (likeMicrosoft.NETCore.App.Ref.9.0.15.nupkg) to the same artifact path simultaneously.Root cause: NativeAOT (27 platforms) all upload to
name: NativeAOTRuntimePacksand Mono (7+ matrix expansions) all upload toname: MonoRuntimePacks— using a shared subdirectory within theIntermediateArtifactsartifact. When two jobs finish at the same time, they race to upload the same file, causing blob storage write conflicts.Compare with CoreCLR, which correctly uses
name: $(osGroup)$(osSubgroup)_$(archType)— unique per platform. ✅Fix
Make each upload name unique per platform (matching the CoreCLR pattern):
NativeAOTRuntimePacksNativeAOTRuntimePacks_$(osGroup)$(osSubgroup)_$(archType)MonoRuntimePacksMonoRuntimePacks_$(osGroup)$(osSubgroup)_$(archType)MonoRuntimePacksMonoRuntimePacks_multithread_$(osGroup)$(osSubgroup)_$(archType)MonoRuntimePacksMonoRuntimePacks_crossaot_$(osGroup)$(osSubgroup)_$(archType)MonoRuntimePacksMonoRuntimePacks_llvmaot_$(osGroup)$(osSubgroup)_$(archType)Updated the Workloads job download patterns from
IntermediateArtifacts/MonoRuntimePacks/Shipping/...toIntermediateArtifacts/MonoRuntimePacks*/Shipping/...so the wildcard matches all per-platform subdirectories.The
CopyFiles@2flatten step (*/Shipping/*.nupkg) still works because each platform subdirectory is still a single path segment.Notes
release/9.0—main/release/10.0have moved to VMR buildsperf-non-wasm-jobs.yml) use the same pattern but only have single-platform uploads (no collision risk), so no changes needed thereFixes #125561
Ref: dotnet/dnceng#1916