Add MAUI iOS Inner Loop measurements for CI by davidnguyen-tech · Pull Request #5187 · dotnet/performance

davidnguyen-tech · 2026-04-03T14:32:43Z

Summary

Adds MAUI iOS Inner Loop performance measurements to CI, supporting both iOS simulators and physical devices.

What's included:

iOSInnerLoopParser.cs — Binlog parser extracting iOS build task/target timings
Startup.cs/Reporter.cs — Wiring and null-safety fix
ioshelper.py — iOS simulator and physical device management
runner.py — IOSINNERLOOP execution branch (build → deploy → measure → incremental)
Scenario scripts — pre.py, test.py, post.py, setup_helix.py
maui_scenarios_ios_innerloop.proj — Helix workitem definition (simulator + physical device)
Pipeline YAML — Job entries in sdk-perf-jobs.yml and routing in run_performance_job.py

Measurements:

First deploy: full build + install + launch timing via binlog parsing
Incremental deploy: source edit → rebuild + reinstall + relaunch timing

Targets:

iOS Simulator (iossimulator-arm64) on macOS Helix machines
Physical iPhone (ios-arm64) on Mac.iPhone.17.Perf queue

Based on:

Existing Android Inner Loop CI scenario (working reference)
Local iOS implementation from feature/measure-maui-ios-deploy branch

- Create iOSInnerLoopParser.cs: binlog parser for iOS inner loop build timings, extracting iOS-specific tasks (AOTCompile, Codesign, MTouch, etc.) and targets (_AOTCompile, _CodesignAppBundle, _CreateAppBundle, etc.) plus shared tasks (Csc, XamlC, LinkAssembliesNoShrink) - Wire into Startup.cs: add iOSInnerLoop to MetricType enum and map it to iOSInnerLoopParser in the parser switch expression - Fix Reporter.cs: guard against null/empty PERFLAB_BUILDTIMESTAMP to prevent ArgumentNullException on DateTime.Parse(null) when the env var is unset (falls back to DateTime.Now) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- const.py: Add IOSINNERLOOP constant and SCENARIO_NAMES mapping - ioshelper.py: New module with iOSHelper class for simulator and physical device management (boot, install, launch, terminate, uninstall, find bundle) - runner.py: Add iosinnerloop subparser, attribute assignment, and full execution branch (first build+deploy+launch, incremental loop with source toggling, binlog parsing, report aggregation, and Helix upload) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

pre.py: Install maui-ios workload, create MAUI template (no-restore for Helix), strip non-iOS TFMs with flexible regex, inject MSBuild properties (AllowMissingPrunePackageData, UseSharedCompilation), copy merged NuGet.config for Helix-side restore, create modified source files for incremental edit loop, check Xcode compatibility. test.py: Thin entrypoint that builds TestTraits and invokes Runner. post.py: Uninstall app from simulator, shut down dotnet build server, clean directories. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Create the Helix machine setup script for MAUI iOS inner loop measurements. This script runs on the macOS Helix machine before test.py and handles: 1. DOTNET_ROOT/PATH configuration from the correlation payload SDK 2. Xcode selection — auto-detects highest versioned Xcode_*.app, matching the pattern used by maui_scenarios_ios.proj PreparePayloadWorkItem 3. iOS simulator runtime validation via xcrun simctl 4. Simulator device boot with graceful already-booted handling 5. maui-ios workload install using rollback file from pre.py, with --ignore-failed-sources for dead NuGet feeds 6. NuGet package restore with --ignore-failed-sources /p:NuGetAudit=false 7. Spotlight indexing disabled via mdutil to prevent file-lock errors Follows the same structure as the Android inner loop setup_helix.py: context dict pattern, step-by-step functions, structured logging to HELIX_WORKITEM_UPLOAD_ROOT for post-mortem debugging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Define the Helix .proj file for iOS inner loop measurements, modeled after the Android inner loop .proj and existing maui_scenarios_ios.proj patterns. Key design decisions: - Build on Helix machine (not build agent) because deploy requires a connected device/simulator. PreparePayloadWorkItem only creates the template and modified source files via pre.py. - Workload packs stripped from correlation payload (RemoveDotnetFromCorrelation Staging) and reinstalled on Helix machine by setup_helix.py. - Environment variables set via shell 'export' in PreCommands (not in Python) because os.environ changes don't persist across process boundaries. - No XHarness — iOS inner loop uses xcrun simctl directly. - Simulator-only for now; physical device support (ios-arm64, code signing) is structured as a future TODO pending runner.py device support. - 01:30 timeout to accommodate iOS build + workload install + NuGet restore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- sdk-perf-jobs.yml: Add Mono Debug job entry for maui_scenarios_ios_innerloop on osx-x64-ios-arm64 (Mac.iPhone.17.Perf queue) - run-performance-job.yml: Add maui_scenarios_ios_innerloop to the in() check so --runtime-flavor is forwarded to run_performance_job.py - run_performance_job.py: Add maui_scenarios_ios_innerloop to get_run_configurations() (CodegenType, RuntimeType, BuildConfig) and to the binlog copy block for PreparePayloadWorkItems artifacts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- ioshelper.py: Add detect_connected_device() with auto-detection via xcrun devicectl (JSON + fallback text parsing), uninstall_app_physical, terminate_app_physical, close_physical_device, and cleanup() dispatch - runner.py: Add --device-type arg (simulator/device) to iosinnerloop subparser, auto-infer from RuntimeIdentifier, auto-detect device UDID, branch setup/install/startup/cleanup for physical vs simulator - setup_helix.py: Detect device type from IOS_RID env var, skip simulator boot for physical device, add detect_physical_device() for Helix - post.py: Handle physical device uninstall via devicectl with UDID auto-detection fallback - maui_scenarios_ios_innerloop.proj: Add physical device HelixWorkItem (conditioned on iOSRid=ios-arm64), pass IOS_RID to Pre/PostCommands, add --device-type arg to both simulator and device workitems Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…e sort Fix 1 (Major): Replace non-existent 'devicectl terminate --bundle-id' with '--terminate-existing' flag on launch command. Make terminate_app_physical() a no-op with documentation explaining why. Fix 2 (Medium): Write devicectl JSON output to temp file instead of /dev/stdout, which mixes human-readable table and JSON. Applied in both ioshelper.py and setup_helix.py with proper temp file cleanup. Fix 3 (Medium): Add standard UUID pattern (8-4-4-4-12) to UDID regex in _detect_device_fallback() for CoreDevice UUID format compatibility. Fix 4 (Medium): Normalize MAUI template to always use Pages/ subdirectory in pre.py. If template puts MainPage files at root, move them to Pages/. Add explanatory comment in .proj documenting the coupling. Fix 5 (Minor): Use tuple-of-ints version sort for Xcode selection instead of string comparison (fixes 16.10 < 16.2 ordering bug). Fix 6 (Minor): Make simulator boot failure fatal with sys.exit(1). Add dynamic fallback to latest available iPhone simulator before failing. Fix 7 (Nit): Add missing trailing newline to runner.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Replace deprecated tempfile.mktemp() with tempfile.mkstemp() in both ioshelper.py and setup_helix.py to avoid TOCTOU race condition. - Fix unreachable fallback in detect_connected_device(): when devicectl exits non-zero (e.g., older Xcode without --json-output), call _detect_device_fallback() instead of returning None immediately. - Guard against missing JSON report in runner.py IOSINNERLOOP branch: Startup.cs only writes reports when PERFLAB_INLAB=1, so local runs would crash with FileNotFoundError. Now degrades gracefully with empty counters and a warning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Temporarily disable all other scenario jobs to speed up CI iteration while validating the new MAUI iOS Inner Loop scenario. This change should be reverted before merging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Capture dotnet build output instead of crashing on CalledProcessError - Create traces/ directory before first build - Fix setup_helix.py to write output.log (matches .proj expectation) - Improve error handling for build failures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The dotnet build stdout/stderr wasn't appearing in Helix console logs, making it impossible to diagnose build failures. Explicitly capture and print build output through Python logging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Build 2943141 hit the 90-minute timeout. iOS first build with AOT compilation can take 30+ minutes, plus 3 incremental iterations. Increasing to 2.5 hours to allow full completion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Helix machines have Xcode 26.2 but the iOS SDK requires 26.3. The minor version difference shouldn't affect build correctness, so bypass the check with ValidateXcodeVersion=false. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Mac.iPhone.17.Perf queue uses Intel x64 machines which need iossimulator-x64, not iossimulator-arm64. Add architecture detection in setup_helix.py and update default RID in .proj. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The traces upload directory already exists from the first build, causing copytree() to fail on subsequent iterations. Clear it before each parsetraces() call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The -v:n flag was added to debug build errors but produces excessive file copy logs. Default verbosity shows errors/warnings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Add /p:MtouchLink=None to disable managed linker for Debug inner loop builds, avoiding MT0180 errors on machines without Xcode 26.3 - Add minimum Xcode version check in setup_helix.py for fast failure with clear diagnostics when machine has Xcode < 26.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove temporary ${{ if false }}: wrappers that disabled all jobs except iOS inner loop during iterative CI debugging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…loop - Separate install from simulator/device setup in ioshelper.py - Capture install time for first and incremental deploys in runner.py - Add "Install Time" counter to both perf reports - Add CoreCLR Debug job entry in pipeline YAML - Add device (ios-arm64) job entries for both Mono and CoreCLR - Wire iOSRid env var through to MSBuild for device builds - [TEMP] Disable non-iOS-inner-loop jobs for CI validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

When the dynamically-resolved manifest references SDK packs not yet propagated to NuGet feeds, fall back to installing without the rollback file. This avoids CI being blocked by transient feed propagation delays. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

When the rollback file references SDK packs not yet propagated to NuGet feeds, retry without the rollback file. Matches the fallback pattern already added to pre.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The simulator HelixWorkItem was unconditionally included, even when iOSRid=ios-arm64. This caused the simulator to receive device RID in _MSBuildArgs, producing ARM64 binaries that can't install on a simulator. Add Condition to exclude it from device jobs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…t.py Consolidate 12 duplicated simulator/device methods in ioshelper.py into a unified API (setup_device, install_app, measure_cold_startup, cleanup) that dispatches internally based on is_physical_device. Removes all if-is_physical dispatch branches from runner.py. Extract merge_build_deploy_and_startup and _make_counter to module-level helpers. Inline the incremental iteration loop (was a nested function with 10 parameters). Simplify post.py to reuse ioshelper instead of duplicating device detection. Extract inject_csproj_properties in pre.py. Net reduction: -232 lines across 4 files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ling Re-apply run_env_vars (including iOSRid) right before perf_send_to_helix() so the MSBuild .proj ItemGroup conditions can correctly exclude the simulator work item from device jobs. Add '|| exit $?' to setup_helix.py PreCommands so that when setup_helix exits non-zero (e.g., Xcode too old), the Helix shell stops instead of continuing to run test.py which would fail with a less clear error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Env var inheritance through msbuild.sh/tools.sh is unreliable for iOSRid. Add ios_rid field to PerfSendToHelixArgs and pass it as /p:iOSRid=<value> on the MSBuild command line so it reaches .proj evaluation deterministically. Also set it via set_environment_variables as belt-and-suspenders. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Mono_InnerLoop → Mono_InnerLoop_Simulator CoreCLR_InnerLoop → CoreCLR_InnerLoop_Simulator Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace simctl (simulator) and devicectl (device) install/launch commands with mlaunch to match the real Visual Studio F5 developer experience: - Simulator: --launchsim combines install + launch (install_app returns 0) - Device: --installdev for install, --launchdev for launch - Device cleanup: --uninstalldevbundleid replaces devicectl uninstall - Simulator cleanup: unchanged (simctl terminate + uninstall) - Added _resolve_mlaunch() to find mlaunch from iOS SDK packs Device detection (devicectl) and simulator management (simctl boot/ terminate/uninstall) remain unchanged. The install_app/measure_cold_startup API is preserved so runner.py requires no changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Instead of making install_app() a no-op for simulator, use mlaunch --installsim to get a separate install measurement. measure_cold_startup() still uses --launchsim for launch timing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…stall After installing the maui-ios workload, read _RecommendedXcodeVersion from the SDK's Versions.props and switch to the matching Xcode_*.app if the currently active Xcode doesn't match. This handles the case where Helix agents have a newer Xcode than the SDK requires. Falls back gracefully to the already-selected Xcode if no matching installation is found. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…vice path The device-job branch was running find_and_stage_signing_artifacts() first, then detect_physical_device() second. On Mac.iPhone.13.Perf the signing artifacts are missing, so the work item bails before logging anything about what physical hardware is (or isn't) attached. Result: when a human asks 'what iPhone is on that queue?' the only answer in the logs is 'we never checked'. Reorder so physical-device detection runs first (its own loud failure banner) and signing-artifact discovery runs after (its own loud failure banner). Both gates check independent infrastructure — a disconnected iPhone vs missing keychain materials — and surface independently. Now the device log always shows what hardware 'xcrun devicectl list devices' returned, regardless of signing state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

* Consolidate maui_scenarios_ios_innerloop.proj into maui_scenarios_ios.proj gated on $(_IsInnerLoop) derived from $(RunKind), matching the Android consolidation. Updates sdk-perf-jobs.yml inner-loop entries to reference the consolidated proj. * Drop the 23 "if false # [TEMP] Disabled for iOS inner loop CI validation" wrappers from eng/pipelines/sdk-perf-jobs.yml. Restores the existing private + scheduled SDK perf matrix that was inadvertently disabled. * Rename iOSInnerLoopParser to IOSInnerLoopParser (and MetricType enum member) to match PascalCase used by sibling parsers; update const.py metric mapping accordingly. * Revert unrelated PERFLAB_BUILDTIMESTAMP fallback change in Reporter.cs. * Drop the local-only run-local.sh script (not used by CI). * Make post.py cleanup best-effort: never sys.exit(1), to match Android. * Remove unused 'import hashlib' in shared/runner.py. * ioshelper._resolve_mlaunch: pick the iOS SDK pack by parsed version key instead of lexicographic sort so 26.10 ranks above 26.9. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…how sudo - Remove obsolete PublishReadyToRunStripDebugInfo workaround (dotnet/runtime#124604 flowed) - Split --msbuild-args on ';' instead of replacing with space + shlex (preserves values containing ';') - Restore edited source files in finally block so reused workspaces start clean - Run 'log show' with sudo to read the root-owned logarchive - Drop simulator fallback to ios-arm64 bundle search (avoid picking up stale device builds) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Comments suppressed due to low confidence (1)

src/scenarios/shared/runner.py:252

testtypes (from shared/testtraits.py) is used in the "Please specify a test type" error message, but it doesn't include the newly added iosinnerloop command. Consider adding const.IOSINNERLOOP to shared/testtraits.py:testtypes so users get an accurate list of available subcommands.

        iosinnerloopparser = subparsers.add_parser(const.IOSINNERLOOP,
                                                 description='measure first and incremental build+deploy time via binlogs (iOS)')
        iosinnerloopparser.add_argument('--csproj-path', help='Path to .csproj file to build', dest='csprojpath')
        iosinnerloopparser.add_argument('--edit-src', help='Modified source file paths, semicolon-separated', dest='editsrc')
        iosinnerloopparser.add_argument('--edit-dest', help='Destination paths for modified files, semicolon-separated', dest='editdest')
        iosinnerloopparser.add_argument('--framework', '-f', help='Target framework (e.g., net11.0-ios)', dest='framework')
        iosinnerloopparser.add_argument('--configuration', '-c', help='Build configuration', dest='configuration', default='Debug')
        iosinnerloopparser.add_argument('--msbuild-args', help='Additional MSBuild arguments', dest='msbuildargs', default='')
        iosinnerloopparser.add_argument('--bundle-id', help='iOS bundle identifier', dest='bundleid')
        iosinnerloopparser.add_argument('--device-id', help='iOS device ID (UDID for physical device, simulator ID or "booted" for simulator)', dest='deviceid', default='booted')
        iosinnerloopparser.add_argument('--device-type', choices=['simulator', 'device'], help='Target device type: simulator (default) or physical device. Auto-detected from RuntimeIdentifier if not set.', dest='devicetype', default=None)
        iosinnerloopparser.add_argument('--inner-loop-iterations', help='Number of incremental build+deploy+startup iterations (1+)', type=int, default=10, dest='innerloopiterations')
        self.add_common_arguments(iosinnerloopparser)

        args = parser.parse_args()

        if not args.testtype:
            getLogger().error("Please specify a test type: %s. Type test.py <test type> -- help for more type-specific subcommands" % testtypes)
            sys.exit(1)

Files reviewed: 19/19 changed files
Comments generated: 2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

iPhone13.Perf fails identically to iPhone17.Perf on main (AppleSdkSettings ..cctor ArgumentNullException at dotnet publish NetiOSDefault.csproj). Keep matrix aligned with main; pool fix is a dnceng infra issue, not in scope of this PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Files reviewed: 18/18 changed files
Comments generated: 2

+            if self.framework:
+                base_cmd.extend(['-f', self.framework])
+            if self.msbuildargs:
+                base_cmd.extend([arg for arg in self.msbuildargs.split(';') if arg])


+            m = re.search(r'Microsoft\.iOS\.Sdk\.([^/\\]+)', p)
+            if not m:
+                return ()
+            parts = re.split(r'[.\-+]', m.group(1))


…n space and semicolon - _resolve_mlaunch now sorts on the pack <version> directory (parent of tools/) rather than the pack-name suffix, so 26.10 ranks above 26.2. - The inner-loop proj builds _MSBuildArgs with a mix of ';' and space separators; splitting --msbuild-args on ';' alone left embedded spaces inside arguments and broke 'dotnet build'. Split on both. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… rm hang The previous code shared a single `ioshelper_startup.logarchive` path across all iterations and bracketed the `sudo log collect` call with `sudo rm -rf` calls to purge any stale archive (since `log collect` refuses to overwrite an existing --output path with exit 74). That works when sudo credentials are cached, but hangs indefinitely in non-TTY background contexts (e.g., campaigns started via detached caffeinate, nohup, or any agent runner) because the sudo password prompt has no terminal to read from. NOPASSWD is only practical to scope to `/usr/bin/log collect` (per the existing /etc/sudoers.d/log-collect convention); scoping it to /bin/rm is broader than most users want. Switch to a per-call unique logarchive path `ioshelper_startup_{pid}_{ms}.logarchive`. macOS's tmp reaper cleans /tmp on its own schedule, so per-iteration archives don't accumulate harmfully. `log collect` succeeds on the first try every time since the path never pre-exists. No sudo rm needed. Validated by a 4-side M365 campaign overnight: 11 cold-startup events per side across all 4 configs, zero hangs, with the orchestrator running fully detached from any user TTY.

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

+            ios_rid_env = os.environ.get('IOS_RID', '')
+            if ios_rid_env and 'RuntimeIdentifier=' in self.msbuildargs:
+                self.msbuildargs = re.sub(
+                    r'RuntimeIdentifier=\S+',


 import os
 import glob
 import re
+import shlex


+
+import os
+import subprocess
+import sys



  <PropertyGroup>
-    <IncludeXHarnessCli>true</IncludeXHarnessCli>
+    <_IsInnerLoop Condition="'$(RunKind)' == 'maui_scenarios_ios_innerloop'">true</_IsInnerLoop>
+    <IncludeXHarnessCli Condition="'$(_IsInnerLoop)' != 'true'">true</IncludeXHarnessCli>
  </PropertyGroup>


- Pulls macos-26 fix for Xcode 26.3 vs 26.5 mismatch - Pulls MAUI Android Inner Loop scaffolding (workload_name signature, shared mauisharedpython.install_latest_maui, AndroidInnerLoop parser enum, runner.py top-level ANDROIDINNERLOOP elif branch) - Adds osx-x64-ios-arm64-innerloop build machine (iPhone 13 + macos-26) for iOS Inner Loop scenarios; regular MAUI iOS jobs stay on osx-x64-ios-arm64 (iPhone 17) - Splices iOS Inner Loop pieces back into shared runner.py / Startup.cs alongside Android Inner Loop equivalents - Re-applies non-iOS job disables for WIP iteration speed (top-level runPublicJobs / runScheduledPrivateJobs + per-block scenario / NativeAOT / Android Inner Loop wrappers) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… iPhone13 The shared maui_scenarios_ios.proj uses $(RunKind) conditions to switch between regular and inner-loop ItemGroups, but RunKind was never set as an env var or pipeline variable on the build agent. As a result, the inner-loop jobs in build 2986285 ran the regular work items (Build Time, SOD, Device Startup) instead of the inner-loop ones. Fix: add run_kind to PerfSendToHelixArgs and emit it via ##vso[task.setvariable]/MSBuild property, symmetric with the iOSRid plumbing added in 5c092e4. Also revert the regular iOS jobs (osx-x64-ios-arm64) from Mac.iPhone.17.Perf back to Mac.iPhone.13.Perf. The iPhone17 queue has unhealthy macOS hosts as of 2026-05 and was producing Device Startup work item failures on Helix. Note: both iPhone13 and iPhone17 queues have estimated removal date 2026-06-01 and will need follow-up coordination with dnceng. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

+            ios_rid_env = os.environ.get('IOS_RID', '')
+            if ios_rid_env and 'RuntimeIdentifier=' in self.msbuildargs:
+                self.msbuildargs = re.sub(
+                    r'RuntimeIdentifier=\S+',


+    ios_rid = os.environ.get("IOS_RID", "iossimulator-x64")
+    is_physical_device = (ios_rid == "ios-arm64")
+
+    # Detect host architecture to select the correct simulator RID.
+    # Mac.iPhone.17.Perf queue uses Intel x64 machines which need
+    # iossimulator-x64, not iossimulator-arm64. Apple Silicon needs
+    # iossimulator-arm64. Physical device builds (ios-arm64) target the
+    # iPhone hardware, not the Mac, so skip architecture override.
+    if not is_physical_device:
+        host_arch = platform.machine()
+        if host_arch == "x86_64":
+            ios_rid = "iossimulator-x64"
+        elif host_arch == "arm64":
+            ios_rid = "iossimulator-arm64"
+        else:
+            log(f"WARNING: Unknown architecture '{host_arch}', "
+                f"keeping IOS_RID={ios_rid}", tee=True)
+        os.environ["IOS_RID"] = ios_rid
+        log(f"Host architecture: {host_arch}, using IOS_RID={ios_rid}", tee=True)
+
+    # The simulator device name env var is no longer used: we always create
+    # our own fresh simulator under this user's CoreSimulator namespace via
+    # create_and_boot_simulator(). Pre-existing devices on Helix machines
+    # may belong to other users and refuse to boot with permission errors.
+
+    # Framework and MSBuild args are passed as command-line arguments when
+    # available (from the .proj PreCommands), or fall back to env vars.
+    framework = sys.argv[1] if len(sys.argv) > 1 else os.environ.get("PERFLAB_Framework", "")
+    msbuild_args = sys.argv[2] if len(sys.argv) > 2 else ""
+
+    ctx = {
+        "framework": framework,
+        "msbuild_args": msbuild_args,
+        "workitem_root": workitem_root,


+            return (the same as what mlaunch internally reports), then
+            verify the process is still alive after a short stabilization
+            window so we don't report success for a crashed launch.
+          - Device:    mlaunch --launchdev (returns immediately with PID)


 ######################################################

- ${{ if parameters.runPublicJobs }}:
+- ${{ if false }}: # parameters.runPublicJobs — disabled for iOS Inner Loop WIP


run_performance_job.py runs 'dotnet msbuild /t:PreparePayloadWorkItems' on the build agent (line 1202) to bundle scenario payloads before SendToHelix. The proj's _IsInnerLoop condition depends on RunKind, but RunKind was never exported to os.environ for this MSBuild invocation — only the subsequent SendToHelix step received it (via the ##vso[task.setvariable] emitted by set_environment_variables). Without RunKind, inner-loop runs silently selected the regular PreparePayloadWorkItem items ('pre.py publish' for mauiios/netios/etc.) instead of the inner-loop items ('pre.py default' in the mauiiosinnerloop payload dir). As a result, rollback_maui.json was never created in the inner-loop payload dir, so the Helix work item's setup_helix.py fell back to installing the latest maui-ios workload manifest — which currently references missing Android cross-targeting packs (microsoft.android.sdk.darwin 36.1.69) and fails. Mirrors the iOSRid propagation pattern at lines 1178-1180. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

davidnguyen-tech and others added 30 commits April 3, 2026 15:54

Remove verbose build output from iOS Inner Loop

f62020a

The -v:n flag was added to debug build errors but produces excessive file copy logs. Default verbosity shows errors/warnings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Re-enable all pipeline jobs after iOS inner loop CI validation

68d6b4c

Remove temporary ${{ if false }}: wrappers that disabled all jobs except iOS inner loop during iterative CI debugging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add workload install fallback to setup_helix.py

995fe94

When the rollback file references SDK packs not yet propagated to NuGet feeds, retry without the rollback file. Matches the fallback pattern already added to pre.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename simulator job identifiers for clarity

edc2ccb

Mono_InnerLoop → Mono_InnerLoop_Simulator CoreCLR_InnerLoop → CoreCLR_InnerLoop_Simulator Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 6, 2026 12:46

Copilot started reviewing on behalf of davidnguyen-tech May 6, 2026 12:47 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

Comment thread src/scenarios/shared/ioshelper.py Outdated

Comment thread src/scenarios/shared/ioshelper.py

Comment thread eng/performance/maui_scenarios_ios_innerloop.proj Outdated

Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated

Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated

Copilot AI review requested due to automatic review settings May 20, 2026 13:39

Copilot started reviewing on behalf of kotlarmilos May 20, 2026 13:40 View session

kotlarmilos force-pushed the nguyendav/maui-ios-inner-loop branch from d8b0cc6 to c877a5f Compare May 20, 2026 13:42

Copilot AI reviewed May 20, 2026

View reviewed changes

Comment thread src/scenarios/shared/runner.py Outdated

Comment thread src/scenarios/mauiiosinnerloop/test.py Outdated

kotlarmilos and others added 2 commits May 20, 2026 16:01

iOS inner loop: fix End-to-End typo, drop unused intermediate_binlogs

56dd9f6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 20, 2026 14:45

Copilot started reviewing on behalf of kotlarmilos May 20, 2026 14:46 View session

Copilot AI reviewed May 20, 2026

View reviewed changes

kotlarmilos and others added 2 commits May 22, 2026 13:18

Copilot AI review requested due to automatic review settings May 28, 2026 13:27

Copilot started reviewing on behalf of davidnguyen-tech May 28, 2026 13:27 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings May 28, 2026 14:55

davidnguyen-tech force-pushed the nguyendav/maui-ios-inner-loop branch from 0935de6 to 3b66e55 Compare May 28, 2026 14:55

Copilot started reviewing on behalf of davidnguyen-tech May 28, 2026 14:56 View session

davidnguyen-tech review requested due to automatic review settings May 28, 2026 14:56

Copilot AI review requested due to automatic review settings May 28, 2026 16:42

Copilot started reviewing on behalf of davidnguyen-tech May 28, 2026 16:42 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MAUI iOS Inner Loop measurements for CI#5187

Add MAUI iOS Inner Loop measurements for CI#5187
davidnguyen-tech wants to merge 105 commits into
dotnet:mainfrom
davidnguyen-tech:nguyendav/maui-ios-inner-loop

davidnguyen-tech commented Apr 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

davidnguyen-tech commented Apr 3, 2026

Summary

What's included:

Measurements:

Targets:

Based on:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants