Draft
Conversation
New scenario that clones dotnet/maui, builds dependencies, patches in PerfLabExporter via BenchmarkDotNet.Extensions, and runs the Core, XAML, and Graphics BDN benchmark suites on desktop (Windows). Key implementation details: - Sparse-clone of dotnet/maui with depth 1 (only needed directories) - Patches Directory.Build.props to disable non-desktop TFMs so all builds (including BDN internal builds) work without mobile workloads - Removes MAUI BDN PackageReference, injects BDN.Extensions ProjectRef - Patches Program.cs with ManualConfig + PerfLabExporter - Branch mapping: net11.0 to net11.0, net10.0 to net10.0 - All heavy work in test.py to keep correlation payload small - Pipeline entries disabled until validated in CI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move lifecycle logic (clone, patch, build, run, collect) from standalone test.py into shared/bdndesktop.py BDNDesktopHelper class, following the same pattern as AndroidInstrumentationHelper. Add BDNDESKTOP test type to const.py and runner.py subparser routing. test.py is now 16 lines, delegating to Runner(traits).run(). Tested locally: Graphics.Benchmarks ran successfully via runner path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move clone, branch mapping, sparse checkout, and dependency build into test.py (MAUI-specific orchestrator). BDNDesktopHelper now accepts repo_dir, benchmark_projects, and disable_props as constructor params, making it reusable for any repo's BDN desktop benchmarks. Remove bdndesktop routing from runner.py and const.py since test.py drives the helper directly with its own arg parsing. Tested locally: Graphics.Benchmarks ran successfully via refactored path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new MAUI Desktop BenchmarkDotNet (BDN) scenario to the repo, including a reusable shared helper to patch/build/run external desktop benchmark suites and (optionally) collect/upload PerfLabExporter results.
Changes:
- Introduces
BDNDesktopHelper(shared/bdndesktop.py) to patch TFMs, inject PerfLabExporter, build, run, and collect BDN results from external repos. - Adds a new
mauiDesktopBenchmarksscenario (pre/test/post scripts) plus a Helix project file to run MAUI desktop BDN suites. - Adds (currently disabled) pipeline job stubs for the new run kind in
sdk-perf-jobs.yml.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| src/scenarios/shared/runner.py | Whitespace-only change in argument parsing. |
| src/scenarios/shared/bdndesktop.py | New shared helper for patching external repos and running/collecting BDN desktop benchmarks. |
| src/scenarios/mauiDesktopBenchmarks/test.py | Clones dotnet/maui, patches props, builds dependencies, runs BDN suites via shared helper. |
| src/scenarios/mauiDesktopBenchmarks/pre.py | Minimal pre-command script (logging only). |
| src/scenarios/mauiDesktopBenchmarks/post.py | Cleans up MAUI repo checkout and combined report artifact. |
| eng/pipelines/sdk-perf-jobs.yml | Adds new MAUI desktop BDN job entries, currently gated by if false. |
| eng/performance/maui_desktop_benchmarks.proj | New Helix project definition for MAUI desktop BDN work item. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| for item_group in root.findall(f'{ns}ItemGroup'): | ||
| for pkg_ref in item_group.findall(f'{ns}PackageReference'): | ||
| include = pkg_ref.get('Include', '') | ||
| if include.startswith('BenchmarkDotNet'): |
Comment on lines
+206
to
+218
| patterns = [ | ||
| 'BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);', | ||
| 'BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args)', | ||
| 'BenchmarkSwitcher.FromAssembly (typeof (Program).Assembly).Run (args);', | ||
| 'BenchmarkSwitcher.FromAssembly (typeof (Program).Assembly).Run (args)', | ||
| ] | ||
|
|
||
| replaced = False | ||
| for pattern in patterns: | ||
| if pattern in content: | ||
| suffix = ';' if pattern.endswith(';') else '' | ||
| content = content.replace(pattern, new_run_call + suffix) | ||
| replaced = True |
Comment on lines
+344
to
+347
| basename = os.path.basename(report_file) | ||
| dest = os.path.join(upload_root, basename) | ||
| shutil.copy2(report_file, dest) | ||
| log.info(f' Copied {basename} to upload root') |
Comment on lines
+63
to
+65
| if os.path.exists(repo_dir): | ||
| shutil.rmtree(repo_dir) | ||
|
|
Comment on lines
+17
to
+20
| if os.path.exists(MAUI_REPO_DIR): | ||
| log.info(f'Removing cloned MAUI repo: {MAUI_REPO_DIR}') | ||
| shutil.rmtree(MAUI_REPO_DIR, ignore_errors=True) | ||
|
|
Comment on lines
+605
to
+620
| # MAUI Desktop BDN benchmarks (private) | ||
| - ${{ if false }}: | ||
| - template: /eng/pipelines/templates/build-machine-matrix.yml | ||
| parameters: | ||
| jobTemplate: /eng/pipelines/templates/run-scenarios-job.yml | ||
| buildMachines: | ||
| - win-x64-viper | ||
| isPublic: false | ||
| jobParameters: | ||
| runKind: maui_desktop_benchmarks | ||
| projectFileName: maui_desktop_benchmarks.proj | ||
| channels: | ||
| - main | ||
| ${{ each parameter in parameters.jobParameters }}: | ||
| ${{ parameter.key }}: ${{ parameter.value }} | ||
|
|
| self.runtimeseconds = args.runtimeseconds | ||
| self.closeToStartDelay = args.closeToStartDelay | ||
|
|
||
|
|
Comment on lines
+106
to
+109
| pattern = rf'(<{prop_name}\b[^>]*>)true(</{prop_name}>)' | ||
| content, count = re.subn(pattern, rf'\g<1>{new_value}\g<2>', content) | ||
| if count > 0: | ||
| log.info(f' {prop_name}: replaced {count} occurrence(s)') |
Comment on lines
+162
to
+169
| # Add ProjectReference to BDN.Extensions | ||
| item_group = ET.SubElement(root, f'{ns}ItemGroup') | ||
| item_group.set('Label', 'PerfLabInjected') | ||
| proj_ref = ET.SubElement(item_group, f'{ns}ProjectReference') | ||
| proj_ref.set('Include', bdn_ext_rel) | ||
|
|
||
| tree.write(csproj_path, xml_declaration=True, encoding='utf-8') | ||
| log.info(f' Added ProjectReference: {bdn_ext_rel}') |
Comment on lines
+308
to
+314
| report_pattern = os.path.join(self.repo_dir, '**', '*-perf-lab-report.json') | ||
| report_files = glob.glob(report_pattern, recursive=True) | ||
|
|
||
| if not report_files: | ||
| log.warning('No perf-lab-report.json files found. ' | ||
| 'PerfLabExporter may not have been active (PERFLAB_INLAB not set?).') | ||
| return |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.