Suite merge script #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

ericaneininger merged 35 commits into MetOffice:main from james-bruten-mo:suite_merge_script

Jan 30, 2026

Collaborator

james-bruten-mo commented Jan 27, 2026 •

edited by ericaneininger

Loading

PR Summary

Sci/Tech Reviewer: @t00sa
Code Reviewer: @ericaneininger

Initial attempt at a script to merge different git branches for use in suites. The changes have been tested with the LFRic suites and with the lfric_apps rose-stem at vn3.0. If we think this is a good idea I'll also modify the UM suites to use this extraction method to maintain consistency.

These changes allow the dependencies.yaml file to specify multiple branches to merge together. This can be done by:

lfric_apps:
    - source: [email protected]:MetOffice/lfric_apps.git
      ref: 2025.12.1
    - source: [email protected]:james-bruten-mo/lfric_apps.git
      ref: test_merge_branch
    - source: [email protected]:james-bruten-mo/lfric_apps.git
      ref: test_branch2
    - source: [email protected]:james-bruten-mo/lfric_apps.git
      ref: conflic_apps

which will clone the first source and then merge each subsequent source into this. It's currently setup to ignore merge conflicts in rose-stem and dependencies.yaml as these files aren't important for running scientific suites.

Code Quality Checklist

I have performed a self-review of my own code
My code follows the project's style guidelines
Comments have been included that aid understanding and enhance the readability of the code
My changes generate no new warnings
All automated checks in the CI pipeline have completed successfully

Testing

This change has been tested appropriately (please describe)

LFRic standalone suites and lfric_apps rose-stem at vn3.0

Security Considerations

I have reviewed my changes for potential security issues
Sensitive data is properly handled (if applicable)
Authentication and authorisation are properly implemented (if applicable)

AI Assistance and Attribution

Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Sci/Tech Review

I understand this area of code and the changes being added
The proposed changes correspond to the pull request description
Documentation is sufficient (do documentation papers need updating)
Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

All dependencies have been resolved
Related Issues have been properly linked and addressed
Code quality standards have been met
Tests are adequate and have passed
Security considerations have been addressed
Performance impact is acceptable

james-bruten-mo added 16 commits

January 26, 2026 17:11


          copy changes from apps

d0326b8


          revert accidental changes

70073e6


          Merge branch 'main' into copy_overwrite_changes

c191b53

fix

538520b


          allow multiple sources

37ee629


          add merge_sources script

e324394


          make path

083fa1c

bug

a0645ca

bug

ef2c488


          fix merge

5a0c657


          Merge branch 'copy_overwrite_changes' into suite_merge_script

2c5eddc


          mirror fix

468c382


          setup merging of branches

66e5dcf


          ruff

da3860a


          remove remote

8a69af2


          working merge script

732a196

github-actions bot assigned james-bruten-mo


          add another error

2ad2543

james-bruten-mo marked this pull request as ready for review

January 28, 2026 08:54

github-actions bot requested a review from t00sa

January 28, 2026 08:54


          modify argument order

43c8f0e

t00sa reviewed

View reviewed changes

Contributor

t00sa left a comment

A handful of suggestions

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/merge_sources.py Outdated Show resolved Hide resolved

github_scripts/merge_sources.py Outdated Show resolved Hide resolved

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

james-bruten-mo and others added 8 commits

January 28, 2026 11:47


          Update github_scripts/get_git_sources.py

7a9a6c7

Co-authored-by: Sam Clarke-Green <[email protected]>


          Update github_scripts/merge_sources.py

60997bd

Co-authored-by: Sam Clarke-Green <[email protected]>


          Update github_scripts/merge_sources.py

9224a82

Co-authored-by: Sam Clarke-Green <[email protected]>


          Update github_scripts/rose_stem_extract_source.py

18f8610

Co-authored-by: Sam Clarke-Green <[email protected]>


          update to use logger

6bc07c6


          loggin gchanges

7fb1810


          loggin gchanges

cf12652


          logging changes

da8239d

james-bruten-mo added 4 commits

January 28, 2026 14:51


          logging changes


          logging changes

5835d36


          logging changes

6f5f623


          logging changes

682968a

james-bruten-mo requested a review from t00sa

January 28, 2026 15:05

t00sa approved these changes

View reviewed changes

Contributor

t00sa left a comment

Looks good. Just one suggestion about where to put the basicConfig() call

github_scripts/get_git_sources.py Show resolved Hide resolved

github-actions bot requested a review from cameronbateman-mo

January 28, 2026 20:56


          move basicconfig

248e6dc

github-actions bot requested a review from ericaneininger

January 29, 2026 08:29

Contributor

t00sa commented Jan 29, 2026

I've ticked the boxes I didn't tick yesterday and the update looks good too


          Merge branch 'main' into suite_merge_script

eda6097

ericaneininger requested changes

View reviewed changes

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/get_git_sources.py Show resolved Hide resolved

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/get_git_sources.py Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

github_scripts/merge_sources.py Outdated Show resolved Hide resolved

github_scripts/merge_sources.py Show resolved Hide resolved

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

james-bruten-mo and others added 3 commits

January 29, 2026 11:59


          Apply suggestion from @ericaneininger

76dd38b

Co-authored-by: Erica Neininger <[email protected]>


          cr changes

c4f4ee5


          Merge branch 'suite_merge_script' of github.com:james-bruten-mo/SimSy…

78bf16c

…s_Scripts into suite_merge_script

james-bruten-mo commented

View reviewed changes

Collaborator Author

james-bruten-mo left a comment

Thanks Erica, those changes are all done. For the opts == None comments, I've not modified by those comments, but have added a function that checks the structure of the dependencies dictionary looks roughly right. An example of it failing is at https://cylchub/services/cylc-review/view/james.bruten?&suite=u-dn674%2Frun3&no_fuzzy_time=0&path=log/job/1/extract/01/job.err

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/get_git_sources.py Outdated Show resolved Hide resolved

github_scripts/get_git_sources.py Show resolved Hide resolved

github_scripts/get_git_sources.py Show resolved Hide resolved

github_scripts/merge_sources.py Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Show resolved Hide resolved

github_scripts/rose_stem_extract_source.py Outdated

    
                      if not isinstance(opts, list):

                          opts = [opts]

                      for i, values in enumerate(opts):

Collaborator Author

james-bruten-mo Jan 29, 2026

Changed

github_scripts/rose_stem_extract_source.py Outdated Show resolved Hide resolved

james-bruten-mo mentioned this pull request

Copy overwrite changes #168

Closed

20 tasks

james-bruten-mo requested a review from ericaneininger

January 30, 2026 07:57

ericaneininger approved these changes

View reviewed changes

Contributor

ericaneininger left a comment

Thanks for taking the time to address all comments. Happy to approve.

ericaneininger merged commit 9297ded into MetOffice:main

26 checks passed

jennyhickson added this to the Spring 2026 milestone

r-sharp pushed a commit to r-sharp/SimSys_Scripts that referenced this pull request


          Suite merge script (MetOffice#169)

35fb697

* copy changes from apps

* revert accidental changes

* fix

* allow multiple sources

* add merge_sources script

* make path

* bug

* bug

* fix merge

* mirror fix

* setup merging of branches

* ruff

* remove remote

* working merge script

* add another error

* modify argument order

* Update github_scripts/get_git_sources.py

Co-authored-by: Sam Clarke-Green <[email protected]>

* Update github_scripts/merge_sources.py

Co-authored-by: Sam Clarke-Green <[email protected]>

* Update github_scripts/merge_sources.py

Co-authored-by: Sam Clarke-Green <[email protected]>

* Update github_scripts/rose_stem_extract_source.py

Co-authored-by: Sam Clarke-Green <[email protected]>

* update to use logger

* loggin gchanges

* loggin gchanges

* logging changes

* logging changes

* logging changes

* logging changes

* logging changes

* move basicconfig

* Apply suggestion from @ericaneininger

Co-authored-by: Erica Neininger <[email protected]>

* cr changes

---------

Co-authored-by: Sam Clarke-Green <[email protected]>
Co-authored-by: Erica Neininger <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet